From robert.virding@REDACTED Wed Jun 1 00:14:38 2011 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 31 May 2011 22:14:38 +0000 (GMT) Subject: [erlang-questions] Erlang and Memory management In-Reply-To: <961326352.243381306879495091.JavaMail.root@zimbra> Message-ID: <1091703077.243401306880078397.JavaMail.root@zimbra> ----- "Kostis Sagonas" wrote: > Robert Virding wrote: > > I just want to point out that these rules are BEAM specific and not > Erlang specific. > > Well, this is not correct: the BEAM nowhere specifies that its > instruction set has to be implemented using a private heap > architecture. > In fact, we successfully used the BEAM instruction set to implement > > both a shared heap and a hybrid heap implementation. See: > > http://user.it.uu.se/~kostis/Papers/heap.pdf Yes, I know that the instruction set says nothing about process heaps. I was referring to the BEAM *IMPLEMENTATION* which is based on separate process heaps and copying messages. The efficiency guides mentioned relate to the implementation and are BEAM specific. > > So on an implementation with a single heap all data is shared. Now > there aren't currently many single heap implementations, to be exact > only one, erjang (Erlang on the JVM). :-) > > There have been more. We had an OTP system with a shared heap and the > hybrid heap system was part of OTP for quite a while. IMO, it's too > bad > that it was not maintained when Erlang/OTP was extended to support SMP > architectures. > > Also, the ETOS (Erlang to Scheme) system was based on a shared heap > architecture. I quite agree with you, I also think a shared heap architecture is the way to go and doing an SMP version would be very interesting. My only gripe with erjang is that it cannot yet use SMPs properly. But I assume they will fix that. A shared heap architecture could change I suppose my main gripe is really that it is not always clear what is language specific and what is implementation specific. Robert From jack@REDACTED Wed Jun 1 01:01:15 2011 From: jack@REDACTED (Jack Moffitt) Date: Tue, 31 May 2011 17:01:15 -0600 Subject: [erlang-questions] Erlang and Memory management In-Reply-To: <1091703077.243401306880078397.JavaMail.root@zimbra> References: <961326352.243381306879495091.JavaMail.root@zimbra> <1091703077.243401306880078397.JavaMail.root@zimbra> Message-ID: > I quite agree with you, I also think a shared heap architecture is the way to go and doing an SMP version would be very interesting. My only gripe with erjang is that it cannot yet use SMPs properly. But I assume they will fix that. A shared heap architecture could change Could you elaborate a bit on the reason why you'd prefer a shared heap? I was under the impression that individual heaps were one of the ways Erlang achieved soft real-time performance. Is CPU power enough improved (or garbage collectors) since Erlang made that decision that it's no longer worth it? jack. From kostis@REDACTED Wed Jun 1 01:17:13 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 01 Jun 2011 02:17:13 +0300 Subject: [erlang-questions] Erlang and Memory management In-Reply-To: <4DE53122.2020904@cs.ntua.gr> References: <1782517589.242981306865189704.JavaMail.root@zimbra> <4DE53122.2020904@cs.ntua.gr> Message-ID: <4DE576F9.8060300@cs.ntua.gr> Kostis Sagonas wrote: > Robert Virding wrote: >> ... >> So on an implementation with a single heap all data is shared. Now >> there aren't currently many single heap implementations, to be exact >> only one, erjang (Erlang on the JVM). :-) > > There have been more. We had an OTP system with a shared heap and the > hybrid heap system was part of OTP for quite a while. IMO, it's too bad > that it was not maintained when Erlang/OTP was extended to support SMP > architectures. Some clarification because the above was slightly ambiguous and I think it has been misunderstood. The "it's too bad that ..." sentence of mine was referring to the *hybrid* heap system, not the shared heap one. Kostis From vinodhg@REDACTED Wed Jun 1 05:23:04 2011 From: vinodhg@REDACTED (vinod hg) Date: Wed, 1 Jun 2011 08:53:04 +0530 Subject: [erlang-questions] bug in ssl? Message-ID: Hi, I am using mochiweb as a https web server for my application (Secured socket layer). I am new to security. One of the users reported that sending random data to the web server port makes beam.smp to consume 100% CPU indefinitely for a very long period. Step to reproduce 1. Start a mochiweb https server (can use mochiweb web storage app example provided with no other options) on a specific port (ex: 8443 ) 2. Run the command ?nc [IP address] 8443 < /dev/urandom? 3. The server rejects the connection (you may get ssl record error sometimes). The command returns to the shell 4. Repeat the above command for some time till the command doesn?t return back to shell. 5. Observe beam.smp taking 100% CPU and also memory getting increase. (It may reach system limit and crash) 6. If you stop the command (nc ^C), it returns to normal Observations It is reproducible in both Linux and windows. It happens for other https web server (tried with musultin) It does not happen for http server (should be in lib ssl). Tried giving different SSL options like {verify, verify_peer} with empty certificate as valid, but did not help. For other web servers written in other language this is not the behavior. Example lighttpd (php) nc just returns back to shell everytime. I debugged the issue to find that in other cases where the nc commands return immediately to shell the ssl connection does not succeed (behaves normally). But for the case where nc does not return back, the call is in an infinite loop ?next_tls_record in ssl_connection.erl? , thus making the CPU to take 100% and in memory increase. This can lead denial of service attack. Is this a bug and should I raise it in bug report forum. Any help will be appreciated. Thanks & Regards, Vinod -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Wed Jun 1 07:48:20 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 1 Jun 2011 07:48:20 +0200 Subject: [erlang-questions] Erlang and Memory management In-Reply-To: References: <1782517589.242981306865189704.JavaMail.root@zimbra> <4DE53122.2020904@cs.ntua.gr> Message-ID: Erlang has generational per process GC. This means that after many read operations where you are likely to create new small terms (e.g. reply tuple with value), the GC may kick in. Generational GC would copy all reachable terms from old heap to new one, before discarding old heap. This leads to double use of memory, which might be the reason of crashes. Best, Gleb On 31 May 2011 21:29, "Paul Meier" wrote: > I'm in a similar situation. I've got a few large tries, implemented > with gb_trees. I keep them in an orddict, which I'm using as the > State variable of a gb_server. For example: > > ------ > init() -> > %% State is an orddict of 3 giant Tries, pre-made and fetched from > the filesystem. > {ok, State}. > > handle_call({keyed_operation_on_large_data, Key}, _From, State) -> > GiantTrie = orddict:find(Key, State), > .... computations using GiantTrie.... > {reply, Result, State}. > > I'm finding that my process is crashing due to inability to mmap, and > the erl_crash.dump file tells me that the gen_server's supervisor (or, > on occasion, the error_logger) is the one with a prohibitively large > Stack+Heap. Also, I never modify the Tries or the orddict containing > them in calls to the gb_server: it's all read-only. > > Is storing the value in an orddict, or using regular terms as opposed > to binaries causing unnecessary copying? While _I_ know the data is > being used in a read-only fashion, is there a place where copying > might be occurring because the VM doesn't know this? > > I was hoping to play with binaries as a next attempt at fixing this > (the links in this thread suggest that large binaries are handled more > conservatively), but this thread appeared fortuitously, and maybe > someone can spot the very likely obvious error that I'm making. > > I'm grateful for your help, whether in this thread or the others I've > enjoyed reading ^_^ > > -Paul > > > On Tue, May 31, 2011 at 11:19 AM, Kostis Sagonas wrote: >> Robert Virding wrote: >>> >>> I just want to point out that these rules are BEAM specific and not Erlang >>> specific. >> >> Well, this is not correct: the BEAM nowhere specifies that its instruction >> set has to be implemented using a private heap architecture. In fact, we >> successfully used the BEAM instruction set to implement both a shared heap >> and a hybrid heap implementation. See: >> >> http://user.it.uu.se/~kostis/Papers/heap.pdf >> >>> So on an implementation with a single heap all data is shared. Now there >>> aren't currently many single heap implementations, to be exact only one, >>> erjang (Erlang on the JVM). :-) >> >> There have been more. We had an OTP system with a shared heap and the >> hybrid heap system was part of OTP for quite a while. IMO, it's too bad >> that it was not maintained when Erlang/OTP was extended to support SMP >> architectures. >> >> Also, the ETOS (Erlang to Scheme) system was based on a shared heap >> architecture. >> >> Kostis >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Wed Jun 1 09:09:08 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 1 Jun 2011 09:09:08 +0200 Subject: [erlang-questions] Erlang and Memory management In-Reply-To: References: <961326352.243381306879495091.JavaMail.root@zimbra> <1091703077.243401306880078397.JavaMail.root@zimbra> Message-ID: <599D588D-FBD4-49D4-BD0C-EF83E3726256@erlang-solutions.com> On 1 Jun 2011, at 01:01, Jack Moffitt wrote: >> I quite agree with you, I also think a shared heap architecture is the way to go and doing an SMP version would be very interesting. My only gripe with erjang is that it cannot yet use SMPs properly. But I assume they will fix that. A shared heap architecture could change > > Could you elaborate a bit on the reason why you'd prefer a shared > heap? I was under the impression that individual heaps were one of the > ways Erlang achieved soft real-time performance. Is CPU power enough > improved (or garbage collectors) since Erlang made that decision that > it's no longer worth it? Actually, one of the innovations of the Erlang team (Robert and Joe [1], I believe) was to realise that when no cyclical data structures are allowed, and data structures are immutable, GC is much helped by the fact that all pointers point from newer data to older data, and GC can easily be made re-entrant. Thus, incremental GC fits Erlang very well, and real-time performance need not be sacrificed. A problem with the old shared-heap implementation was that the incremental GC was never implemented (this, as it turned out, was good news for Scala [2]). A compromise was to introduce a hybrid-heap system, where only data that is actually shared between processes gets put on a shared heap. As it turns out, the hybrid-heap system ought to be a good compromise in the long term as well, since a copying GC has some advantages over (reference-counting) shared-heap GC. In particular, reference counting doesn't add much value for non-shared data, but it does increase memory usage. I am certain Kostis and Robert will correct me if I've uttered any falsehoods. :) BR, Ulf W [1] http://www.erlang.se/publications/memory1995.ps One Pass Real-Time Generational Mark-Sweep Garbage Collection. Joe Armstrong and Robert Virding. International Workshop on Memory Management. Kinross, Scotland, September 27-29, 1995. [2] Instead of joining Ericsson and implementing the shared-heap GC, Erik Stenmnan became project manager in Martin Odersky's Scala group and helped bring about Scala 1.0. Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From ulf.wiger@REDACTED Wed Jun 1 09:13:12 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 1 Jun 2011 09:13:12 +0200 Subject: [erlang-questions] Erlang and Memory management In-Reply-To: <599D588D-FBD4-49D4-BD0C-EF83E3726256@erlang-solutions.com> References: <961326352.243381306879495091.JavaMail.root@zimbra> <1091703077.243401306880078397.JavaMail.root@zimbra> <599D588D-FBD4-49D4-BD0C-EF83E3726256@erlang-solutions.com> Message-ID: <22F543A3-CC0D-4B9D-AEA1-D821972A8873@erlang-solutions.com> On 1 Jun 2011, at 09:09, Ulf Wiger wrote: > Actually, one of the innovations of the Erlang team (Robert and Joe [1], I believe) was to realise that when no cyclical data structures are allowed, and data structures are immutable, GC is much helped by the fact that all pointers point from newer data to older data, and GC can easily be made re-entrant. Thus, incremental GC fits Erlang very well, and real-time performance need not be sacrificed. In fact, the Erlang Processor (ECOMP) made use of this fact and ran garbage collection in parallel with program execution. In that way, no CPU cycles were sacrificed for garbage collection, which ought to be about as real-time as it gets. :) BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From tony.arcieri@REDACTED Wed Jun 1 09:20:54 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Wed, 1 Jun 2011 01:20:54 -0600 Subject: [erlang-questions] Erlang and Memory management In-Reply-To: References: <961326352.243381306879495091.JavaMail.root@zimbra> <1091703077.243401306880078397.JavaMail.root@zimbra> Message-ID: On Tue, May 31, 2011 at 5:01 PM, Jack Moffitt wrote: > Could you elaborate a bit on the reason why you'd prefer a shared > heap? I was under the impression that individual heaps were one of the > ways Erlang achieved soft real-time performance. Is CPU power enough > improved (or garbage collectors) since Erlang made that decision that > it's no longer worth it? Erlang has to copy terms whenever they're sent in a message, aside from the exceptions noted earlier regarding binaries. With a shared heap, sending messages could be done by reference instead of actually copying data between process heaps. There are realtime (and even hard realtime) garbage collectors in the Java world which use a shared heap. Azul systems with 768 cores and 500GB+ heaps managed to keep GC pauses in the 10-20ms range, albeit with hardware memory barriers. Perhaps commodity x64 chips will get them at some point in the future as the number of CPU cores continues to grow. Shared heaps can scale well and eliminate the need to copy data on message sends. -- Tony Arcieri -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Jun 1 09:45:28 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 1 Jun 2011 09:45:28 +0200 Subject: [erlang-questions] file systems events mac -> erlang Message-ID: I want write a backup program for my macbook Once I have backed up *everything* which takes a while I then want to backup "changed files." Rather than re-scanning my entire disk I'd like to be informed when a file changes. Is there an erlang interface to the "file system events API" in the mac? (or something like inotify) or a background daemon that writes the names of all changed files to a log. /Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From amphibian.ltd@REDACTED Wed Jun 1 09:51:27 2011 From: amphibian.ltd@REDACTED (Tim Carpenter) Date: Wed, 1 Jun 2011 08:51:27 +0100 Subject: [erlang-questions] file systems events mac -> erlang In-Reply-To: References: Message-ID: <31F58095-3253-4820-B731-45094ED982E6@gmail.com> Joe, I would think Spotlight would be the place to look. IIRC there should be access, as tools like Automator and scripts would logically need such info. Tim Sent from my iPhone On 1 Jun 2011, at 08:45, Joe Armstrong wrote: > I want write a backup program for my macbook > > Once I have backed up *everything* which takes a while > I then want to backup "changed files." Rather than re-scanning > my entire disk I'd like to be informed when a file changes. > > Is there an erlang interface to the "file system events API" in the mac? > (or something like inotify) or a background daemon that writes the names of all changed files > to a log. > > /Joe > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jeffm@REDACTED Wed Jun 1 09:54:07 2011 From: jeffm@REDACTED (jm) Date: Wed, 01 Jun 2011 17:54:07 +1000 Subject: [erlang-questions] file systems events mac -> erlang In-Reply-To: References: Message-ID: <4DE5F01F.1090103@ghostgun.com> It appears to be called "FSEvents" http://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/FSEvents_ProgGuide/Introduction/Introduction.html I don't know of an erlang interface for it. I did the original erlang-inotify, but am no longer maintaining it. It would be interesting to add support for max os x and that other platform. Jeff. On 1/06/11 5:45 PM, Joe Armstrong wrote: > I want write a backup program for my macbook > > Once I have backed up *everything* which takes a while > I then want to backup "changed files." Rather than re-scanning > my entire disk I'd like to be informed when a file changes. > > Is there an erlang interface to the "file system events API" in the mac? > (or something like inotify) or a background daemon that writes the > names of all changed files > to a log. > > /Joe > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Wed Jun 1 10:35:46 2011 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 1 Jun 2011 10:35:46 +0200 Subject: [erlang-questions] bug in ssl? In-Reply-To: References: Message-ID: Hi! Will the following patch make the problem go away? diff --git a/lib/ssl/src/ssl_record.erl b/lib/ssl/src/ssl_record.erl index f1c0073..ed57166 100644 --- a/lib/ssl/src/ssl_record.erl +++ b/lib/ssl/src/ssl_record.erl @@ -62,6 +62,8 @@ -compile(inline). +-define(INITIAL_BYTES, 5). + %%==================================================================== %% Internal application API %%==================================================================== @@ -368,8 +370,12 @@ get_tls_records_aux(<<1:1, Length0:15, _/binary>>,_Acc) ?ALERT_REC(?FATAL, ?RECORD_OVERFLOW); get_tls_records_aux(Data, Acc) -> - {lists:reverse(Acc), Data}. - + case size(Data) =< ?MAX_CIPHER_TEXT_LENGTH + ?INITIAL_BYTES of + true -> + {lists:reverse(Acc), Data}; + false -> + ?ALERT_REC(?FATAL, ?UNEXPECTED_MESSAGE) + end. %%-------------------------------------------------------------------- -spec protocol_version(tls_atom_version() | tls_version()) -> tls_version() | tls_atom_version(). Regards Ingela Erlang/OTP team - Ericsson AB 2011/6/1 vinod hg : > Hi, > > > > I am using mochiweb as a https web server for my application (Secured socket > layer). I am new to security. > > One of the users reported that sending random data to the web server port > makes beam.smp to consume 100% CPU indefinitely for a very long period. > > > > Step to reproduce > > 1.?????? Start a mochiweb https server (can use mochiweb web storage app > example provided with no other options) on a specific port (ex: 8443 ) > > 2.?????? Run the command ?nc? [IP address]? 8443 > 3.?????? The server rejects the connection (you may get ssl record error > sometimes). The command returns to the shell > > 4.?????? Repeat the above command for some time till the command doesn?t > return back to shell. > > 5.?????? Observe beam.smp taking 100% CPU and also? memory getting increase. > (It may reach system limit and crash) > > 6.?????? If you stop the command (nc ^C), it returns to normal > > > > Observations > > It is reproducible in both Linux and windows. > > It happens for other https web server (tried with musultin) > > It does not happen for http server (should be in lib ssl). > > Tried giving different SSL options like {verify, verify_peer} with empty > certificate as valid, but did not help. > > For other web servers written in other language this is not the behavior. > Example lighttpd (php) nc just returns back to shell everytime. > > > > > > I debugged the issue to find that in other cases where the nc commands > return immediately to shell the ssl connection does not succeed (behaves > normally). > > But for the case where nc does not return back, the call is in an infinite > loop ?next_tls_record in ssl_connection.erl? , thus making the CPU to take > 100% and in memory increase. > > > > This can lead denial of service attack. Is this a bug and should I raise it > in bug report forum. > > > > Any help will be appreciated. > > Thanks & Regards, > > Vinod > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From roberto@REDACTED Wed Jun 1 13:04:17 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 1 Jun 2011 13:04:17 +0200 Subject: [erlang-questions] garbage collection: when? Message-ID: Dear list, afaik garbage collection in erlang is per process, but if a process gets large, it is automatically switched over to a generational scheme. What is unclear to me is when the collection happens. Let me illustrate with a very simple example. start() -> Var = [{one, 1}, {two, 2}, ...,{thousand, 1000}], loop(Var). loop(Var) -> ... other_stuff(Var). other_stuff(Var) -> ... NewVar = lists:keyreplace(one, 1, Var, {one, "one"}), do_some_other_stuff(NewVar). do_some_other_stuff(Var) -> ..., loop(Var). My question is: when will the original list be garbage collected? Only when this whole process exits? When we go back to loop/1? What if Var was quite big [or you had many of these processes] and you wanted to optimized memory management? Any input welcome.. r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinodhg@REDACTED Wed Jun 1 13:46:15 2011 From: vinodhg@REDACTED (vinod hg) Date: Wed, 1 Jun 2011 17:16:15 +0530 Subject: [erlang-questions] bug in ssl? In-Reply-To: References: Message-ID: Thanks Ingela. I tried with your patch and it is working fine. Will run it whole day and let you know. Will this be part of official release? On Wed, Jun 1, 2011 at 2:05 PM, Ingela Andin wrote: > Hi! > > Will the following patch make the problem go away? > > diff --git a/lib/ssl/src/ssl_record.erl b/lib/ssl/src/ssl_record.erl > index f1c0073..ed57166 100644 > --- a/lib/ssl/src/ssl_record.erl > +++ b/lib/ssl/src/ssl_record.erl > @@ -62,6 +62,8 @@ > > -compile(inline). > > +-define(INITIAL_BYTES, 5). > + > %%==================================================================== > %% Internal application API > %%==================================================================== > @@ -368,8 +370,12 @@ get_tls_records_aux(<<1:1, Length0:15, > _/binary>>,_Acc) > ?ALERT_REC(?FATAL, ?RECORD_OVERFLOW); > > get_tls_records_aux(Data, Acc) -> > - {lists:reverse(Acc), Data}. > - > + case size(Data) =< ?MAX_CIPHER_TEXT_LENGTH + ?INITIAL_BYTES of > + true -> > + {lists:reverse(Acc), Data}; > + false -> > + ?ALERT_REC(?FATAL, ?UNEXPECTED_MESSAGE) > + end. > %%-------------------------------------------------------------------- > -spec protocol_version(tls_atom_version() | tls_version()) -> > tls_version() | tls_atom_version(). > > > Regards Ingela Erlang/OTP team - Ericsson AB > > > 2011/6/1 vinod hg : > > Hi, > > > > > > > > I am using mochiweb as a https web server for my application (Secured > socket > > layer). I am new to security. > > > > One of the users reported that sending random data to the web server port > > makes beam.smp to consume 100% CPU indefinitely for a very long period. > > > > > > > > Step to reproduce > > > > 1. Start a mochiweb https server (can use mochiweb web storage app > > example provided with no other options) on a specific port (ex: 8443 ) > > > > 2. Run the command ?nc [IP address] 8443 < /dev/urandom? > > > > 3. The server rejects the connection (you may get ssl record error > > sometimes). The command returns to the shell > > > > 4. Repeat the above command for some time till the command doesn?t > > return back to shell. > > > > 5. Observe beam.smp taking 100% CPU and also memory getting > increase. > > (It may reach system limit and crash) > > > > 6. If you stop the command (nc ^C), it returns to normal > > > > > > > > Observations > > > > It is reproducible in both Linux and windows. > > > > It happens for other https web server (tried with musultin) > > > > It does not happen for http server (should be in lib ssl). > > > > Tried giving different SSL options like {verify, verify_peer} with empty > > certificate as valid, but did not help. > > > > For other web servers written in other language this is not the behavior. > > Example lighttpd (php) nc just returns back to shell everytime. > > > > > > > > > > > > I debugged the issue to find that in other cases where the nc commands > > return immediately to shell the ssl connection does not succeed (behaves > > normally). > > > > But for the case where nc does not return back, the call is in an > infinite > > loop ?next_tls_record in ssl_connection.erl? , thus making the CPU to > take > > 100% and in memory increase. > > > > > > > > This can lead denial of service attack. Is this a bug and should I raise > it > > in bug report forum. > > > > > > > > Any help will be appreciated. > > > > Thanks & Regards, > > > > Vinod > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- Thanks & Regards, Vinod H. G. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Wed Jun 1 14:00:18 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 01 Jun 2011 14:00:18 +0200 Subject: [erlang-questions] garbage collection: when? In-Reply-To: References: Message-ID: <4DE629D2.4090100@gmail.com> On 06/01/2011 01:04 PM, Roberto Ostinelli wrote: > afaik garbage collection in erlang is per process, but if a process gets > large, it is automatically switched over to a generational scheme. No such magic. Each process (in the standard runtime system) has its own heap, which is individually garbage collected. The current implementation is a copying, generational collector. This means that each process heap is individually divided into generations (although when a process is new, there is only one generation). Newer generations get collected more often than older ones. > What is unclear to me is when the collection happens. Let me illustrate with > a very simple example. > > > start() -> > Var = [{one, 1}, {two, 2}, ...,{thousand, 1000}], > loop(Var). > > loop(Var) -> > ... > other_stuff(Var). > > other_stuff(Var) -> > ... > NewVar = lists:keyreplace(one, 1, Var, {one, "one"}), > do_some_other_stuff(NewVar). > > do_some_other_stuff(Var) -> > ..., > loop(Var). > > > My question is: when will the original list be garbage collected? Only > when this whole process exits? When we go back to loop/1? What if Var > was quite big [or you had many of these processes] and you wanted to > optimized memory management? Garbage collection is not triggered by any particular event (except an explicit call to garbage_collect()), but rather, when the code tries to do something that requires more memory, e.g., to create a tuple or cons cell, than what is currently easily available on the heap. It then calls the garbage collector to try to get some more free space from the newest generation - this moves the used memory to one end and all the free memory to the other end. If this creates enough contiguous space, the code can continue with the allocation. Otherwise, the system will try to garbage collect the next older generation, and so on. If all generations have been garbage collected and there's still not enough memory for the allocation, the Erlang runtime system will enlarge the process' heap (by allocating more memory from the operating system). Thus, how often garbage collection is triggered depends on how quickly you create tuples and other data structures, and the size of the process heap depends on whether or not it allocates new data faster than it releases old data. If it releases data at the same speed or faster, then it will stay at the same size (or even shrink), because garbage collection will always be able to reclaim enough space from the existing heap. In your example above, the original list has no more references to it after the call to lists:keyreplace(), so it might get collected at that point, or at any later point, depending on whether your program needs to allocate more data structures and how much space is currently free on the process heap. A process that does not try to allocate more data does not waste time doing garbage collection either. /Richard From jesper.louis.andersen@REDACTED Wed Jun 1 14:09:58 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 1 Jun 2011 14:09:58 +0200 Subject: [erlang-questions] garbage collection: when? In-Reply-To: References: Message-ID: On Wed, Jun 1, 2011 at 13:04, Roberto Ostinelli wrote: > > start() -> > ??? Var = [{one, 1}, {two, 2}, ...,{thousand, 1000}], > ??? loop(Var). So, initially, Var is a reference to a list [{one, ...}, ...] > > loop(Var) -> > ??? ... > ??? other_stuff(Var). This will just pass the reference of Var to other_stuff in the function call. > other_stuff(Var) -> > ??? ... > ??? NewVar = lists:keyreplace(one, 1, Var, {one, "one"}), Ok, so the list is: Var = [{one, 1} | RestOfList], the keyreplace will create a new list: NewVar = [{one, "one"} | RestOfList] and with the tail RestOfList shared between those two because you did not alter anything in that part and due to how keyreplace works. > ??? do_some_other_stuff(NewVar). Now we pass NewVar > do_some_other_stuff(Var) -> > ??? ..., > ??? loop(Var). Here, Var is really the same thing as the NewVar reference. > My question is: when will the original list be garbage collected? Only when > this whole process exits? When we go back to loop/1? What if Var was quite > big [or you had many of these processes] and you wanted to optimized memory > management? Notice that the cell with the original head containing {one, 1} is now dead. Hence it will be GC'ed when the next garbage collection runs. It will run as soon as the process has allocated enough data to force the collection. If you have knowledge that you just gave back a lot of data you can hint the system to do the collection. Etorrent has an example in the file system processes: https://github.com/jlouis/etorrent/blob/master/apps/etorrent/src/etorrent_io_file.erl Notice we set {fullsweep_after, 0} as a spawn opt and a garbage_collect() is to be found in a timeout. The basic idea is that the process is rather long-lived, has a very small heap, and shouldn't be keeping data around see the 'erlang' module for the details. The majority of the original list, namely RestOfList will never be collected as it is still live (Its tail was shared with "NewVar" which is now live). To optimize memory management you must: * Use data structures and primitives that use less memory. Lists are notoriously good at consuming memory :) * Use the halfword emulator. A lot of stuff in Erlang happen to be pointers. Squashing them in size help a lot. * Use ETS to share data common to your processes. * Note that large binaries are shared. -- J. From michael.eugene.turner@REDACTED Wed Jun 1 15:49:25 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 1 Jun 2011 22:49:25 +0900 Subject: [erlang-questions] Erlang and Memory management In-Reply-To: References: <961326352.243381306879495091.JavaMail.root@zimbra> <1091703077.243401306880078397.JavaMail.root@zimbra> Message-ID: "Shared heaps can scale well and eliminate the need to copy data on message sends." The advantages seem obvious, but ... with cache-per-core, there will be copying anyway: into the core's data cache. Immutable data is nice, because you don't have to worry so much cache coherence. But how many CPU architectures are mutability-aware, at least at the right level? Copying a data structure is also an opportunity to construct a "fresh" data structure that's relatively unfragmented. Lower fragmentation of data structures would make cache fetches more efficient, and cache hits hits more likely, for the receiver. Probably most Erlang code would faster with a shared heap, but some code might slow down -- maybe by a lot. Putting certain smarts in Send might help: e.g., if the receiving process isn't ready to run, copy; if it is ready, just pass the pointer to shared memory. I'm sure lots more strategies might be devised. And I'm about equally sure they'll all have their own limits and pitfalls. -michael turner On Wed, Jun 1, 2011 at 4:20 PM, Tony Arcieri wrote: > On Tue, May 31, 2011 at 5:01 PM, Jack Moffitt wrote: > >> Could you elaborate a bit on the reason why you'd prefer a shared >> heap? I was under the impression that individual heaps were one of the >> ways Erlang achieved soft real-time performance. Is CPU power enough >> improved (or garbage collectors) since Erlang made that decision that >> it's no longer worth it? > > > Erlang has to copy terms whenever they're sent in a message, aside from the > exceptions noted earlier regarding binaries. With a shared heap, sending > messages could be done by reference instead of actually copying data between > process heaps. > > There are realtime (and even hard realtime) garbage collectors in the Java > world which use a shared heap. Azul systems with 768 cores and 500GB+ heaps > managed to keep GC pauses in the 10-20ms range, albeit with hardware memory > barriers. Perhaps commodity x64 chips will get them at some point in the > future as the number of CPU cores continues to grow. > > Shared heaps can scale well and eliminate the need to copy data on message > sends. > > -- > Tony Arcieri > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Jun 1 16:19:51 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 1 Jun 2011 16:19:51 +0200 Subject: [erlang-questions] Erlang and Memory management In-Reply-To: References: <961326352.243381306879495091.JavaMail.root@zimbra> <1091703077.243401306880078397.JavaMail.root@zimbra> Message-ID: On Wed, Jun 1, 2011 at 15:49, Michael Turner wrote: > The advantages seem obvious, but ... with cache-per-core, there will be > copying anyway: into the core's data cache. Immutable data is nice, because > you don't have to worry so much cache coherence. ?But how many CPU > architectures are mutability-aware, at least at the right level? Also, you need a copy when you go to another distributed machine anyway. So suddenly locality will play a role. From a point of performance, locality matter. > Copying a data structure is also an opportunity to construct a "fresh" data > structure that's relatively unfragmented. Lower fragmentation of data > structures would make cache fetches more efficient, and cache hits hits more > likely, for the receiver. Garbage Collection will usually unfragment data anyway. But the process heap acts as a crude region-based memory manager: When a process dies, we can *instantly* reset its memory area and give it back. And that kind of cleanup is really effective. This gives us a region of memory of which we have some control, due to the lifetime of the process. In a shared heap, you loose that control and have to wait until the next GC for the memory to be reclaimed. The fact that you can tune the GC of each process rather than for the whole VM is also a pretty good thing. -- J. From t.greenwoodgeer@REDACTED Wed Jun 1 17:00:24 2011 From: t.greenwoodgeer@REDACTED (Todd Greenwood) Date: Wed, 01 Jun 2011 08:00:24 -0700 Subject: [erlang-questions] file systems events mac -> erlang In-Reply-To: <4DE5F01F.1090103@ghostgun.com> References: <4DE5F01F.1090103@ghostgun.com> Message-ID: <4DE65408.4080303@gmail.com> Another approach would be to use the mac replacement for cron, called launchd. Your script can be launched by events such as cron-like calendar intervals (StartCalendarInterval), file changes (WatchPaths), mounts added/removed (StartOnMount), network stuff (Sockets), etc: http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man5/launchd.plist.5.html Here's a tutorial for backing up w/ launchd: http://www.macresearch.org/tutorial_backups_with_launchd -Todd On 6/1/11 12:54 AM, jm wrote: > It appears to be called "FSEvents" > > http://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/FSEvents_ProgGuide/Introduction/Introduction.html > > I don't know of an erlang interface for it. I did the original > erlang-inotify, but am no longer maintaining it. It would be interesting > to add support for max os x and that other platform. > > Jeff. > > On 1/06/11 5:45 PM, Joe Armstrong wrote: >> I want write a backup program for my macbook >> >> Once I have backed up *everything* which takes a while >> I then want to backup "changed files." Rather than re-scanning >> my entire disk I'd like to be informed when a file changes. >> >> Is there an erlang interface to the "file system events API" in the mac? >> (or something like inotify) or a background daemon that writes the >> names of all changed files >> to a log. >> >> /Joe >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ingela@REDACTED Wed Jun 1 18:48:55 2011 From: ingela@REDACTED (Ingela Andin) Date: Wed, 1 Jun 2011 18:48:55 +0200 Subject: [erlang-questions] bug in ssl? In-Reply-To: References: Message-ID: Hi! Well alas it just missed the R14B03 ?release, ?but it will of course be part of the next release. It will also be on github next week. Regards Ingela Erlang/OTP team - Ericsson AB > 2011/6/1 vinod hg : >> Thanks Ingela. I tried with your patch and it is working fine. Will run it >> whole day and let you know. Will this be part of official release? >> >> On Wed, Jun 1, 2011 at 2:05 PM, Ingela Andin wrote: >>> >>> Hi! >>> >>> Will the following patch make the problem go away? >>> >>> diff --git a/lib/ssl/src/ssl_record.erl b/lib/ssl/src/ssl_record.erl >>> index f1c0073..ed57166 100644 >>> --- a/lib/ssl/src/ssl_record.erl >>> +++ b/lib/ssl/src/ssl_record.erl >>> @@ -62,6 +62,8 @@ >>> >>> ?-compile(inline). >>> >>> +-define(INITIAL_BYTES, 5). >>> + >>> ?%%==================================================================== >>> ?%% Internal application API >>> ?%%==================================================================== >>> @@ -368,8 +370,12 @@ get_tls_records_aux(<<1:1, Length0:15, >>> _/binary>>,_Acc) >>> ? ? ?ALERT_REC(?FATAL, ?RECORD_OVERFLOW); >>> >>> ?get_tls_records_aux(Data, Acc) -> >>> - ? ?{lists:reverse(Acc), Data}. >>> - >>> + ? ?case size(Data) =< ?MAX_CIPHER_TEXT_LENGTH + ?INITIAL_BYTES of >>> + ? ? ? true -> >>> + ? ? ? ? ? {lists:reverse(Acc), Data}; >>> + ? ? ? false -> >>> + ? ? ? ? ? ?ALERT_REC(?FATAL, ?UNEXPECTED_MESSAGE) >>> + ? ? ? end. >>> ?%%-------------------------------------------------------------------- >>> ?-spec protocol_version(tls_atom_version() | tls_version()) -> >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?tls_version() | tls_atom_version(). >>> >>> >>> Regards Ingela Erlang/OTP team - Ericsson AB >>> >>> >>> 2011/6/1 vinod hg : >>> > Hi, >>> > >>> > >>> > >>> > I am using mochiweb as a https web server for my application (Secured >>> > socket >>> > layer). I am new to security. >>> > >>> > One of the users reported that sending random data to the web server >>> > port >>> > makes beam.smp to consume 100% CPU indefinitely for a very long period. >>> > >>> > >>> > >>> > Step to reproduce >>> > >>> > 1.?????? Start a mochiweb https server (can use mochiweb web storage app >>> > example provided with no other options) on a specific port (ex: 8443 ) >>> > >>> > 2.?????? Run the command ?nc? [IP address]? 8443 >> > >>> > 3.?????? The server rejects the connection (you may get ssl record error >>> > sometimes). The command returns to the shell >>> > >>> > 4.?????? Repeat the above command for some time till the command doesn?t >>> > return back to shell. >>> > >>> > 5.?????? Observe beam.smp taking 100% CPU and also? memory getting >>> > increase. >>> > (It may reach system limit and crash) >>> > >>> > 6.?????? If you stop the command (nc ^C), it returns to normal >>> > >>> > >>> > >>> > Observations >>> > >>> > It is reproducible in both Linux and windows. >>> > >>> > It happens for other https web server (tried with musultin) >>> > >>> > It does not happen for http server (should be in lib ssl). >>> > >>> > Tried giving different SSL options like {verify, verify_peer} with empty >>> > certificate as valid, but did not help. >>> > >>> > For other web servers written in other language this is not the >>> > behavior. >>> > Example lighttpd (php) nc just returns back to shell everytime. >>> > >>> > >>> > >>> > >>> > >>> > I debugged the issue to find that in other cases where the nc commands >>> > return immediately to shell the ssl connection does not succeed (behaves >>> > normally). >>> > >>> > But for the case where nc does not return back, the call is in an >>> > infinite >>> > loop ?next_tls_record in ssl_connection.erl? , thus making the CPU to >>> > take >>> > 100% and in memory increase. >>> > >>> > >>> > >>> > This can lead denial of service attack. Is this a bug and should I raise >>> > it >>> > in bug report forum. >>> > >>> > >>> > >>> > Any help will be appreciated. >>> > >>> > Thanks & Regards, >>> > >>> > Vinod >>> > >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> > >> >> >> >> -- >> >> Thanks & Regards, >> Vinod H. G. >> > From raould@REDACTED Wed Jun 1 20:36:06 2011 From: raould@REDACTED (Raoul Duke) Date: Wed, 1 Jun 2011 11:36:06 -0700 Subject: [erlang-questions] image app? Message-ID: hi, no i have no free time to do this myself :-( and yes i asked this same question on the Lift list, since i would guess Erlang and Lift communities are the most likely to have non-suckful apps :-) so i've been searching high and low for an image storage and categorization system that doesn't suck. i've checked out the top online image places (flickr, imageshack, photobucket, et. al.) and inevitably they have some super stunningly stupid suckful aspect to them that makes me want to scream. i've started to check out some off-the-shelf turn-key roll-your-own image cms's that i could plonk down on my home server and just say 'screw it' to the issues of backup and bandwidth for now -- it is more urgent for me to just find a usable user interface for managing things. but then i worry about putting some random php system on my machine - like that isn't just begging to get hacked that way. so. does such an image app beast happen to exist in Erlang-land? i bet i can draw up a short product spec for free (that's a little joke. i'd actually charge $80/hour. ok, ok, that's a joke too.) about what i'd want in a system... :-) sincerely hoping it can't hurt to ask... From steve@REDACTED Wed Jun 1 21:26:39 2011 From: steve@REDACTED (Steve Strong) Date: Wed, 1 Jun 2011 21:26:39 +0200 Subject: [erlang-questions] Understanding supervisor / start_link behaviour Message-ID: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> Hi, I've got some strange behaviour with gen_event within a supervision tree which I don't fully understand. Consider the following supervisor (completely standard, feel free to skip over): -module(sup). -behaviour(supervisor). -export([start_link/0, init/1]). -define(SERVER, ?MODULE). start_link() -> supervisor:start_link({local, ?SERVER}, ?MODULE, []). init([]) -> Child1 = {child, {child, start_link, []}, permanent, 2000, worker, [child]}, {ok, {{one_for_all, 1000, 3600}, [Child1]}}. and corresponding gen_server (interesting code in bold): -module(child). -behaviour(gen_server). -export([start_link/0, init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). start_link() -> gen_server:start_link({local, child}, child, [], []). init([]) -> io:format("about to start gen_event~n"), X = gen_event:start_link({local, my_gen_event}), io:format("gen_event started with ~p~n", [X]), {ok, _Pid} = X, {ok, {}, 2000}. handle_call(_Request, _From, State) -> {reply, ok, State}. handle_cast(_Msg, State) -> {noreply, State}. handle_info(_Info, State) -> io:format("about to crash...~n"), 1 = 2, {noreply, State}. terminate(_Reason, _State) -> ok. code_change(_OldVsn, State, _Extra) -> {ok, State}. If I run this from an erl shell like this: --> erl Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.2 (abort with ^G) 1> application:start(sasl), supervisor:start_link(sup, []). Then the supervisor & server start as expected. After 2 seconds the server gets a timeout message and crashes itself; the supervisor obviously spots this and restarts it. Within the init of the gen_server, it also does a start_link on a gen_event process. By my understanding, whenever the gen_server process exits, the gen_event will also be terminated. However, every now and then I see the following output (a ton of sasl trace omitted for clarity!): about to crash... about to start gen_event gen_event started with {error,{already_started,<0.79.0>}} about to start gen_event gen_event started with {error,{already_started,<0.79.0>}} about to start gen_event What is happening is that the gen_server is crashing but on its restart the gen_event process is still running - hence the gen_server fails in its init and gets restarted again. Sometimes this loop clears after a few iterations, other times it can continue until the parent supervisor gives up, packs its bags and goes home. So, my question is whether this is expected behaviour or not. I assume that the termination of the linked child is happening asynchronously, and that the supervisor is hence restarting its children before things have cleaned up correctly - is that correct? I can fix this particular scenario by trapping exits within the gen_server, and then calling gen_event:stop within the terminate. Is this type of processing necessary whenever a process is start_link'ed within a supervisor tree, or is what I'm doing considered bad practice? Thanks for your time, Steve -- Steve Strong, Director, id3as twitter.com/srstrong -------------- next part -------------- An HTML attachment was scrubbed... URL: From moxford@REDACTED Wed Jun 1 21:39:49 2011 From: moxford@REDACTED (Mike Oxford) Date: Wed, 1 Jun 2011 12:39:49 -0700 Subject: [erlang-questions] "speed question" Message-ID: ETS-backed (replicated, ram-only) mnesia dirty-reads (or ets:lookup) vs application:get_env calls. I would guess get_env > ets > dirty_reads, but does anyone have any recent performance background around these? I can find some stuff from 8 years ago but nothing recent. TIA! -mox -------------- next part -------------- An HTML attachment was scrubbed... URL: From iman1003@REDACTED Wed Jun 1 22:21:40 2011 From: iman1003@REDACTED (Ian Plosker) Date: Wed, 1 Jun 2011 16:21:40 -0400 Subject: [erlang-questions] image app? In-Reply-To: References: Message-ID: It isn't an image storage system, but Riak (http://wiki.basho.com/) could satisfy your needs. It is a distributed key-value storage engine with Erlang, HTTP, and Protocol Buffers interfaces. Categorization could be done with either metadata, Riak links, or the use of multiple buckets. On Wed, Jun 1, 2011 at 2:36 PM, Raoul Duke wrote: > hi, > > no i have no free time to do this myself :-( and yes i asked this same > question on the Lift list, since i would guess Erlang and Lift > communities are the most likely to have non-suckful apps :-) > > so i've been searching high and low for an image storage and > categorization system that doesn't suck. i've checked out the top > online image places (flickr, imageshack, photobucket, et. al.) and > inevitably they have some super stunningly stupid suckful aspect to > them that makes me want to scream. i've started to check out some > off-the-shelf turn-key roll-your-own image cms's that i could plonk > down on my home server and just say 'screw it' to the issues of backup > and bandwidth for now -- it is more urgent for me to just find a > usable user interface for managing things. but then i worry about > putting some random php system on my machine - like that isn't just > begging to get hacked that way. > > so. does such an image app beast happen to exist in Erlang-land? > > i bet i can draw up a short product spec for free (that's a little > joke. i'd actually charge $80/hour. ok, ok, that's a joke too.) about > what i'd want in a system... :-) > > sincerely hoping it can't hurt to ask... > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Wed Jun 1 23:15:29 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 1 Jun 2011 23:15:29 +0200 Subject: [erlang-questions] Understanding supervisor / start_link behaviour In-Reply-To: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> References: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> Message-ID: hi steve, your gen_event should be started by your supervisor too. in this case, since you specified a one_for_all behaviour, when gen_server crashes, gen_event will be restarted too. r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From spawn.think@REDACTED Wed Jun 1 23:57:37 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Wed, 1 Jun 2011 23:57:37 +0200 Subject: [erlang-questions] Understanding supervisor / start_link behaviour In-Reply-To: References: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> Message-ID: Agree with Roberto, you should put under supervisor. Regarding your case, i would guess you are trapping exit in your init in my_gen_event? On Wed, Jun 1, 2011 at 11:15 PM, Roberto Ostinelli wrote: > hi steve, > > your gen_event should be started by your supervisor too. in this case, > since you specified a one_for_all behaviour, when gen_server crashes, > gen_event will be restarted too. > > r. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think -------------- next part -------------- An HTML attachment was scrubbed... URL: From Philip.Baker@REDACTED Thu Jun 2 00:32:36 2011 From: Philip.Baker@REDACTED (Philip Baker) Date: Wed, 1 Jun 2011 18:32:36 -0400 Subject: [erlang-questions] Strange behaviour in unicode:characters_to_list/2 In-Reply-To: References: Message-ID: <3173206D0F5F6B418F76F5997B2AE75202C63387@gatex01.CMLnet.peinet.peinc.com> I've only been using erlang for a couple of days, so I may be missing something, but I've come across some behavior that I find baffling with the unicode:characters_to_list function, when it is passed a binary with an illegal utf8 character. Here is some code that demonstrates it: -define(TESTCALL(N, Call), io:format("~p: ~s: ~n\t\t~p~n~n", [N, ??Call, Call]), timer:sleep(1)). test2( )-> ?TESTCALL(1, unicode:characters_to_list(<<16#c3, 16#a0, 98, 99, 16#81,0,0,0 >>, utf8)), ?TESTCALL(2, Rest = <<16#c3, 16#a0, 98, 99, 16#81,0,0,0 >>), ?TESTCALL(3, unicode:characters_to_list(Rest, utf8)), ?TESTCALL(4, <> = <<16#c3, 16#a0, 98, 99, 16#81,0,0,0 >>), ?TESTCALL(5, unicode:characters_to_list(Rest2, utf8)), ?TESTCALL(6, <<_:1/binary, Rest3/binary>> = <<0, 16#c3, 16#a0, 98, 99, 16#81,0,0,0 >>), ?TESTCALL(7, Rest3), ?TESTCALL(8, unicode:characters_to_list(Rest3, utf8)), ?TESTCALL(9, Rest4 = list_to_binary([Rest3])), ?TESTCALL(10, unicode:characters_to_list(Rest4, utf8)), ?TESTCALL(11, Rest4 =:= Rest3), ?TESTCALL(12, unicode:characters_to_list(Rest3, utf8) =:= unicode:characters_to_list(Rest4, utf8)). Calling test2() gives this output: 1: unicode : characters_to_list ( << 195 , 160 , 98 , 99 , 129 , 0 , 0 , 0 >> , utf8 ): {error,"?bc",<<129,0,0,0>>} 2: Rest = << 195 , 160 , 98 , 99 , 129 , 0 , 0 , 0 >>: <<195,160,98,99,129,0,0,0>> 3: unicode : characters_to_list ( Rest , utf8 ): {error,"?bc",<<129,0,0,0>>} 4: << Rest2 / binary >> = << 195 , 160 , 98 , 99 , 129 , 0 , 0 , 0 >>: <<195,160,98,99,129,0,0,0>> 5: unicode : characters_to_list ( Rest2 , utf8 ): {error,"?bc",<<129,0,0,0>>} 6: << _ : 1 / binary , Rest3 / binary >> = << 0 , 195 , 160 , 98 , 99 , 129 , 0 , 0 , 0 >>: <<0,195,160,98,99,129,0,0,0>> 7: Rest3: <<195,160,98,99,129,0,0,0>> 8: unicode : characters_to_list ( Rest3 ): {error,"?bc",<<99,129,0,0>>} 9: Rest4 = list_to_binary ( [ Rest3 ] ): <<195,160,98,99,129,0,0,0>> 10: unicode : characters_to_list ( Rest4 , utf8 ): {error,"?bc",<<129,0,0,0>>} 11: Rest4 =:= Rest3: true 12: unicode : characters_to_list ( Rest3 , utf8 ) =:= unicode : characters_to_list ( Rest4 , utf8 ): false Looking at lines 11 and 12, I don't understand how if Rest4 and Rest3 are equal, the calls to characters_to_list could give unequal results. The earlier lines demonstrate various ways I have constructed binaries to pass to characters_to_list. In most cases the results are what I expect, but lines 6-8 show that when I use pattern matching to extract it from a larger binary, characters_to_list produces output where the letter 'c' is included in both the converted list, and the unconverted "RestData" binary. In lines 9 and 10, I create an apparently identical binary using list_to_binary/1 with a list containing only the "broken" binary. This new binary produces the correct output again. I'm using R14B03 on Windows, in case that makes a difference. If anyone can tell me if there is something I am misunderstanding, or if this is a bug, I'd appreciate it greatly. Philip Baker Software Developer Cassidian Communications, an EADS North America Company 75 Boul. de la Technologie Gatineau, Qu?bec Canada,?J8Z 3G4 819.778.2053, x243 DIRECT www.CassidianCommunications.com From robert.virding@REDACTED Thu Jun 2 02:23:44 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 2 Jun 2011 00:23:44 +0000 (GMT) Subject: [erlang-questions] Strange behaviour in unicode:characters_to_list/2 In-Reply-To: <975626033.248541306974188329.JavaMail.root@zimbra> Message-ID: <99524589.248561306974224704.JavaMail.root@zimbra> A quick comment though as to why test 12 returns false is that the values in tests 8 and 10 differ. The question is of course why they differ as they should be the same. I can confirm your results running from the shell with some extra examples. 34> B = << 0 , 195 , 160 , 98 , 99 , 129 , 0 , 0 , 0 >>. <<0,195,160,98,99,129,0,0,0>> 35> B1 = << 195 , 160 , 98 , 99 , 129 , 0 , 0 , 0 , 0 >>. <<195,160,98,99,129,0,0,0,0>> 36> <<_:1/binary,Rest3/binary>> = B. <<0,195,160,98,99,129,0,0,0>> 37> Rest4 = list_to_binary([Rest3]). <<195,160,98,99,129,0,0,0>> 38> Rest3==Rest4. true 41> Rest5 = binary:copy(Rest3). <<195,160,98,99,129,0,0,0>> 42> <> = B1. <<195,160,98,99,129,0,0,0,0>> 44> unicode:characters_to_list(Rest3,utf8). {error,"?bc",<<99,129,0,0>>} 45> unicode:characters_to_list(Rest4,utf8). {error,"?bc",<<129,0,0,0>>} 46> unicode:characters_to_list(Rest5,utf8). {error,"?bc",<<129,0,0,0>>} 48> unicode:characters_to_list(Rest6,utf8). {error,"?bc",<<129,0,0,0>>} So Rest3 does behave differently *in this case* from other binaries which contain the same bytes but have been created in a different way. Interestingly if Rest3 has been sent to another process it there behaves normally. My partially educated guess is that when Rest3 has been created in that way then there is some "rest" in it which causes unicode:characters_to_list/2 to behave in a strange way. This goes away when the binary is copied in any way. But just guessing. I am running R14B02 on Mac OSX. Robert ----- "Philip Baker" wrote: > I've only been using erlang for a couple of days, so I may be missing > something, but I've come across some behavior that I find baffling > with the unicode:characters_to_list function, when it is passed a > binary with an illegal utf8 character. > > Here is some code that demonstrates it: > > -define(TESTCALL(N, Call), io:format("~p: ~s: ~n\t\t~p~n~n", [N, > ??Call, Call]), timer:sleep(1)). > > test2( )-> > ?TESTCALL(1, unicode:characters_to_list(<<16#c3, 16#a0, 98, 99, > 16#81,0,0,0 >>, utf8)), > ?TESTCALL(2, Rest = <<16#c3, 16#a0, 98, 99, 16#81,0,0,0 >>), > ?TESTCALL(3, unicode:characters_to_list(Rest, utf8)), > ?TESTCALL(4, <> = <<16#c3, 16#a0, 98, 99, > 16#81,0,0,0 >>), > ?TESTCALL(5, unicode:characters_to_list(Rest2, utf8)), > ?TESTCALL(6, <<_:1/binary, Rest3/binary>> = <<0, 16#c3, 16#a0, 98, > 99, 16#81,0,0,0 >>), > ?TESTCALL(7, Rest3), > ?TESTCALL(8, unicode:characters_to_list(Rest3, utf8)), > ?TESTCALL(9, Rest4 = list_to_binary([Rest3])), > ?TESTCALL(10, unicode:characters_to_list(Rest4, utf8)), > ?TESTCALL(11, Rest4 =:= Rest3), > ?TESTCALL(12, unicode:characters_to_list(Rest3, utf8) =:= > unicode:characters_to_list(Rest4, utf8)). > > Calling test2() gives this output: > > 1: unicode : characters_to_list ( << 195 , 160 , 98 , 99 , 129 , 0 , > 0 , 0 >> , utf8 ): > {error,"?bc",<<129,0,0,0>>} > > 2: Rest = << 195 , 160 , 98 , 99 , 129 , 0 , 0 , 0 >>: > <<195,160,98,99,129,0,0,0>> > > 3: unicode : characters_to_list ( Rest , utf8 ): > {error,"?bc",<<129,0,0,0>>} > > 4: << Rest2 / binary >> = << 195 , 160 , 98 , 99 , 129 , 0 , 0 , 0 > >>: > <<195,160,98,99,129,0,0,0>> > > 5: unicode : characters_to_list ( Rest2 , utf8 ): > {error,"?bc",<<129,0,0,0>>} > > 6: << _ : 1 / binary , Rest3 / binary >> = << 0 , 195 , 160 , 98 , 99 > , 129 , 0 , 0 , 0 >>: > <<0,195,160,98,99,129,0,0,0>> > > 7: Rest3: > <<195,160,98,99,129,0,0,0>> > > 8: unicode : characters_to_list ( Rest3 ): > {error,"?bc",<<99,129,0,0>>} > > 9: Rest4 = list_to_binary ( [ Rest3 ] ): > <<195,160,98,99,129,0,0,0>> > > 10: unicode : characters_to_list ( Rest4 , utf8 ): > {error,"?bc",<<129,0,0,0>>} > > 11: Rest4 =:= Rest3: > true > > 12: unicode : characters_to_list ( Rest3 , utf8 ) =:= unicode : > characters_to_list ( Rest4 , utf8 ): > false > > > Looking at lines 11 and 12, I don't understand how if Rest4 and Rest3 > are equal, the calls to characters_to_list could give unequal results. > > > The earlier lines demonstrate various ways I have constructed binaries > to pass to characters_to_list. In most cases the results are what I > expect, but lines 6-8 show that when I use pattern matching to extract > it from a larger binary, characters_to_list produces output where the > letter 'c' is included in both the converted list, and the unconverted > "RestData" binary. In lines 9 and 10, I create an apparently identical > binary using list_to_binary/1 with a list containing only the "broken" > binary. This new binary produces the correct output again. > > I'm using R14B03 on Windows, in case that makes a difference. > > If anyone can tell me if there is something I am misunderstanding, or > if this is a bug, I'd appreciate it greatly. > > > Philip Baker > Software Developer > > Cassidian Communications, an EADS North America Company > 75 Boul. de la Technologie > Gatineau, Qu?bec > Canada,?J8Z 3G4 > 819.778.2053, x243 DIRECT > www.CassidianCommunications.com > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From b17flyboy@REDACTED Thu Jun 2 03:56:12 2011 From: b17flyboy@REDACTED (Frank Singleton) Date: Wed, 01 Jun 2011 20:56:12 -0500 Subject: [erlang-questions] CSLab, the first 10 years In-Reply-To: <413169C8-6907-4CF8-82B6-016F8B3EE5EA@erlang-solutions.com> References: <413169C8-6907-4CF8-82B6-016F8B3EE5EA@erlang-solutions.com> Message-ID: <4DE6EDBC.1010403@gmail.com> On 05/30/2011 04:58 AM, Ulf Wiger wrote: > > Bjarne D?cker once wrote a nice booklet called "Datalogilaboratoriet - > de f?rsta 10 ?ren" (The Computer Science Lab - the first 10 years). > > http://www.erlang.se/publications/jubileumsskrift.html (unfortunately > without the great pictures found in the printed book) > > I just tried running it through Google Translate > Ulf, this was very interesting reading, thanks !! Cheers / Frank From igorrs@REDACTED Thu Jun 2 06:33:34 2011 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Thu, 2 Jun 2011 01:33:34 -0300 Subject: [erlang-questions] Unidirectional linking? In-Reply-To: <20110525094512.GB6149@erix.ericsson.se> References: <20110525094512.GB6149@erix.ericsson.se> Message-ID: Well... I'm using something like the last code I posted (with a third process that creates two monitors) and it's working fine so far. As for supervision trees, I use them for some cases (usually for processes that run indefinitely, which I spawn as gen_servers), mostly because I don't see how they "fit" in the other cases (e.g. in this discussion). But maybe I just need to have a better look at supervisors. I got "Erlang and OTP in Action" from a friend for my birthday. It could be the opportunity. :) Thank you. Igor. On Wed, May 25, 2011 at 6:45 AM, Raimo Niskanen wrote: > On Wed, May 25, 2011 at 09:28:31AM +0200, Mazen Harake wrote: >> Perhaps something like this: >> >> process_flag(trap_exit, true), >> Parent = self(), >> Child = spawn_link(fun() -> Parent ! (catch Op) end), >> receive >> ? {'EXIT', ParentsParent, Reason} -> >> ? ? exit(Reason); >> ? Result -> >> ? ? Result >> ?after N -> >> ? ? unlink(Child), >> ? ? exit(Child, timeout), >> ? ? timeout >> end. > > Can the parent always have trap_exit true? > > If so this becomes (rather) simple: > > ?process_flag(trap_exit, true), > ?, > ?, > ?Parent = self(), > ?Ref = make_ref(), > ?Child = spawn_link(fun() -> Parent ! {Ref,Op()} end), > ?receive > ? ?{'EXIT',Child,Reason} -> > ? ? ?receive > ? ? ? ?{Ref,Result} -> > ? ? ? ? ?{ok,Result} > ? ? ?after 0 -> > ? ? ? ?{error,Reason} > ? ? ?end; > ?after N -> > ? ?exit(Child, timeout), > ? ? ?receive > ? ? ? ?{'EXIT',Child,timeout} -> > ? ? ? ? ?receive > ? ? ? ? ? ?{Ref,Result} -> > ? ? ? ? ? ? ?{ok,Result} > ? ? ? ? ?after 0 -> > ? ? ? ? ? ?timeout > ? ? ? ? ?end > ? ? ?end > ?end > > If you have no control over the parent trap_exit flag I see > no other alternative than involving a third process. > > Note that when a linked process exits normally, the link > causes no exit at the other end. > > You maybe should think about using supervisers > and all the other heavy OTP stuff. It was designed for > this kind of problems. > > > >> >> if you get an exit signal from the parent's parent you will exit if you are >> in the receive clause and you will ignore it (unless it is a kill) if you >> are in between the unlink and exit child? >> >> I do however still think that this is very defensive and that the >> probability of getting an exit between unlink and exit is very small unless >> the timeout in N is exactly the same (very very close to) the timeout that >> the parent's parent has for exiting the parent. >> >> >> >> On 25 May 2011 08:51, Igor Ribeiro Sucupira wrote: >> >> > Supposing someone used the same logic (with timeout/kill) to spawn the >> > parent process, the parent could be killed at any moment (but yeah: it >> > would be bad luck to have that happening right after it called >> > unlink). >> > >> > Thanks. >> > Igor. >> > >> > On Wed, May 25, 2011 at 3:20 AM, Mazen Harake >> > wrote: >> > > Why would someone "kill the parent"? Do you have processes which are >> > > randomly choosing other processes to terminate? ;) >> > > >> > > If your answer is "No" then I would suggest that you just kill the worker >> > > processes that is taking to long, kill it in cold blood, imo. >> > > >> > > There is no reason to think about too many "if"-scenarios when the >> > scenarios >> > > are too far fetched. Try the simple version first :) >> > > >> > > otherwise you can either trap_exits or use monitors instead. >> > > >> > > /M >> > > >> > > On 25 May 2011 08:00, Igor Ribeiro Sucupira wrote: >> > >> >> > >> Suppose there is a heavy operation Op that in some cases takes so long >> > >> to finish that the caller loses interest in the result and gives up. >> > >> >> > >> I want to perform that operation in a way that allows me to: >> > >> 1) Interrupt its execution if it does not finish in N milliseconds. >> > >> 2) Interrupt its execution if the calling process exits (here I'm >> > >> already supposing Op has to be run in another Erlang process, due to >> > >> goal 1). >> > >> >> > >> To implement that, it seems unidirectional linking would be needed. Is >> > >> there another safe and convenient way to do it? >> > >> >> > >> The first idea I had was something like this: >> > >> >> > >> Parent = self(), >> > >> Child = spawn_link(fun() -> Parent ! (catch Op) end), >> > >> receive Result -> Result >> > >> after N -> unlink(Child), exit(Child, timeout), timeout >> > >> end. >> > >> >> > >> But, if Parent is killed by another process right after calling >> > >> unlink, Child would be left executing. >> > >> Another problem is that I don't want Parent to die if Child exits for >> > >> non-timeout reasons (although it seems very unlikely in the code >> > >> above, with the catch). >> > >> >> > >> I was now thinking of substituting unlink(Child) with >> > >> process_flag(trap_exit, true) and then kill Child, receive its exit >> > >> message, set trap_exit to false again (I'm assuming it was false), and >> > >> finally check if there were other exit messages (suiciding if it was >> > >> the case). >> > >> >> > >> But then the code would become too ugly, so I got lazy and decided to >> > >> post to this list. ?:-) >> > >> >> > >> What do you think? >> > >> >> > >> Thanks. >> > >> Igor. > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ulf.wiger@REDACTED Thu Jun 2 07:56:32 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 2 Jun 2011 07:56:32 +0200 Subject: [erlang-questions] "speed question" In-Reply-To: References: Message-ID: <3ABD146D-DC6A-4637-A70E-E97AA718D299@erlang-solutions.com> Generally, the performance should be insignificant, but? ets:lookup/2: 1 ets read (duh), but obviously, you need to write more code application:get_env/2: 1 ets read application:get_env/1: 2 ets reads (one to find the current application) mnesia:dirty_read/1: - 1 get() - 3 ets reads, if data is local (where_to_read, storage_type, lookup) Beyond the performance differences, there are semantic issues. Application environment variables are really meant to be static, either hard-coded in the .app file, or overridden at startup. They can also be changed at upgrades, but the expectation isn't that they change dynamically. In fact, for data that is truly static, the fastest access is if the data is simply hard-coded in an erlang module. Given that modules are easy to reload, these values can still be changed with low frequency. So for system preferences, it is generally better to store the values in a database. In my experience, this type of data (env- and configuration data) should be cached by the application, and an event should be generated when it is updated. I wrote some support in gproc for doing this. https://github.com/esl/gproc/blob/master/doc/gproc.md#get_env-3 The idea is that gproc works as both a cache and pub/sup framework, so the processes that have cached a value can easily be notified if the value changes. BR, Ulf W On 1 Jun 2011, at 21:39, Mike Oxford wrote: > ETS-backed (replicated, ram-only) mnesia dirty-reads (or ets:lookup) vs application:get_env calls. > > I would guess get_env > ets > dirty_reads, but does anyone have any recent performance background around these? > I can find some stuff from 8 years ago but nothing recent. > > TIA! > > -mox > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve@REDACTED Thu Jun 2 09:15:17 2011 From: steve@REDACTED (Steve Strong) Date: Thu, 2 Jun 2011 09:15:17 +0200 Subject: [erlang-questions] Understanding supervisor / start_link behaviour In-Reply-To: References: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> Message-ID: Yeah, that makes perfect sense and would obviously solve the problem. The reason we'd gone down this path was that we had a number of "sub" processes (the gen_event just being one example) that we felt would be "polluting" the supervisor; these sub-processes were just helpers of the primary gen_servers that the supervisor was controlling - using a start_link in the primary gen_servers felt like a very clean and easy way of spinning up these other processes in a way that (we thought) would still be resilient to failures. The thing that bit us was that we naively thought that, due to the sub-process being linked, it would die when the parent died. Of course, it does, but its death is asynchronous to the notification that the supervisor receives and hence it may well still be alive (doomed, but alive) when the supervisor begins the restart cycle. Our servers don't crash that often, and when they do this race condition is was rarely seen, which was reinforced our misconceptions. The only thing that does surprise me is how many times the supervisor can go round the restart loop before the doomed process finally exits - we have seen it thrash round this loop about 1000 times before the supervisor itself finally fails; I guess it's just down to how things are being scheduled by the VM, and in those cases we were just getting unlucky. Sounds like best-practice within the OTP world is to have everything started via a supervisor - is that a fair comment? Cheers, Steve -- Steve Strong, Director, id3as twitter.com/srstrong On Wednesday, 1 June 2011 at 23:57, Ahmed Omar wrote: > Agree with Roberto, you should put under supervisor. Regarding your case, i would guess you are trapping exit in your init in my_gen_event? > > On Wed, Jun 1, 2011 at 11:15 PM, Roberto Ostinelli wrote: > > hi steve, > > > > your gen_event should be started by your supervisor too. in this case, since you specified a one_for_all behaviour, when gen_server crashes, gen_event will be restarted too. > > > > r. > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think (http://twitter.com/#!/spawn_think) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wiener.guy@REDACTED Thu Jun 2 09:50:38 2011 From: wiener.guy@REDACTED (Guy Wiener) Date: Thu, 2 Jun 2011 10:50:38 +0300 Subject: [erlang-questions] Original Erlang Prolog syntax? Message-ID: Hello, Does someone still have the original syntax of Erlang as Prolog terms? I don't need to actually run it. I am just interested to see how it was crafted as a Prolog internal DSL - Operators definitions, term expanders, priorities, etc. Thanks in advance, Guy Wiener. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lenartlad@REDACTED Thu Jun 2 10:23:05 2011 From: lenartlad@REDACTED (Ladislav Lenart) Date: Thu, 02 Jun 2011 10:23:05 +0200 Subject: [erlang-questions] Understanding supervisor / start_link behaviour In-Reply-To: References: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> Message-ID: <4DE74869.5010803@volny.cz> Hello. I am by no means an expert on the topic but I would like to point out that the only reason you get {already_started, ...} error is because you attempt to register the helper process with {local, ...}. If it is a helper, there should be no reason for it to be globally accessible. And if it wasn't registered, the gen_server would be restarted without issues creating new helper process. The old helper would die eventually just as you expect it to. Ladislav Lenart On 2.6.2011 09:15, Steve Strong wrote: > Yeah, that makes perfect sense and would obviously solve the problem. > > The reason we'd gone down this path was that we had a number of "sub" processes (the gen_event just being one example) that we felt would be "polluting" the supervisor; these sub-processes were just > helpers of the primary gen_servers that the supervisor was controlling - using a start_link in the primary gen_servers felt like a very clean and easy way of spinning up these other processes in a way > that (we thought) would still be resilient to failures. > > The thing that bit us was that we naively thought that, due to the sub-process being linked, it would die when the parent died. Of course, it does, but its death is asynchronous to the notification > that the supervisor receives and hence it may well still be alive (doomed, but alive) when the supervisor begins the restart cycle. Our servers don't crash that often, and when they do this race > condition is was rarely seen, which was reinforced our misconceptions. The only thing that does surprise me is how many times the supervisor can go round the restart loop before the doomed process > finally exits - we have seen it thrash round this loop about 1000 times before the supervisor itself finally fails; I guess it's just down to how things are being scheduled by the VM, and in those > cases we were just getting unlucky. > > Sounds like best-practice within the OTP world is to have everything started via a supervisor - is that a fair comment? > > Cheers, > > Steve > > -- > Steve Strong, Director, id3as > twitter.com/srstrong > > On Wednesday, 1 June 2011 at 23:57, Ahmed Omar wrote: > >> Agree with Roberto, you should put under supervisor. Regarding your case, i would guess you are trapping exit in your init in my_gen_event? >> >> On Wed, Jun 1, 2011 at 11:15 PM, Roberto Ostinelli > wrote: >>> hi steve, >>> >>> your gen_event should be started by your supervisor too. in this case, since you specified a one_for_all behaviour, when gen_server crashes, gen_event will be restarted too. >>> >>> r. >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> >> -- >> Best Regards, >> - Ahmed Omar >> http://nl.linkedin.com/in/adiaa >> Follow me on twitter >> @spawn_think >> > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From antoine.koener@REDACTED Thu Jun 2 10:50:43 2011 From: antoine.koener@REDACTED (Antoine Koener) Date: Thu, 2 Jun 2011 10:50:43 +0200 Subject: [erlang-questions] Understanding supervisor / start_link behaviour In-Reply-To: References: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> Message-ID: <8589F507-324F-4693-985C-7085B15CB3C9@gmail.com> On Jun 2, 2011, at 09:15 , Steve Strong wrote: > Yeah, that makes perfect sense and would obviously solve the problem. > > The reason we'd gone down this path was that we had a number of > "sub" processes (the gen_event just being one example) that we felt > would be "polluting" the supervisor; these sub-processes were just > helpers of the primary gen_servers that the supervisor was > controlling - using a start_link in the primary gen_servers felt > like a very clean and easy way of spinning up these other processes > in a way that (we thought) would still be resilient to failures. I dealt with such a situation using a supervisor that starts all mandatory process and a child supervisor. The child supervisor starts all other processes that needs those mandatory processes. > > The thing that bit us was that we naively thought that, due to the > sub-process being linked, it would die when the parent died. Of > course, it does, but its death is asynchronous to the notification > that the supervisor receives and hence it may well still be alive > (doomed, but alive) when the supervisor begins the restart cycle. > Our servers don't crash that often, and when they do this race > condition is was rarely seen, which was reinforced our > misconceptions. The only thing that does surprise me is how many > times the supervisor can go round the restart loop before the doomed > process finally exits - we have seen it thrash round this loop about > 1000 times before the supervisor itself finally fails; I guess it's > just down to how things are being scheduled by the VM, and in those > cases we were just getting unlucky. > > Sounds like best-practice within the OTP world is to have everything > started via a supervisor - is that a fair comment? Yes, and application:start(sasl). is a way "observe" the whole starting process, and see what could be wrong. > > Cheers, > > Steve > > -- > Steve Strong, Director, id3as > twitter.com/srstrong > > On Wednesday, 1 June 2011 at 23:57, Ahmed Omar wrote: > >> Agree with Roberto, you should put under supervisor. Regarding your >> case, i would guess you are trapping exit in your init in >> my_gen_event? >> >> On Wed, Jun 1, 2011 at 11:15 PM, Roberto Ostinelli > > wrote: >>> hi steve, >>> >>> your gen_event should be started by your supervisor too. in this >>> case, since you specified a one_for_all behaviour, when gen_server >>> crashes, gen_event will be restarted too. >>> >>> r. >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> >> -- >> Best Regards, >> - Ahmed Omar >> http://nl.linkedin.com/in/adiaa >> Follow me on twitter >> @spawn_think >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From wsong.cn@REDACTED Thu Jun 2 10:53:44 2011 From: wsong.cn@REDACTED (Wenqiang Song) Date: Thu, 2 Jun 2011 16:53:44 +0800 Subject: [erlang-questions] Erlang and Memory management In-Reply-To: References: <1782517589.242981306865189704.JavaMail.root@zimbra> <4DE53122.2020904@cs.ntua.gr> Message-ID: A related question, for code like following: test(<<255, T/binary>>) -> ok; test(<<_, T/binary>>) -> test(T). Does Erlang copy the input binary for each recursion? If the input is huge, it would be very low efficient. If so, what should we do? Thanks Andy On Wed, Jun 1, 2011 at 3:28 AM, Paul Meier wrote: > I'm in a similar situation. I've got a few large tries, implemented > with gb_trees. I keep them in an orddict, which I'm using as the > State variable of a gb_server. For example: > > ------ > init() -> > %% State is an orddict of 3 giant Tries, pre-made and fetched from > the filesystem. > {ok, State}. > > handle_call({keyed_operation_on_large_data, Key}, _From, State) -> > GiantTrie = orddict:find(Key, State), > .... computations using GiantTrie.... > {reply, Result, State}. > > I'm finding that my process is crashing due to inability to mmap, and > the erl_crash.dump file tells me that the gen_server's supervisor (or, > on occasion, the error_logger) is the one with a prohibitively large > Stack+Heap. Also, I never modify the Tries or the orddict containing > them in calls to the gb_server: it's all read-only. > > Is storing the value in an orddict, or using regular terms as opposed > to binaries causing unnecessary copying? While _I_ know the data is > being used in a read-only fashion, is there a place where copying > might be occurring because the VM doesn't know this? > > I was hoping to play with binaries as a next attempt at fixing this > (the links in this thread suggest that large binaries are handled more > conservatively), but this thread appeared fortuitously, and maybe > someone can spot the very likely obvious error that I'm making. > > I'm grateful for your help, whether in this thread or the others I've > enjoyed reading ^_^ > > -Paul > > > On Tue, May 31, 2011 at 11:19 AM, Kostis Sagonas > wrote: > > Robert Virding wrote: > >> > >> I just want to point out that these rules are BEAM specific and not > Erlang > >> specific. > > > > Well, this is not correct: the BEAM nowhere specifies that its > instruction > > set has to be implemented using a private heap architecture. In fact, we > > successfully used the BEAM instruction set to implement both a shared > heap > > and a hybrid heap implementation. See: > > > > http://user.it.uu.se/~kostis/Papers/heap.pdf > > > >> So on an implementation with a single heap all data is shared. Now there > >> aren't currently many single heap implementations, to be exact only one, > >> erjang (Erlang on the JVM). :-) > > > > There have been more. We had an OTP system with a shared heap and the > > hybrid heap system was part of OTP for quite a while. IMO, it's too bad > > that it was not maintained when Erlang/OTP was extended to support SMP > > architectures. > > > > Also, the ETOS (Erlang to Scheme) system was based on a shared heap > > architecture. > > > > Kostis > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From spawn.think@REDACTED Thu Jun 2 11:03:18 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Thu, 2 Jun 2011 11:03:18 +0200 Subject: [erlang-questions] Understanding supervisor / start_link behaviour In-Reply-To: <4DE74869.5010803@volny.cz> References: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> <4DE74869.5010803@volny.cz> Message-ID: True ({local, Name} -> register locally, not globally, with Name) but you have to be careful if having two instances alive in the same time is acceptable or not. Steve, let's put it this way it's better to start processes under supervisor specially if you want to benefit from standard restarting strategies, it keeps your application cleaner. (as a hack, your case can also be solved by a monitor in the gen_server init before starting the gen_server erlang:monitor(process, my_gen_event), receive {'DOWN', Ref, process, Pid, Reason}-> ok end, ) But using supervisor is much cleaner and safer, and easier to design with in my opinion On Thu, Jun 2, 2011 at 10:23 AM, Ladislav Lenart wrote: > Hello. > > I am by no means an expert on the topic but I would like to point out > that the only reason you get {already_started, ...} error is because > you attempt to register the helper process with {local, ...}. If it is > a helper, there should be no reason for it to be globally accessible. > And if it wasn't registered, the gen_server would be restarted without > issues creating new helper process. The old helper would die eventually > just as you expect it to. > > > Ladislav Lenart > > > > On 2.6.2011 09:15, Steve Strong wrote: > >> Yeah, that makes perfect sense and would obviously solve the problem. >> >> The reason we'd gone down this path was that we had a number of "sub" >> processes (the gen_event just being one example) that we felt would be >> "polluting" the supervisor; these sub-processes were just >> helpers of the primary gen_servers that the supervisor was controlling - >> using a start_link in the primary gen_servers felt like a very clean and >> easy way of spinning up these other processes in a way >> that (we thought) would still be resilient to failures. >> >> The thing that bit us was that we naively thought that, due to the >> sub-process being linked, it would die when the parent died. Of course, it >> does, but its death is asynchronous to the notification >> that the supervisor receives and hence it may well still be alive (doomed, >> but alive) when the supervisor begins the restart cycle. Our servers don't >> crash that often, and when they do this race >> condition is was rarely seen, which was reinforced our misconceptions. The >> only thing that does surprise me is how many times the supervisor can go >> round the restart loop before the doomed process >> finally exits - we have seen it thrash round this loop about 1000 times >> before the supervisor itself finally fails; I guess it's just down to how >> things are being scheduled by the VM, and in those >> cases we were just getting unlucky. >> >> Sounds like best-practice within the OTP world is to have everything >> started via a supervisor - is that a fair comment? >> >> Cheers, >> >> Steve >> >> -- >> Steve Strong, Director, id3as >> twitter.com/srstrong >> >> On Wednesday, 1 June 2011 at 23:57, Ahmed Omar wrote: >> >> Agree with Roberto, you should put under supervisor. Regarding your case, >>> i would guess you are trapping exit in your init in my_gen_event? >>> >>> On Wed, Jun 1, 2011 at 11:15 PM, Roberto Ostinelli >> roberto@REDACTED>> wrote: >>> >>>> hi steve, >>>> >>>> your gen_event should be started by your supervisor too. in this case, >>>> since you specified a one_for_all behaviour, when gen_server crashes, >>>> gen_event will be restarted too. >>>> >>>> r. >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >>> >>> -- >>> Best Regards, >>> - Ahmed Omar >>> http://nl.linkedin.com/in/adiaa >>> Follow me on twitter >>> @spawn_think >>> >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think -------------- next part -------------- An HTML attachment was scrubbed... URL: From lenartlad@REDACTED Thu Jun 2 11:14:57 2011 From: lenartlad@REDACTED (Ladislav Lenart) Date: Thu, 02 Jun 2011 11:14:57 +0200 Subject: [erlang-questions] Understanding supervisor / start_link behaviour In-Reply-To: References: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> <4DE74869.5010803@volny.cz> Message-ID: <4DE75491.9090307@volny.cz> On 2.6.2011 11:03, Ahmed Omar wrote: > True ({local, Name} -> register locally, not globally, with Name) but you have to be careful if having two instances alive in the same time is acceptable or not. My bad. By "globally accessible" I meant that the locally registered process will be available to all processes on the local node. Ladislav Lenart > Steve, let's put it this way it's better to start processes under supervisor specially if you want to benefit from standard restarting strategies, it keeps your application cleaner. > (as a hack, your case can also be solved by a monitor in the gen_server init before starting the gen_server > erlang:monitor(process, my_gen_event), > receive > {'DOWN', Ref, process, Pid, Reason}-> > ok > end, > ) > But using supervisor is much cleaner and safer, and easier to design with in my opinion > On Thu, Jun 2, 2011 at 10:23 AM, Ladislav Lenart > wrote: > > Hello. > > I am by no means an expert on the topic but I would like to point out > that the only reason you get {already_started, ...} error is because > you attempt to register the helper process with {local, ...}. If it is > a helper, there should be no reason for it to be globally accessible. > And if it wasn't registered, the gen_server would be restarted without > issues creating new helper process. The old helper would die eventually > just as you expect it to. > > > Ladislav Lenart > > > > On 2.6.2011 09:15, Steve Strong wrote: > > Yeah, that makes perfect sense and would obviously solve the problem. > > The reason we'd gone down this path was that we had a number of "sub" processes (the gen_event just being one example) that we felt would be "polluting" the supervisor; these sub-processes > were just > helpers of the primary gen_servers that the supervisor was controlling - using a start_link in the primary gen_servers felt like a very clean and easy way of spinning up these other processes > in a way > that (we thought) would still be resilient to failures. > > The thing that bit us was that we naively thought that, due to the sub-process being linked, it would die when the parent died. Of course, it does, but its death is asynchronous to the > notification > that the supervisor receives and hence it may well still be alive (doomed, but alive) when the supervisor begins the restart cycle. Our servers don't crash that often, and when they do this race > condition is was rarely seen, which was reinforced our misconceptions. The only thing that does surprise me is how many times the supervisor can go round the restart loop before the doomed process > finally exits - we have seen it thrash round this loop about 1000 times before the supervisor itself finally fails; I guess it's just down to how things are being scheduled by the VM, and in those > cases we were just getting unlucky. > > Sounds like best-practice within the OTP world is to have everything started via a supervisor - is that a fair comment? > > Cheers, > > Steve > > -- > Steve Strong, Director, id3as > twitter.com/srstrong > > On Wednesday, 1 June 2011 at 23:57, Ahmed Omar wrote: > > Agree with Roberto, you should put under supervisor. Regarding your case, i would guess you are trapping exit in your init in my_gen_event? > > On Wed, Jun 1, 2011 at 11:15 PM, Roberto Ostinelli >> wrote: > > hi steve, > > your gen_event should be started by your supervisor too. in this case, since you specified a one_for_all behaviour, when gen_server crashes, gen_event will be restarted too. > > r. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > From jesper.louis.andersen@REDACTED Thu Jun 2 11:33:11 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 2 Jun 2011 11:33:11 +0200 Subject: [erlang-questions] Erlang and Memory management In-Reply-To: References: <1782517589.242981306865189704.JavaMail.root@zimbra> <4DE53122.2020904@cs.ntua.gr> Message-ID: On Thu, Jun 2, 2011 at 10:53, Wenqiang Song wrote: > A related question, for code like following: > test(<<255, T/binary>>) -> > ? ? ok; > test(<<_, T/binary>>) -> > ? ? test(T). > Does Erlang copy the input binary for each recursion? If the input is huge, > it would be very low?efficient. If so, what should we do? No. It just has a pointer into the existing binary. Immutability is nice :) http://www.erlang.org/doc/efficiency_guide/binaryhandling.html is the source you want to look at. Compile your code with the bin_opt_info flag to see if optimizations trigger as well. -- J. From jesper.louis.andersen@REDACTED Thu Jun 2 11:36:46 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 2 Jun 2011 11:36:46 +0200 Subject: [erlang-questions] "speed question" In-Reply-To: <3ABD146D-DC6A-4637-A70E-E97AA718D299@erlang-solutions.com> References: <3ABD146D-DC6A-4637-A70E-E97AA718D299@erlang-solutions.com> Message-ID: On Thu, Jun 2, 2011 at 07:56, Ulf Wiger wrote: > > Generally, the performance should be insignificant, but? > ets:lookup/2: > ??1 ets read (duh), but obviously, you need to write more code And ets:lookup_element/3: 1 ets read, optimal amount of data copied to the process -- J. From mazen.harake@REDACTED Thu Jun 2 11:53:02 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Thu, 2 Jun 2011 11:53:02 +0200 Subject: [erlang-questions] Understanding supervisor / start_link behaviour In-Reply-To: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> References: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> Message-ID: Steve, I wouldn't say that you are wrong. I think that you are reasoning good about not putting the gen_event module under a supervisor because *that is what links are for*. Just because you have a supervisor doesn't mean the you shove everything underneath there! If the gen_server and the gen_event are truly linked (meaning: gen_server doesn't act as a "supervisor" keeping track of its gen_event process and restarts it all the time but rather that they really are linked and they crash together) then your approach, in my opinion, is good. There are great benefits in doing it in that way. Many will claim that it is best practice to put *everything* under a supervisor but this is simply not true. 90% of cases it *is* the best thing to do and many times it is more about how you designed your application rather than where to put the supervisors and their children but doing it the way you did is not necessarily wrong. The only problem I see with your approach is that you have registered the gen_event process which clearly isn't useful (since only the gen_server should know about it, after all, it started it). Other than that, this approach is extremely helpful and a nice way to clean up things after they die/shutdown (Again: assuming truly linked). There is a big misconception in the community that everything should/must look like the supervisor-tree model which shows how gen_servers are put under supervisors and more supervisors under the "top" supervisor but that is not enforced and the design principles doesn't take many cases into account where this setup actually brings more headache to the table than to just exit and clean up using linked processes (because they do exist). /M On 1 June 2011 21:26, Steve Strong wrote: > Hi, > > I've got some strange behaviour with gen_event within a supervision tree > which I don't fully understand. ?Consider the following supervisor > (completely standard, feel free to skip over): > > -module(sup). > -behaviour(supervisor). > -export([start_link/0, init/1]). > -define(SERVER, ?MODULE). > start_link() -> > ? ? supervisor:start_link({local, ?SERVER}, ?MODULE, []). > init([]) -> > ? ? Child1 = {child, {child, start_link, []}, permanent, 2000, worker, > [child]}, > ? ? {ok, {{one_for_all, 1000, 3600}, [Child1]}}. > > and corresponding gen_server (interesting code in bold): > > -module(child). > -behaviour(gen_server). > -export([start_link/0, init/1, handle_call/3, handle_cast/2, > handle_info/2, terminate/2, code_change/3]). > start_link() -> > ? ? gen_server:start_link({local, child}, child, [], []). > init([]) -> > ? ? io:format("about to start gen_event~n"), > ? ? X = gen_event:start_link({local, my_gen_event}), > ? ? io:format("gen_event started with ~p~n", [X]), > ? ? {ok, _Pid} = X, > ? ? {ok, {}, 2000}. > handle_call(_Request, _From, State) -> > ? ? {reply, ok, State}. > handle_cast(_Msg, State) -> > ? ? {noreply, State}. > handle_info(_Info, State) -> > ? ? io:format("about to crash...~n"), > ? ? 1 = 2, > ? ? {noreply, State}. > terminate(_Reason, _State) -> > ? ? ok. > code_change(_OldVsn, State, _Extra) -> > ? ? {ok, State}. > > If I run this from an erl shell like this: > > --> erl > Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > Eshell V5.8.2 ?(abort with ^G) > 1> application:start(sasl), supervisor:start_link(sup, []). > > > Then the supervisor & server start as expected. ?After 2 seconds the server > gets a timeout message and crashes itself; the supervisor obviously spots > this and restarts it. ?Within the init of the gen_server, it also does a > start_link on a gen_event process. ?By my understanding, whenever the > gen_server process exits, the gen_event will also be terminated. > However, every now and then I see the following output (a ton of sasl trace > omitted for clarity!): > > about to crash... > about to start gen_event > gen_event started with {error,{already_started,<0.79.0>}} > about to start gen_event > gen_event started with {error,{already_started,<0.79.0>}} > about to start gen_event > > What is happening is that the gen_server is crashing but on its restart the > gen_event process is still running - hence the gen_server fails in its init > and gets restarted again. ?Sometimes this loop clears after a few > iterations, other times it can continue until the parent supervisor gives > up, packs its bags and goes home. > So, my question is whether this is expected behaviour or not. ?I assume that > the termination of the linked child is happening asynchronously, and that > the supervisor is hence restarting its children before things have cleaned > up correctly - is that correct? > I can fix this particular scenario by trapping exits within the gen_server, > and then calling gen_event:stop within the terminate. ?Is this type of > processing necessary whenever a process is start_link'ed within a supervisor > tree, or is what I'm doing considered bad practice? > Thanks for your time, > Steve > -- > Steve Strong, Director, id3as > twitter.com/srstrong > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From watson.timothy@REDACTED Thu Jun 2 11:55:13 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 2 Jun 2011 10:55:13 +0100 Subject: [erlang-questions] Understanding supervisor / start_link behaviour In-Reply-To: <4DE75491.9090307@volny.cz> References: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> <4DE74869.5010803@volny.cz> <4DE75491.9090307@volny.cz> Message-ID: On 2 June 2011 10:14, Ladislav Lenart wrote: > On 2.6.2011 11:03, Ahmed Omar wrote: >> >> True ({local, Name} ?-> register locally, not globally, with Name) but you >> have to be careful if having two instances alive in the same time is >> acceptable or not. > > My bad. By "globally accessible" I meant that the locally > registered process will be available to all processes on > the local node. You might consider gproc for these kinds of use cases. It provides a great deal of simplification around synchronising startups and registering names etc. From watson.timothy@REDACTED Thu Jun 2 11:57:04 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 2 Jun 2011 10:57:04 +0100 Subject: [erlang-questions] image app? In-Reply-To: References: Message-ID: On 1 June 2011 21:21, Ian Plosker wrote: > It isn't an image storage system, but Riak (http://wiki.basho.com/) could > satisfy your needs. It is a?distributed key-value?storage engine with > Erlang, HTTP, and Protocol Buffers interfaces. Categorization could be done > with either metadata, Riak links, or the use of multiple buckets. And https://github.com/basho/luwak will provide large (binary) object storage if you need it. You can also use riak-search to deal with lucene style indexing of associated meta-data. Very cool stuff. From hynek@REDACTED Thu Jun 2 11:59:01 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Thu, 2 Jun 2011 11:59:01 +0200 Subject: [erlang-questions] Erlang and Memory management In-Reply-To: References: <1782517589.242981306865189704.JavaMail.root@zimbra> <4DE53122.2020904@cs.ntua.gr> Message-ID: Of course not. But you can easily read it in efficiency guide or just try it and measure. If it would have quadratic time behavior it does copying, if linear it doesn't. You should learn a lot during this exercise about Erlang. On Thu, Jun 2, 2011 at 10:53 AM, Wenqiang Song wrote: > A related question, for code like following: > test(<<255, T/binary>>) -> > ? ? ok; > test(<<_, T/binary>>) -> > ? ? test(T). > Does Erlang copy the input binary for each recursion? If the input is huge, > it would be very low?efficient. If so, what should we do? > Thanks > Andy > On Wed, Jun 1, 2011 at 3:28 AM, Paul Meier wrote: >> >> I'm in a similar situation. ?I've got a few large tries, implemented >> with gb_trees. ?I keep them in an orddict, which I'm using as the >> State variable of a gb_server. ?For example: >> >> ------ >> init() -> >> ? ?%% State is an orddict of 3 giant Tries, pre-made and fetched from >> the filesystem. >> ? ?{ok, State}. >> >> handle_call({keyed_operation_on_large_data, Key}, _From, State) -> >> ? ?GiantTrie = orddict:find(Key, State), >> ? ?.... computations using GiantTrie.... >> ? ?{reply, Result, State}. >> >> I'm finding that my process is crashing due to inability to mmap, and >> the erl_crash.dump file tells me that the gen_server's supervisor (or, >> on occasion, the error_logger) is the one with a prohibitively large >> Stack+Heap. ?Also, I never modify the Tries or the orddict containing >> them in calls to the gb_server: it's all read-only. >> >> Is storing the value in an orddict, or using regular terms as opposed >> to binaries causing unnecessary copying? ?While _I_ know the data is >> being used in a read-only fashion, is there a place where copying >> might be occurring because the VM doesn't know this? >> >> I was hoping to play with binaries as a next attempt at fixing this >> (the links in this thread suggest that large binaries are handled more >> conservatively), but this thread appeared fortuitously, and maybe >> someone can spot the very likely obvious error that I'm making. >> >> I'm grateful for your help, whether in this thread or the others I've >> enjoyed reading ^_^ >> >> -Paul >> >> >> On Tue, May 31, 2011 at 11:19 AM, Kostis Sagonas >> wrote: >> > Robert Virding wrote: >> >> >> >> I just want to point out that these rules are BEAM specific and not >> >> Erlang >> >> specific. >> > >> > Well, this is not correct: the BEAM nowhere specifies that its >> > instruction >> > set has to be implemented using a private heap architecture. ?In fact, >> > we >> > successfully used the BEAM instruction set to implement both a shared >> > heap >> > and a hybrid heap implementation. ?See: >> > >> > ?http://user.it.uu.se/~kostis/Papers/heap.pdf >> > >> >> So on an implementation with a single heap all data is shared. Now >> >> there >> >> aren't currently many single heap implementations, to be exact only >> >> one, >> >> erjang (Erlang on the JVM). :-) >> > >> > There have been more. ?We had an OTP system with a shared heap and the >> > hybrid heap system was part of OTP for quite a while. ?IMO, it's too bad >> > that it was not maintained when Erlang/OTP was extended to support SMP >> > architectures. >> > >> > Also, the ETOS (Erlang to Scheme) system was based on a shared heap >> > architecture. >> > >> > Kostis >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From watson.timothy@REDACTED Thu Jun 2 12:12:16 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 2 Jun 2011 11:12:16 +0100 Subject: [erlang-questions] Erlang and Memory management In-Reply-To: References: <961326352.243381306879495091.JavaMail.root@zimbra> <1091703077.243401306880078397.JavaMail.root@zimbra> Message-ID: > Probably most Erlang code would faster with a shared heap, but some code > might slow down -- maybe by a lot. Putting certain smarts in Send might > help: e.g., if the receiving process isn't ready to run, copy; if it is > ready, just pass the pointer to shared memory. I'm sure lots more strategies > might be devised. And I'm about equally sure they'll all have their own > limits and pitfalls. Indeed. This is why the JVM provides a number of different (configurable) GC strategies, and does quite a lot of "self tuning" where applicable. From steve@REDACTED Thu Jun 2 14:23:23 2011 From: steve@REDACTED (Steve Strong) Date: Thu, 2 Jun 2011 14:23:23 +0200 Subject: [erlang-questions] Understanding supervisor / start_link behaviour In-Reply-To: References: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> Message-ID: <4FCB9C50D02B4DF8B2E6DE67576554CB@srstrong.com> That makes a good deal of sense. I guess the point that something should get promoted up to a supervision tree rather than being start-linked is when it starts getting to a complexity level such that it may have issues if multiple instances of the process are running simultaneously. At that point it stops sound like a trivial helper process and something that should be managed more actively. Completely agree on the fact that having the gen_event register wasn't a useful thing, and that not doing so would solve the problem - that was pretty obvious as soon as I spotted the issue, this thread was more to get opinion on how things should be best structured. -- Steve Strong, Director, id3as twitter.com/srstrong On Thursday, 2 June 2011 at 11:53, Mazen Harake wrote: > Steve, > > I wouldn't say that you are wrong. I think that you are reasoning good > about not putting the gen_event module under a supervisor because > *that is what links are for*. Just because you have a supervisor > doesn't mean the you shove everything underneath there! If the > gen_server and the gen_event are truly linked (meaning: gen_server > doesn't act as a "supervisor" keeping track of its gen_event process > and restarts it all the time but rather that they really are linked > and they crash together) then your approach, in my opinion, is good. > > There are great benefits in doing it in that way. Many will claim that > it is best practice to put *everything* under a supervisor but this is > simply not true. 90% of cases it *is* the best thing to do and many > times it is more about how you designed your application rather than > where to put the supervisors and their children but doing it the way > you did is not necessarily wrong. > > The only problem I see with your approach is that you have registered > the gen_event process which clearly isn't useful (since only the > gen_server should know about it, after all, it started it). Other than > that, this approach is extremely helpful and a nice way to clean up > things after they die/shutdown (Again: assuming truly linked). > > There is a big misconception in the community that everything > should/must look like the supervisor-tree model which shows how > gen_servers are put under supervisors and more supervisors under the > "top" supervisor but that is not enforced and the design principles > doesn't take many cases into account where this setup actually brings > more headache to the table than to just exit and clean up using linked > processes (because they do exist). > > /M > > On 1 June 2011 21:26, Steve Strong wrote: > > Hi, > > > > I've got some strange behaviour with gen_event within a supervision tree > > which I don't fully understand. Consider the following supervisor > > (completely standard, feel free to skip over): > > > > -module(sup). > > -behaviour(supervisor). > > -export([start_link/0, init/1]). > > -define(SERVER, ?MODULE). > > start_link() -> > > supervisor:start_link({local, ?SERVER}, ?MODULE, []). > > init([]) -> > > Child1 = {child, {child, start_link, []}, permanent, 2000, worker, > > [child]}, > > {ok, {{one_for_all, 1000, 3600}, [Child1]}}. > > > > and corresponding gen_server (interesting code in bold): > > > > -module(child). > > -behaviour(gen_server). > > -export([start_link/0, init/1, handle_call/3, handle_cast/2, > > handle_info/2, terminate/2, code_change/3]). > > start_link() -> > > gen_server:start_link({local, child}, child, [], []). > > init([]) -> > > io:format("about to start gen_event~n"), > > X = gen_event:start_link({local, my_gen_event}), > > io:format("gen_event started with ~p~n", [X]), > > {ok, _Pid} = X, > > {ok, {}, 2000}. > > handle_call(_Request, _From, State) -> > > {reply, ok, State}. > > handle_cast(_Msg, State) -> > > {noreply, State}. > > handle_info(_Info, State) -> > > io:format("about to crash...~n"), > > 1 = 2, > > {noreply, State}. > > terminate(_Reason, _State) -> > > ok. > > code_change(_OldVsn, State, _Extra) -> > > {ok, State}. > > > > If I run this from an erl shell like this: > > > > --> erl > > Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] > > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.2 (abort with ^G) > > 1> application:start(sasl), supervisor:start_link(sup, []). > > > > > > Then the supervisor & server start as expected. After 2 seconds the server > > gets a timeout message and crashes itself; the supervisor obviously spots > > this and restarts it. Within the init of the gen_server, it also does a > > start_link on a gen_event process. By my understanding, whenever the > > gen_server process exits, the gen_event will also be terminated. > > However, every now and then I see the following output (a ton of sasl trace > > omitted for clarity!): > > > > about to crash... > > about to start gen_event > > gen_event started with {error,{already_started,<0.79.0>}} > > about to start gen_event > > gen_event started with {error,{already_started,<0.79.0>}} > > about to start gen_event > > > > What is happening is that the gen_server is crashing but on its restart the > > gen_event process is still running - hence the gen_server fails in its init > > and gets restarted again. Sometimes this loop clears after a few > > iterations, other times it can continue until the parent supervisor gives > > up, packs its bags and goes home. > > So, my question is whether this is expected behaviour or not. I assume that > > the termination of the linked child is happening asynchronously, and that > > the supervisor is hence restarting its children before things have cleaned > > up correctly - is that correct? > > I can fix this particular scenario by trapping exits within the gen_server, > > and then calling gen_event:stop within the terminate. Is this type of > > processing necessary whenever a process is start_link'ed within a supervisor > > tree, or is what I'm doing considered bad practice? > > Thanks for your time, > > Steve > > -- > > Steve Strong, Director, id3as > > twitter.com/srstrong (http://twitter.com/srstrong) > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From hynek@REDACTED Thu Jun 2 14:46:28 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Thu, 2 Jun 2011 14:46:28 +0200 Subject: [erlang-questions] Tool for performace evaluation In-Reply-To: References: <315503444.211231306153617444.JavaMail.root@zimbra> Message-ID: Try use -X option to your ssh command. Don't forget that X11 protocol forwarding is also permitted in sshd configuration (X11Forwarding option in sshd_config). On Tue, May 31, 2011 at 8:32 AM, Andy W. Song wrote: > I run Erlang on a remote machine through SSH. When I fire up etop it get: > 1> etop:start(). > Application initialization failed: no display name and no $DISPLAY > environment variable > Error in startup script: no display name and no $DISPLAY environment > variable > while executing > "load /usr/lib/libtk8.4.so.0 Tk" > ("package ifneeded" script) > invoked from within > "package require Tk 8.3" > (file "/usr/lib/erlang/lib/gs-1.5.12/priv/gstk.tcl" line 7) > How can I use text mode? > Thanks > Andy > On Mon, May 23, 2011 at 8:26 PM, Gustav Simonsson > wrote: >> >> etop:start(). >> >> gives information similar to 'top' in UNIX: >> http://www.erlang.org/doc/man/etop.html >> >> Regards, >> Gustav Simonsson >> >> ----- Original Message ----- >> From: "Muhammad Yousaf" >> To: erlang-questions@REDACTED >> Sent: Monday, May 23, 2011 1:00:32 PM GMT +01:00 Amsterdam / Berlin / Bern >> / Rome / Stockholm / Vienna >> Subject: [erlang-questions] Tool for performace evaluation >> >> >> Hi, >> >> Is there any tools to evaluate the performance of a module in Erlang that >> can show graphical interface for time elapse, memory usage, performance >> benchmarks etc ?? >> >> or is there any alternative to do this in Erlang >> >> Thanks in advance >> >> Regards, >> >> >> Muhammad Yousaf >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss. Be a data hero! Try GoodData now for free: www.gooddata.com From spawn.think@REDACTED Thu Jun 2 15:20:01 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Thu, 2 Jun 2011 15:20:01 +0200 Subject: [erlang-questions] Tool for performace evaluation In-Reply-To: References: <315503444.211231306153617444.JavaMail.root@zimbra> Message-ID: As a note, you can start etop in text mode etop:start([{output, text}]). 2011/6/2 Hynek Vychodil > Try use -X option to your ssh command. Don't forget that X11 protocol > forwarding is also permitted in sshd configuration (X11Forwarding > option in sshd_config). > > On Tue, May 31, 2011 at 8:32 AM, Andy W. Song wrote: > > I run Erlang on a remote machine through SSH. When I fire up etop it get: > > 1> etop:start(). > > Application initialization failed: no display name and no $DISPLAY > > environment variable > > Error in startup script: no display name and no $DISPLAY environment > > variable > > while executing > > "load /usr/lib/libtk8.4.so.0 Tk" > > ("package ifneeded" script) > > invoked from within > > "package require Tk 8.3" > > (file "/usr/lib/erlang/lib/gs-1.5.12/priv/gstk.tcl" line 7) > > How can I use text mode? > > Thanks > > Andy > > On Mon, May 23, 2011 at 8:26 PM, Gustav Simonsson > > wrote: > >> > >> etop:start(). > >> > >> gives information similar to 'top' in UNIX: > >> http://www.erlang.org/doc/man/etop.html > >> > >> Regards, > >> Gustav Simonsson > >> > >> ----- Original Message ----- > >> From: "Muhammad Yousaf" > >> To: erlang-questions@REDACTED > >> Sent: Monday, May 23, 2011 1:00:32 PM GMT +01:00 Amsterdam / Berlin / > Bern > >> / Rome / Stockholm / Vienna > >> Subject: [erlang-questions] Tool for performace evaluation > >> > >> > >> Hi, > >> > >> Is there any tools to evaluate the performance of a module in Erlang > that > >> can show graphical interface for time elapse, memory usage, performance > >> benchmarks etc ?? > >> > >> or is there any alternative to do this in Erlang > >> > >> Thanks in advance > >> > >> Regards, > >> > >> > >> Muhammad Yousaf > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > > --------------------------------------------------------------- > > ???????????????????? > > ???????????????????? > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try GoodData now for free: www.gooddata.com > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think -------------- next part -------------- An HTML attachment was scrubbed... URL: From wsongcn@REDACTED Thu Jun 2 15:27:19 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Thu, 2 Jun 2011 21:27:19 +0800 Subject: [erlang-questions] Tool for performace evaluation In-Reply-To: References: <315503444.211231306153617444.JavaMail.root@zimbra> Message-ID: Great. Thanks. 2011/6/2 Ahmed Omar > As a note, you can start etop in text mode > etop:start([{output, text}]). > > > 2011/6/2 Hynek Vychodil > >> Try use -X option to your ssh command. Don't forget that X11 protocol >> forwarding is also permitted in sshd configuration (X11Forwarding >> option in sshd_config). >> >> On Tue, May 31, 2011 at 8:32 AM, Andy W. Song wrote: >> > I run Erlang on a remote machine through SSH. When I fire up etop it >> get: >> > 1> etop:start(). >> > Application initialization failed: no display name and no $DISPLAY >> > environment variable >> > Error in startup script: no display name and no $DISPLAY environment >> > variable >> > while executing >> > "load /usr/lib/libtk8.4.so.0 Tk" >> > ("package ifneeded" script) >> > invoked from within >> > "package require Tk 8.3" >> > (file "/usr/lib/erlang/lib/gs-1.5.12/priv/gstk.tcl" line 7) >> > How can I use text mode? >> > Thanks >> > Andy >> > On Mon, May 23, 2011 at 8:26 PM, Gustav Simonsson >> > wrote: >> >> >> >> etop:start(). >> >> >> >> gives information similar to 'top' in UNIX: >> >> http://www.erlang.org/doc/man/etop.html >> >> >> >> Regards, >> >> Gustav Simonsson >> >> >> >> ----- Original Message ----- >> >> From: "Muhammad Yousaf" >> >> To: erlang-questions@REDACTED >> >> Sent: Monday, May 23, 2011 1:00:32 PM GMT +01:00 Amsterdam / Berlin / >> Bern >> >> / Rome / Stockholm / Vienna >> >> Subject: [erlang-questions] Tool for performace evaluation >> >> >> >> >> >> Hi, >> >> >> >> Is there any tools to evaluate the performance of a module in Erlang >> that >> >> can show graphical interface for time elapse, memory usage, performance >> >> benchmarks etc ?? >> >> >> >> or is there any alternative to do this in Erlang >> >> >> >> Thanks in advance >> >> >> >> Regards, >> >> >> >> >> >> Muhammad Yousaf >> >> >> >> >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > >> > -- >> > --------------------------------------------------------------- >> > ???????????????????? >> > ???????????????????? >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> >> >> >> -- >> --Hynek (Pichi) Vychodil >> >> Analyze your data in minutes. Share your insights instantly. Thrill >> your boss. Be a data hero! >> Try GoodData now for free: www.gooddata.com >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > > -- --------------------------------------------------------------- ???????????????????? ???????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From freza@REDACTED Thu Jun 2 15:45:29 2011 From: freza@REDACTED (Jachym Holecek) Date: Thu, 2 Jun 2011 14:45:29 +0100 Subject: [erlang-questions] Erlang and Memory management In-Reply-To: References: <961326352.243381306879495091.JavaMail.root@zimbra> <1091703077.243401306880078397.JavaMail.root@zimbra> Message-ID: <20110602134529.GA6438@hanele> # Jesper Louis Andersen 2011-06-01: > On Wed, Jun 1, 2011 at 15:49, Michael Turner > wrote: > > Copying a data structure is also an opportunity to construct a "fresh" data > > structure that's relatively unfragmented. Lower fragmentation of data > > structures would make cache fetches more efficient, and cache hits hits more > > likely, for the receiver. > > Garbage Collection will usually unfragment data anyway. But the > process heap acts as a crude region-based memory manager: When a > process dies, we can *instantly* reset its memory area and give it > back. And that kind of cleanup is really effective. This gives us a > region of memory of which we have some control, due to the lifetime of > the process. In a shared heap, you loose that control and have to wait > until the next GC for the memory to be reclaimed. The fact that you > can tune the GC of each process rather than for the whole VM is also a > pretty good thing. Well, sometimes you know your process acts as a lightweight message dispatcher (do very little work on received requests and then forward them to some other process), other times you know your process is a transient worker with heavy memory consumption... So I guess it would be sweet to have both, and select suitable strategy at process creation time. Not that I'm volunteering to implement that though :-). BR, -- Jachym From fred.hebert@REDACTED Thu Jun 2 16:10:53 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Thu, 2 Jun 2011 10:10:53 -0400 Subject: [erlang-questions] Understanding supervisor / start_link behaviour In-Reply-To: References: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> Message-ID: There are disadvantages to *not* putting workers under the supervision tree, though. Namely, you'll be losing the ability to have the release handlers walk down the supervision trees to find which processes to suspend/update, and you'll then need to find a different way of doing things. This is a serious point to consider if you ever plan on going the way of releases/appups if the workers you use are to be long-lived (you don't want them to be killed during a purge). I'm not saying you didn't know this, but I felt I should point it out for the sake of having the arguments clear on the mailing list. -- Fred H?bert http://www.erlang-solutions.com On 2011-06-02, at 05:53 AM, Mazen Harake wrote: > Steve, > > I wouldn't say that you are wrong. I think that you are reasoning good > about not putting the gen_event module under a supervisor because > *that is what links are for*. Just because you have a supervisor > doesn't mean the you shove everything underneath there! If the > gen_server and the gen_event are truly linked (meaning: gen_server > doesn't act as a "supervisor" keeping track of its gen_event process > and restarts it all the time but rather that they really are linked > and they crash together) then your approach, in my opinion, is good. > > There are great benefits in doing it in that way. Many will claim that > it is best practice to put *everything* under a supervisor but this is > simply not true. 90% of cases it *is* the best thing to do and many > times it is more about how you designed your application rather than > where to put the supervisors and their children but doing it the way > you did is not necessarily wrong. > > The only problem I see with your approach is that you have registered > the gen_event process which clearly isn't useful (since only the > gen_server should know about it, after all, it started it). Other than > that, this approach is extremely helpful and a nice way to clean up > things after they die/shutdown (Again: assuming truly linked). > > There is a big misconception in the community that everything > should/must look like the supervisor-tree model which shows how > gen_servers are put under supervisors and more supervisors under the > "top" supervisor but that is not enforced and the design principles > doesn't take many cases into account where this setup actually brings > more headache to the table than to just exit and clean up using linked > processes (because they do exist). > > /M > > On 1 June 2011 21:26, Steve Strong wrote: >> Hi, >> >> I've got some strange behaviour with gen_event within a supervision tree >> which I don't fully understand. Consider the following supervisor >> (completely standard, feel free to skip over): >> >> -module(sup). >> -behaviour(supervisor). >> -export([start_link/0, init/1]). >> -define(SERVER, ?MODULE). >> start_link() -> >> supervisor:start_link({local, ?SERVER}, ?MODULE, []). >> init([]) -> >> Child1 = {child, {child, start_link, []}, permanent, 2000, worker, >> [child]}, >> {ok, {{one_for_all, 1000, 3600}, [Child1]}}. >> >> and corresponding gen_server (interesting code in bold): >> >> -module(child). >> -behaviour(gen_server). >> -export([start_link/0, init/1, handle_call/3, handle_cast/2, >> handle_info/2, terminate/2, code_change/3]). >> start_link() -> >> gen_server:start_link({local, child}, child, [], []). >> init([]) -> >> io:format("about to start gen_event~n"), >> X = gen_event:start_link({local, my_gen_event}), >> io:format("gen_event started with ~p~n", [X]), >> {ok, _Pid} = X, >> {ok, {}, 2000}. >> handle_call(_Request, _From, State) -> >> {reply, ok, State}. >> handle_cast(_Msg, State) -> >> {noreply, State}. >> handle_info(_Info, State) -> >> io:format("about to crash...~n"), >> 1 = 2, >> {noreply, State}. >> terminate(_Reason, _State) -> >> ok. >> code_change(_OldVsn, State, _Extra) -> >> {ok, State}. >> >> If I run this from an erl shell like this: >> >> --> erl >> Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] >> [async-threads:0] [hipe] [kernel-poll:false] >> Eshell V5.8.2 (abort with ^G) >> 1> application:start(sasl), supervisor:start_link(sup, []). >> >> >> Then the supervisor & server start as expected. After 2 seconds the server >> gets a timeout message and crashes itself; the supervisor obviously spots >> this and restarts it. Within the init of the gen_server, it also does a >> start_link on a gen_event process. By my understanding, whenever the >> gen_server process exits, the gen_event will also be terminated. >> However, every now and then I see the following output (a ton of sasl trace >> omitted for clarity!): >> >> about to crash... >> about to start gen_event >> gen_event started with {error,{already_started,<0.79.0>}} >> about to start gen_event >> gen_event started with {error,{already_started,<0.79.0>}} >> about to start gen_event >> >> What is happening is that the gen_server is crashing but on its restart the >> gen_event process is still running - hence the gen_server fails in its init >> and gets restarted again. Sometimes this loop clears after a few >> iterations, other times it can continue until the parent supervisor gives >> up, packs its bags and goes home. >> So, my question is whether this is expected behaviour or not. I assume that >> the termination of the linked child is happening asynchronously, and that >> the supervisor is hence restarting its children before things have cleaned >> up correctly - is that correct? >> I can fix this particular scenario by trapping exits within the gen_server, >> and then calling gen_event:stop within the terminate. Is this type of >> processing necessary whenever a process is start_link'ed within a supervisor >> tree, or is what I'm doing considered bad practice? >> Thanks for your time, >> Steve >> -- >> Steve Strong, Director, id3as >> twitter.com/srstrong >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From steve@REDACTED Thu Jun 2 16:23:40 2011 From: steve@REDACTED (Steve Strong) Date: Thu, 2 Jun 2011 16:23:40 +0200 Subject: [erlang-questions] Understanding supervisor / start_link behaviour In-Reply-To: References: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> Message-ID: That is an interesting point, and not something I'd considered to date -- Steve Strong Sent with Sparrow (http://www.sparrowmailapp.com) On Thursday, June 2, 2011 at 4:10 PM, Fr?d?ric Trottier-H?bert wrote: > There are disadvantages to *not* putting workers under the supervision tree, though. Namely, you'll be losing the ability to have the release handlers walk down the supervision trees to find which processes to suspend/update, and you'll then need to find a different way of doing things. > > This is a serious point to consider if you ever plan on going the way of releases/appups if the workers you use are to be long-lived (you don't want them to be killed during a purge). I'm not saying you didn't know this, but I felt I should point it out for the sake of having the arguments clear on the mailing list. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > On 2011-06-02, at 05:53 AM, Mazen Harake wrote: > > > Steve, > > > > I wouldn't say that you are wrong. I think that you are reasoning good > > about not putting the gen_event module under a supervisor because > > *that is what links are for*. Just because you have a supervisor > > doesn't mean the you shove everything underneath there! If the > > gen_server and the gen_event are truly linked (meaning: gen_server > > doesn't act as a "supervisor" keeping track of its gen_event process > > and restarts it all the time but rather that they really are linked > > and they crash together) then your approach, in my opinion, is good. > > > > There are great benefits in doing it in that way. Many will claim that > > it is best practice to put *everything* under a supervisor but this is > > simply not true. 90% of cases it *is* the best thing to do and many > > times it is more about how you designed your application rather than > > where to put the supervisors and their children but doing it the way > > you did is not necessarily wrong. > > > > The only problem I see with your approach is that you have registered > > the gen_event process which clearly isn't useful (since only the > > gen_server should know about it, after all, it started it). Other than > > that, this approach is extremely helpful and a nice way to clean up > > things after they die/shutdown (Again: assuming truly linked). > > > > There is a big misconception in the community that everything > > should/must look like the supervisor-tree model which shows how > > gen_servers are put under supervisors and more supervisors under the > > "top" supervisor but that is not enforced and the design principles > > doesn't take many cases into account where this setup actually brings > > more headache to the table than to just exit and clean up using linked > > processes (because they do exist). > > > > /M > > > > On 1 June 2011 21:26, Steve Strong wrote: > > > Hi, > > > > > > I've got some strange behaviour with gen_event within a supervision tree > > > which I don't fully understand. Consider the following supervisor > > > (completely standard, feel free to skip over): > > > > > > -module(sup). > > > -behaviour(supervisor). > > > -export([start_link/0, init/1]). > > > -define(SERVER, ?MODULE). > > > start_link() -> > > > supervisor:start_link({local, ?SERVER}, ?MODULE, []). > > > init([]) -> > > > Child1 = {child, {child, start_link, []}, permanent, 2000, worker, > > > [child]}, > > > {ok, {{one_for_all, 1000, 3600}, [Child1]}}. > > > > > > and corresponding gen_server (interesting code in bold): > > > > > > -module(child). > > > -behaviour(gen_server). > > > -export([start_link/0, init/1, handle_call/3, handle_cast/2, > > > handle_info/2, terminate/2, code_change/3]). > > > start_link() -> > > > gen_server:start_link({local, child}, child, [], []). > > > init([]) -> > > > io:format("about to start gen_event~n"), > > > X = gen_event:start_link({local, my_gen_event}), > > > io:format("gen_event started with ~p~n", [X]), > > > {ok, _Pid} = X, > > > {ok, {}, 2000}. > > > handle_call(_Request, _From, State) -> > > > {reply, ok, State}. > > > handle_cast(_Msg, State) -> > > > {noreply, State}. > > > handle_info(_Info, State) -> > > > io:format("about to crash...~n"), > > > 1 = 2, > > > {noreply, State}. > > > terminate(_Reason, _State) -> > > > ok. > > > code_change(_OldVsn, State, _Extra) -> > > > {ok, State}. > > > > > > If I run this from an erl shell like this: > > > > > > --> erl > > > Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] > > > [async-threads:0] [hipe] [kernel-poll:false] > > > Eshell V5.8.2 (abort with ^G) > > > 1> application:start(sasl), supervisor:start_link(sup, []). > > > > > > > > > Then the supervisor & server start as expected. After 2 seconds the server > > > gets a timeout message and crashes itself; the supervisor obviously spots > > > this and restarts it. Within the init of the gen_server, it also does a > > > start_link on a gen_event process. By my understanding, whenever the > > > gen_server process exits, the gen_event will also be terminated. > > > However, every now and then I see the following output (a ton of sasl trace > > > omitted for clarity!): > > > > > > about to crash... > > > about to start gen_event > > > gen_event started with {error,{already_started,<0.79.0>}} > > > about to start gen_event > > > gen_event started with {error,{already_started,<0.79.0>}} > > > about to start gen_event > > > > > > What is happening is that the gen_server is crashing but on its restart the > > > gen_event process is still running - hence the gen_server fails in its init > > > and gets restarted again. Sometimes this loop clears after a few > > > iterations, other times it can continue until the parent supervisor gives > > > up, packs its bags and goes home. > > > So, my question is whether this is expected behaviour or not. I assume that > > > the termination of the linked child is happening asynchronously, and that > > > the supervisor is hence restarting its children before things have cleaned > > > up correctly - is that correct? > > > I can fix this particular scenario by trapping exits within the gen_server, > > > and then calling gen_event:stop within the terminate. Is this type of > > > processing necessary whenever a process is start_link'ed within a supervisor > > > tree, or is what I'm doing considered bad practice? > > > Thanks for your time, > > > Steve > > > -- > > > Steve Strong, Director, id3as > > > twitter.com/srstrong (http://twitter.com/srstrong) > > > > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From systemio@REDACTED Thu Jun 2 16:27:48 2011 From: systemio@REDACTED (systemio systemio) Date: Thu, 02 Jun 2011 18:27:48 +0400 Subject: [erlang-questions] there is a blocking in the process evaluating the huge factorial Message-ID: Hi all, My Erlang is (Erlang R13B03 (erts-5.7.4) [smp:2:2] [rq:2] [async-threads:0]). WinXP SP1. I've found the strange behavior when i spawn the process to evaluate the huge factorial. Here is the factorial module: -module(test). -compile(export_all). fac(0) -> 1; fac(N) -> N * fac(N-1). Here is the Erlang Shell code: 1> Fac = fun() -> test:fac(100000) end. #Fun 2> S1 = fun()-> spawn(Fac), timer:sleep(3000), io:format("Will I am", []), spawn(Fac) end. #Fun 3> S2 = fun()-> spawn(Fac), io:format("Will I am", []), spawn(Fac) end. #Fun 4> S1(). Will I am<0.38.0> 5> S2(). Will I am<0.42.0> In my PC the fun Fac evaluates about 11 sec. If i start S1() i will see the message "Will I am" after 14 sec and it is strange as i expect it after 3 seconds. After message appeares i type "i()." and hit enter. This command is blocked and result appeares after 11 sec until second process evaluates. if i start S2() i will see the message "Will i am" immediately but i will not be able to type anything in the shell and my typing will appear after 22 sec. Also i've tested it with spawn(Node, ...) and the result is IAE same. Are these behaviors valid or do i do smth wrong? ~ Alex From freza@REDACTED Thu Jun 2 16:37:39 2011 From: freza@REDACTED (Jachym Holecek) Date: Thu, 2 Jun 2011 15:37:39 +0100 Subject: [erlang-questions] Understanding supervisor / start_link behaviour In-Reply-To: References: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> Message-ID: <20110602143739.GB6438@hanele> # Mazen Harake 2011-06-02: > I wouldn't say that you are wrong. I think that you are reasoning good > about not putting the gen_event module under a supervisor because > *that is what links are for*. Just because you have a supervisor > doesn't mean the you shove everything underneath there! If the > gen_server and the gen_event are truly linked (meaning: gen_server > doesn't act as a "supervisor" keeping track of its gen_event process > and restarts it all the time but rather that they really are linked > and they crash together) then your approach, in my opinion, is good. FWIW couldn't agree more with this. For completeness (it's obvious and you're no doubt aware of it): 'normal' exits don't kill linked peers, which takes a little getting used to, but is trivial to manage. As a more general point, designing sensible supervision trees was probably the most difficult engineering aspect of OTP for me to learn, so I guess people shouldn't feel too bad if it feels intimidating initially. :-) BR, -- Jachym From playground@REDACTED Thu Jun 2 17:01:50 2011 From: playground@REDACTED (wuchubuzai) Date: Thu, 2 Jun 2011 08:01:50 -0700 (PDT) Subject: [erlang-questions] Erlang Programmer required... Message-ID: <1307026910539-3568554.post@n4.nabble.com> Erlangers, We need to duplicate the MySQL module from ejabberd so that it works with Apache Cassandra via the thrift Erlang library / generated code. If you are interested in working on this project please respond and let us know. We are eager to get started and "yes" the final code will be open sourced to give back to the community. However, for clarity sake, "yes" you will be compensated as well as giving the gift of code to the world ;-) Thanks in advance. Scott -- View this message in context: http://erlang.2086793.n4.nabble.com/Erlang-Programmer-required-tp3568554p3568554.html Sent from the Erlang Questions mailing list archive at Nabble.com. From moxford@REDACTED Thu Jun 2 20:54:43 2011 From: moxford@REDACTED (Mike Oxford) Date: Thu, 2 Jun 2011 11:54:43 -0700 Subject: [erlang-questions] there is a blocking in the process evaluating the huge factorial In-Reply-To: References: Message-ID: As a guess; io:format sends the message to the group leader for output. If your group leader were to change based on the spawn() call, you may have your io:format message in the run queue behind the fac call. More information may be found here... http://www.trapexit.org/forum/viewtopic.php?p=43855 Again, just a guess. :) G'luck! -mox On Thu, Jun 2, 2011 at 7:27 AM, systemio systemio wrote: > Hi all, > > My Erlang is (Erlang R13B03 (erts-5.7.4) [smp:2:2] [rq:2] > [async-threads:0]). > WinXP SP1. > > I've found the strange behavior when i spawn the process to evaluate the > huge > factorial. > > Here is the factorial module: > -module(test). > -compile(export_all). > > fac(0) -> > 1; > fac(N) -> > N * fac(N-1). > > Here is the Erlang Shell code: > 1> Fac = fun() -> test:fac(100000) end. > #Fun > 2> S1 = fun()-> spawn(Fac), timer:sleep(3000), io:format("Will I am", []), > spawn(Fac) end. > #Fun > 3> S2 = fun()-> spawn(Fac), io:format("Will I am", []), spawn(Fac) end. > #Fun > 4> S1(). > Will I am<0.38.0> > 5> S2(). > Will I am<0.42.0> > > In my PC the fun Fac evaluates about 11 sec. > > If i start S1() i will see the message "Will I am" after 14 sec and it is > strange as i expect it after 3 seconds. After message appeares i type > "i()." > and hit enter. This command is blocked and result appeares after 11 sec > until > second process evaluates. > > if i start S2() i will see the message "Will i am" immediately but i will > not > be able to type anything in the shell and my typing will appear after 22 > sec. > > Also i've tested it with spawn(Node, ...) and the result is IAE same. > > Are these behaviors valid or do i do smth wrong? > > ~ > Alex > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Thu Jun 2 21:09:40 2011 From: bob@REDACTED (Bob Cowdery) Date: Thu, 02 Jun 2011 20:09:40 +0100 Subject: [erlang-questions] Losing data Message-ID: <4DE7DFF4.3020809@bobcowdery.plus.com> Can anyone give me some hints on how to debug this problem. I know its a long-shot but maybe someone has experience of a similar problem. I have a real-time system that reads from a device using UDP and writes back to the same device using UDP. The data processing chain comprises 6 processes: Reader - Decoder - Input to linked-in-driver - output from linked-in-driver - Encoder - Writer With the parameters I'm using I should be writing the same number of blocks that I am reading. There are sequence numbers on both input and output blocks. For a while this is what happens, then I start to lose blocks. I believe in snatches rather than a regular pattern. I can sometimes hear the disruption in the output and if I stop at that point I've lost blocks, as many as 40 in one case. However it can run on way longer than that with no lost blocks. I've checked every stage of the code with debug statements but everything seems to pass through correctly. I've also checked the message queue length at each stage and it never builds up on any process. I have no catch-all message handlers. The messages are all large binary. The symptoms look like it just can't keep up but If a process was getting behind I would expect the message q to build up. CPU is 5-7%. Any ideas would be very welcome. Bob From jesper.louis.andersen@REDACTED Thu Jun 2 21:20:52 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 2 Jun 2011 21:20:52 +0200 Subject: [erlang-questions] Losing data In-Reply-To: <4DE7DFF4.3020809@bobcowdery.plus.com> References: <4DE7DFF4.3020809@bobcowdery.plus.com> Message-ID: On Thu, Jun 2, 2011 at 21:09, Bob Cowdery wrote: > > The symptoms look like it just can't keep up but If a process was > getting behind I would expect the message q to build up. CPU is 5-7%. > > Any ideas would be very welcome. > Ladies and Gentlemen! Welcome to the Circus of the Kernel! Tonight we will have artist of all kinds: Jugglers! Trapezoids! Old Erlang programmers. A balancing act with a VAX! And above all, there will be UDP throwing! Your problem is that you are sending data out too fast and the kernel is then throwing your data away. This is especially true on Linux where this happens silently. On FreeBSD I have seen ENOBUFS being sent back when this happen, but Linux just choses to silently drop packets. The only way to fix this is to pace packets so they are sent at a slower rate - and build in a retransmit mechanism. It is the only way to be sure. Note that the send buffer can be ridicously small on some systems, so you can't rely on it. Though my code using UDP can surely send around 8K on Linux without too much trouble. Since the queue is tail-end dropping you see the problem where many packets are lost suddenly. The queue fill up, and them you are in for the surprise when the next 30 packets are just skipped and dropped. There are a couple of fixes depending on what you want to do, but if possible go check out SCTP which can give you many of the things you want from UDP. It is kind of the hybrid between UDP and TCP and then not. it is also a very interesting protocol - check it out! -- J. From moxford@REDACTED Thu Jun 2 21:26:38 2011 From: moxford@REDACTED (Mike Oxford) Date: Thu, 2 Jun 2011 12:26:38 -0700 Subject: [erlang-questions] Losing data In-Reply-To: <4DE7DFF4.3020809@bobcowdery.plus.com> References: <4DE7DFF4.3020809@bobcowdery.plus.com> Message-ID: Can you pin down where you think you're losing the data? You state a hard number....where in the chain are you measuring? Usually in audio-processing you're going to run into issues in two main places; buffer starvation and codec corruption. Buffer starvation is pretty self-explanitory -- something in the chain cannot keep up, usually due to CPU load, lock contention or other data flow interruption. Codec corruption happens when the codec tries to use the new incoming packets against it's stored run-state. If something happens (out of sequence, null packets, etc) the codec will usually do one of two things: do its best and give you corrupted audio or just drop the packet on the floor as if it never got it. One more caveat with audio: output packet size is not usually equal to input packet size. The most codecs do lossy-compression. Just one more thing to check. -mox On Thu, Jun 2, 2011 at 12:09 PM, Bob Cowdery wrote: > Can anyone give me some hints on how to debug this problem. I know its a > long-shot but maybe someone has experience of a similar problem. > > I have a real-time system that reads from a device using UDP and writes > back to the same device using UDP. The data processing chain comprises 6 > processes: > > Reader - Decoder - Input to linked-in-driver - output from > linked-in-driver - Encoder - Writer > > With the parameters I'm using I should be writing the same number of > blocks that I am reading. There are sequence numbers on both input and > output blocks. For a while this is what happens, then I start to lose > blocks. I believe in snatches rather than a regular pattern. I can > sometimes hear the disruption in the output and if I stop at that point > I've lost blocks, as many as 40 in one case. However it can run on way > longer than that with no lost blocks. > > I've checked every stage of the code with debug statements but > everything seems to pass through correctly. I've also checked the > message queue length at each stage and it never builds up on any > process. I have no catch-all message handlers. The messages are all > large binary. > > The symptoms look like it just can't keep up but If a process was > getting behind I would expect the message q to build up. CPU is 5-7%. > > Any ideas would be very welcome. > > Bob > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Thu Jun 2 23:02:20 2011 From: bob@REDACTED (Bob Cowdery) Date: Thu, 02 Jun 2011 22:02:20 +0100 Subject: [erlang-questions] Losing data In-Reply-To: References: <4DE7DFF4.3020809@bobcowdery.plus.com> Message-ID: <4DE7FA5C.3060506@bobcowdery.plus.com> On 02/06/2011 20:20, Jesper Louis Andersen wrote: > On Thu, Jun 2, 2011 at 21:09, Bob Cowdery wrote: > >> The symptoms look like it just can't keep up but If a process was >> getting behind I would expect the message q to build up. CPU is 5-7%. >> >> Any ideas would be very welcome. >> > Ladies and Gentlemen! Welcome to the Circus of the Kernel! Tonight we > will have artist of all kinds: Jugglers! Trapezoids! Old Erlang > programmers. A balancing act with a VAX! And above all, there will be > UDP throwing! > > Your problem is that you are sending data out too fast and the kernel > is then throwing your data away. This is especially true on Linux > where this happens silently. On FreeBSD I have seen ENOBUFS being sent > back when this happen, but Linux just choses to silently drop packets. > The only way to fix this is to pace packets so they are sent at a > slower rate - and build in a retransmit mechanism. It is the only way > to be sure. Note that the send buffer can be ridicously small on some > systems, so you can't rely on it. Though my code using UDP can surely > send around 8K on Linux without too much trouble. If the data was being thrown away after it left the application I would expect to see the input and output sequence numbers keep in step. The input sequence is added by the hardware device and I add the output sequence number before transmission. I check the input sequence and shout if there is a sequence error and the device checks the output sequence and lights a nice little LED if it sees a sequence error. Both show correct sequences so I'm losing blocks inside Erlang somewhere. > Since the queue is tail-end dropping you see the problem where many > packets are lost suddenly. The queue fill up, and them you are in for > the surprise when the next 30 packets are just skipped and dropped. It sounds really plausible if only it fitted the facts. > There are a couple of fixes depending on what you want to do, but if > possible go check out SCTP which can give you many of the things you > want from UDP. It is kind of the hybrid between UDP and TCP and then > not. it is also a very interesting protocol - check it out! > I don't have control over the hardware device so it has to be UDP. Looks an interesting protocol though for future reference. From bob@REDACTED Thu Jun 2 23:02:34 2011 From: bob@REDACTED (Bob Cowdery) Date: Thu, 02 Jun 2011 22:02:34 +0100 Subject: [erlang-questions] Losing data In-Reply-To: References: <4DE7DFF4.3020809@bobcowdery.plus.com> Message-ID: <4DE7FA6A.80406@bobcowdery.plus.com> On 02/06/2011 20:26, Mike Oxford wrote: > Can you pin down where you think you're losing the data? You state a > hard number....where in the chain are you measuring? > I've been trying to do that for several days but it's really hard to track it all the way through in one go because the buffer size changes in different stages and its not even multiples, it only comes together in fixed size blocks at each end. When I check each stage individually I'm failing to find any problem. I see I've lost blocks overall but the stage I'm looking at has no breaks in the pattern. They all seem to process the data correctly - and most of the time the input and output keep pace. There is no code in the path that could just junk blocks. > Usually in audio-processing you're going to run into issues in two > main places; buffer starvation and codec corruption. > Buffer starvation is pretty self-explanitory -- something in the chain > cannot keep up, usually due to CPU load, lock contention or other data > flow interruption. > Codec corruption happens when the codec tries to use the new incoming > packets against it's stored run-state. If > something happens (out of sequence, null packets, etc) the codec will > usually do one of two things: do its best and give you corrupted > audio or just drop the packet on the floor as if it never got it. > This is akin to audio processing but the signal processing is under my control so there are no unknowns in the chain. > One more caveat with audio: output packet size is not usually equal to > input packet size. The most codecs do lossy-compression. Just one > more thing to check. > I'm using 48k input and output rate for the test. > -mox > > On Thu, Jun 2, 2011 at 12:09 PM, Bob Cowdery > wrote: > > Can anyone give me some hints on how to debug this problem. I know > its a > long-shot but maybe someone has experience of a similar problem. > > I have a real-time system that reads from a device using UDP and > writes > back to the same device using UDP. The data processing chain > comprises 6 > processes: > > Reader - Decoder - Input to linked-in-driver - output from > linked-in-driver - Encoder - Writer > > With the parameters I'm using I should be writing the same number of > blocks that I am reading. There are sequence numbers on both input and > output blocks. For a while this is what happens, then I start to lose > blocks. I believe in snatches rather than a regular pattern. I can > sometimes hear the disruption in the output and if I stop at that > point > I've lost blocks, as many as 40 in one case. However it can run on way > longer than that with no lost blocks. > > I've checked every stage of the code with debug statements but > everything seems to pass through correctly. I've also checked the > message queue length at each stage and it never builds up on any > process. I have no catch-all message handlers. The messages are all > large binary. > > The symptoms look like it just can't keep up but If a process was > getting behind I would expect the message q to build up. CPU is 5-7%. > > Any ideas would be very welcome. > > Bob > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From v@REDACTED Thu Jun 2 23:05:25 2011 From: v@REDACTED (Valentin Micic) Date: Thu, 2 Jun 2011 23:05:25 +0200 Subject: [erlang-questions] Losing data In-Reply-To: <4DE7FA5C.3060506@bobcowdery.plus.com> References: <4DE7DFF4.3020809@bobcowdery.plus.com> <4DE7FA5C.3060506@bobcowdery.plus.com> Message-ID: <05D9DEE0-99AF-4E1C-870F-44D5B0BB5075@pharos-avantgard.com> Just increase the UDP buffer size on both machines. BR V/ On 02 Jun 2011, at 11:02 PM, Bob Cowdery wrote: > On 02/06/2011 20:20, Jesper Louis Andersen wrote: >> On Thu, Jun 2, 2011 at 21:09, Bob Cowdery wrote: >> >>> The symptoms look like it just can't keep up but If a process was >>> getting behind I would expect the message q to build up. CPU is 5-7%. >>> >>> Any ideas would be very welcome. >>> >> Ladies and Gentlemen! Welcome to the Circus of the Kernel! Tonight we >> will have artist of all kinds: Jugglers! Trapezoids! Old Erlang >> programmers. A balancing act with a VAX! And above all, there will be >> UDP throwing! >> >> Your problem is that you are sending data out too fast and the kernel >> is then throwing your data away. This is especially true on Linux >> where this happens silently. On FreeBSD I have seen ENOBUFS being sent >> back when this happen, but Linux just choses to silently drop packets. >> The only way to fix this is to pace packets so they are sent at a >> slower rate - and build in a retransmit mechanism. It is the only way >> to be sure. Note that the send buffer can be ridicously small on some >> systems, so you can't rely on it. Though my code using UDP can surely >> send around 8K on Linux without too much trouble. > If the data was being thrown away after it left the application I would > expect to see the input and output sequence numbers keep in step. The > input sequence is added by the hardware device and I add the output > sequence number before transmission. I check the input sequence and > shout if there is a sequence error and the device checks the output > sequence and lights a nice little LED if it sees a sequence error. Both > show correct sequences so I'm losing blocks inside Erlang somewhere. >> Since the queue is tail-end dropping you see the problem where many >> packets are lost suddenly. The queue fill up, and them you are in for >> the surprise when the next 30 packets are just skipped and dropped. > It sounds really plausible if only it fitted the facts. >> There are a couple of fixes depending on what you want to do, but if >> possible go check out SCTP which can give you many of the things you >> want from UDP. It is kind of the hybrid between UDP and TCP and then >> not. it is also a very interesting protocol - check it out! >> > I don't have control over the hardware device so it has to be UDP. Looks > an interesting protocol though for future reference. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bob@REDACTED Thu Jun 2 23:21:24 2011 From: bob@REDACTED (Bob Cowdery) Date: Thu, 02 Jun 2011 22:21:24 +0100 Subject: [erlang-questions] Losing data In-Reply-To: <05D9DEE0-99AF-4E1C-870F-44D5B0BB5075@pharos-avantgard.com> References: <4DE7DFF4.3020809@bobcowdery.plus.com> <4DE7FA5C.3060506@bobcowdery.plus.com> <05D9DEE0-99AF-4E1C-870F-44D5B0BB5075@pharos-avantgard.com> Message-ID: <4DE7FED4.60408@bobcowdery.plus.com> On 02/06/2011 22:05, Valentin Micic wrote: > Just increase the UDP buffer size on both machines. Why would that help? Am I missing something in my analysis? There is only one computer, the other end is dedicated hardware. This is only intended to work on a local network. Bob > BR > > V/ > > On 02 Jun 2011, at 11:02 PM, Bob Cowdery wrote: > >> On 02/06/2011 20:20, Jesper Louis Andersen wrote: >>> On Thu, Jun 2, 2011 at 21:09, Bob Cowdery wrote: >>> >>>> The symptoms look like it just can't keep up but If a process was >>>> getting behind I would expect the message q to build up. CPU is 5-7%. >>>> >>>> Any ideas would be very welcome. >>>> >>> Ladies and Gentlemen! Welcome to the Circus of the Kernel! Tonight we >>> will have artist of all kinds: Jugglers! Trapezoids! Old Erlang >>> programmers. A balancing act with a VAX! And above all, there will be >>> UDP throwing! >>> >>> Your problem is that you are sending data out too fast and the kernel >>> is then throwing your data away. This is especially true on Linux >>> where this happens silently. On FreeBSD I have seen ENOBUFS being sent >>> back when this happen, but Linux just choses to silently drop packets. >>> The only way to fix this is to pace packets so they are sent at a >>> slower rate - and build in a retransmit mechanism. It is the only way >>> to be sure. Note that the send buffer can be ridicously small on some >>> systems, so you can't rely on it. Though my code using UDP can surely >>> send around 8K on Linux without too much trouble. >> If the data was being thrown away after it left the application I would >> expect to see the input and output sequence numbers keep in step. The >> input sequence is added by the hardware device and I add the output >> sequence number before transmission. I check the input sequence and >> shout if there is a sequence error and the device checks the output >> sequence and lights a nice little LED if it sees a sequence error. Both >> show correct sequences so I'm losing blocks inside Erlang somewhere. >>> Since the queue is tail-end dropping you see the problem where many >>> packets are lost suddenly. The queue fill up, and them you are in for >>> the surprise when the next 30 packets are just skipped and dropped. >> It sounds really plausible if only it fitted the facts. >>> There are a couple of fixes depending on what you want to do, but if >>> possible go check out SCTP which can give you many of the things you >>> want from UDP. It is kind of the hybrid between UDP and TCP and then >>> not. it is also a very interesting protocol - check it out! >>> >> I don't have control over the hardware device so it has to be UDP. Looks >> an interesting protocol though for future reference. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From moxford@REDACTED Fri Jun 3 00:12:57 2011 From: moxford@REDACTED (Mike Oxford) Date: Thu, 2 Jun 2011 15:12:57 -0700 Subject: [erlang-questions] Losing data In-Reply-To: <4DE7FA6A.80406@bobcowdery.plus.com> References: <4DE7DFF4.3020809@bobcowdery.plus.com> <4DE7FA6A.80406@bobcowdery.plus.com> Message-ID: I'm guessing you've already tried increasing SASL debugging, and using dbg:tracer()? How about putting a debugger (gdb or the like) on your driver? Have you tried synthetic testing (eg, loop the same packet into your system, injected just after the UDP layer and then check the output on the other side?" You say that you're breaking the packets up for processing, and putting them back together, with sequencing staying the same... but you're losing packets? That seems only possible if your hardware isn't actually sending it (or there's a bug in your sequencing numbers/checks.) Have you put Ethereal/Wireshark/tcpdump etc on the wire during the tests? Have you seen a pause in output? Hard to tell from this far away. =/ On Thu, Jun 2, 2011 at 2:02 PM, Bob Cowdery wrote: > On 02/06/2011 20:26, Mike Oxford wrote: > > Can you pin down where you think you're losing the data? You state a > hard number....where in the chain are you measuring? > > I've been trying to do that for several days but it's really hard to > track it all the way through in one go because the buffer size changes in > different stages and its not even multiples, it only comes together in fixed > size blocks at each end. When I check each stage individually I'm failing to > find any problem. I see I've lost blocks overall but the stage I'm looking > at has no breaks in the pattern. They all seem to process the data correctly > - and most of the time the input and output keep pace. There is no code in > the path that could just junk blocks. > > Usually in audio-processing you're going to run into issues in two main > places; buffer starvation and codec corruption. > Buffer starvation is pretty self-explanitory -- something in the chain > cannot keep up, usually due to CPU load, lock contention or other data flow > interruption. > Codec corruption happens when the codec tries to use the new incoming > packets against it's stored run-state. If > something happens (out of sequence, null packets, etc) the codec will > usually do one of two things: do its best and give you corrupted audio or > just drop the packet on the floor as if it never got it. > > This is akin to audio processing but the signal processing is under my > control so there are no unknowns in the chain. > > One more caveat with audio: output packet size is not usually equal to > input packet size. The most codecs do lossy-compression. Just one more > thing to check. > > I'm using 48k input and output rate for the test. > > -mox > > On Thu, Jun 2, 2011 at 12:09 PM, Bob Cowdery wrote: > >> Can anyone give me some hints on how to debug this problem. I know its a >> long-shot but maybe someone has experience of a similar problem. >> >> I have a real-time system that reads from a device using UDP and writes >> back to the same device using UDP. The data processing chain comprises 6 >> processes: >> >> Reader - Decoder - Input to linked-in-driver - output from >> linked-in-driver - Encoder - Writer >> >> With the parameters I'm using I should be writing the same number of >> blocks that I am reading. There are sequence numbers on both input and >> output blocks. For a while this is what happens, then I start to lose >> blocks. I believe in snatches rather than a regular pattern. I can >> sometimes hear the disruption in the output and if I stop at that point >> I've lost blocks, as many as 40 in one case. However it can run on way >> longer than that with no lost blocks. >> >> I've checked every stage of the code with debug statements but >> everything seems to pass through correctly. I've also checked the >> message queue length at each stage and it never builds up on any >> process. I have no catch-all message handlers. The messages are all >> large binary. >> >> The symptoms look like it just can't keep up but If a process was >> getting behind I would expect the message q to build up. CPU is 5-7%. >> >> Any ideas would be very welcome. >> >> Bob >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ebegumisa@REDACTED Fri Jun 3 02:08:50 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Fri, 03 Jun 2011 10:08:50 +1000 Subject: [erlang-questions] Losing data In-Reply-To: <4DE7DFF4.3020809@bobcowdery.plus.com> References: <4DE7DFF4.3020809@bobcowdery.plus.com> Message-ID: On Fri, 03 Jun 2011 05:09:40 +1000, Bob Cowdery wrote: > Can anyone give me some hints on how to debug this problem. I know its a > long-shot but maybe someone has experience of a similar problem. > > I have a real-time system that reads from a device using UDP and writes > back to the same device using UDP. The data processing chain comprises 6 > processes: > > Reader - Decoder - Input to linked-in-driver - output from > linked-in-driver - Encoder - Writer > Wondering: is there any particular reason why the reader and decoder are separate processes, and the encoder plus writer are also separate processes? If the reader+decoder were one single process (reader module calling decoder module) and the encoder+writer another single process, you could reduce the amount of message-passing and get better performance. Probably won't solve your immediate problem since you're not witnessing message-bottlenecks (yet) but worth looking into. - Edmond - > With the parameters I'm using I should be writing the same number of > blocks that I am reading. There are sequence numbers on both input and > output blocks. For a while this is what happens, then I start to lose > blocks. I believe in snatches rather than a regular pattern. I can > sometimes hear the disruption in the output and if I stop at that point > I've lost blocks, as many as 40 in one case. However it can run on way > longer than that with no lost blocks. > > I've checked every stage of the code with debug statements but > everything seems to pass through correctly. I've also checked the > message queue length at each stage and it never builds up on any > process. I have no catch-all message handlers. The messages are all > large binary. > > The symptoms look like it just can't keep up but If a process was > getting behind I would expect the message q to build up. CPU is 5-7%. > > Any ideas would be very welcome. > > Bob > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From maruthavanan_s@REDACTED Fri Jun 3 06:24:33 2011 From: maruthavanan_s@REDACTED (Maruthavanan Subbarayan) Date: Fri, 3 Jun 2011 00:24:33 -0400 Subject: [erlang-questions] net_adm:ping/1 hostname Message-ID: Hi, I wanted to run a distributed system and I tried to do the below and got a error 1>net_adm:ping('tcpserver@REDACTED').pang =ERROR REPORT==== 3-Jun-2011::09:48:45 ===** System NOT running to use fully qualified hostnames **** Hostname 192.168.10.125 is illegal ** Do I need to use only hostname and I can't use ip addresses? I would like to know why because few of the networks which I came through did not have a DNS among LAN? One way I could find to solve this is add the ip against hostname in system file and proceed. (but this mostly requires admin /root privileges). Are there any other workarounds? Thanks,Marutha -------------- next part -------------- An HTML attachment was scrubbed... URL: From langxianzhe@REDACTED Fri Jun 3 07:22:19 2011 From: langxianzhe@REDACTED (jasonlxw) Date: Thu, 2 Jun 2011 22:22:19 -0700 (PDT) Subject: [erlang-questions] net_adm:ping/1 hostname In-Reply-To: References: Message-ID: you can try.. (s@REDACTED)3> net_adm:ping('s@REDACTED'). pong 2011/6/3 maruthavanan s [via Erlang] < ml-node+3569984-1483686616-67370@REDACTED> > Hi, > > I wanted to run a distributed system and I tried to do the below and got a > error > > 1>net_adm:ping('[hidden email] > '). > pang > > =ERROR REPORT==== 3-Jun-2011::09:48:45 === > ** System NOT running to use fully qualified hostnames ** > ** Hostname 192.168.10.125 is illegal ** > > Do I need to use only hostname and I can't use ip addresses? I would like > to know why because few of the networks which I came through did not have a > DNS among LAN? One way I could find to solve this is add the ip against > hostname in system file and proceed. (but this mostly requires admin > /root privileges). Are there any other workarounds? > > Thanks, > Marutha > > _______________________________________________ > erlang-questions mailing list > [hidden email] > http://erlang.org/mailman/listinfo/erlang-questions > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://erlang.2086793.n4.nabble.com/net-adm-ping-1-hostname-tp3569984p3569984.html > To start a new topic under Erlang Questions, email > ml-node+2086794-1604045721-67370@REDACTED > To unsubscribe from Erlang Questions, click here. > > ----- Never give up -- View this message in context: http://erlang.2086793.n4.nabble.com/net-adm-ping-1-hostname-tp3569984p3570036.html Sent from the Erlang Questions mailing list archive at Nabble.com. -------------- next part -------------- An HTML attachment was scrubbed... URL: From langxianzhe@REDACTED Fri Jun 3 07:29:10 2011 From: langxianzhe@REDACTED (jasonlxw) Date: Thu, 2 Jun 2011 22:29:10 -0700 (PDT) Subject: [erlang-questions] net_adm:ping/1 hostname In-Reply-To: References: Message-ID: jason@REDACTED:~$ erl -name 'aa@REDACTED' -setcookie 123 Erlang R13B03 (erts-5.7.4) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.4 (abort with ^G) (aa@REDACTED)1> ------------------- jason@REDACTED:~$ erl -name 'b@REDACTED' -setcookie 123 Erlang R13B03 (erts-5.7.4) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.4 (abort with ^G) (b@REDACTED)1> net_adm:ping('aa@REDACTED') (b@REDACTED)1> . pong (b@REDACTED)2> --------------- OK.There is not any problem. 2011/6/3 maruthavanan s [via Erlang] < ml-node+3569984-1483686616-67370@REDACTED> > Hi, > > I wanted to run a distributed system and I tried to do the below and got a > error > > 1>net_adm:ping('[hidden email] > '). > pang > > =ERROR REPORT==== 3-Jun-2011::09:48:45 === > ** System NOT running to use fully qualified hostnames ** > ** Hostname 192.168.10.125 is illegal ** > > Do I need to use only hostname and I can't use ip addresses? I would like > to know why because few of the networks which I came through did not have a > DNS among LAN? One way I could find to solve this is add the ip against > hostname in system file and proceed. (but this mostly requires admin > /root privileges). Are there any other workarounds? > > Thanks, > Marutha > > _______________________________________________ > erlang-questions mailing list > [hidden email] > http://erlang.org/mailman/listinfo/erlang-questions > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://erlang.2086793.n4.nabble.com/net-adm-ping-1-hostname-tp3569984p3569984.html > To start a new topic under Erlang Questions, email > ml-node+2086794-1604045721-67370@REDACTED > To unsubscribe from Erlang Questions, click here. > > ----- Never give up -- View this message in context: http://erlang.2086793.n4.nabble.com/net-adm-ping-1-hostname-tp3569984p3570042.html Sent from the Erlang Questions mailing list archive at Nabble.com. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Fri Jun 3 09:47:35 2011 From: bob@REDACTED (Bob Cowdery) Date: Fri, 03 Jun 2011 08:47:35 +0100 Subject: [erlang-questions] Losing data In-Reply-To: References: <4DE7DFF4.3020809@bobcowdery.plus.com> <4DE7FA6A.80406@bobcowdery.plus.com> Message-ID: <4DE89197.6090601@bobcowdery.plus.com> On 02/06/2011 23:12, Mike Oxford wrote: > I'm guessing you've already tried increasing SASL debugging, and using > dbg:tracer()? > How about putting a debugger (gdb or the like) on your driver? > Have you tried synthetic testing (eg, loop the same packet into your > system, injected just after > the UDP layer and then check the output on the other side?" > You say that you're breaking the packets up for processing, and > putting them back together, with > sequencing staying the same... but you're losing packets? That seems > only possible if your > hardware isn't actually sending it (or there's a bug in your > sequencing numbers/checks.) > I started out with a simulator (in Erlang) and when I was happy moved onto the hardware. I think I will move back to the simulator because I can then see both ends, and I can vary the data rate easily and should be able to reproduce the fault. Nothing makes sense at the moment because I'm pretty sure the sequence numbering is sound and because of it I can pretty much state I'm not losing packets either in reception or transmission and don't have queues building in Erlang but blocks are still going missing. > Have you put Ethereal/Wireshark/tcpdump etc on the wire during the > tests? Have you seen a pause > in output? > I've used Wireshark but the disruption in output is not seen because its missing data but the sequence is correct. It's not a silent pause. > Hard to tell from this far away. =/ I know, but sometime just talking through the problem helps to clarify. > > On Thu, Jun 2, 2011 at 2:02 PM, Bob Cowdery > wrote: > > On 02/06/2011 20:26, Mike Oxford wrote: >> Can you pin down where you think you're losing the data? You >> state a hard number....where in the chain are you measuring? >> > I've been trying to do that for several days but it's really hard > to track it all the way through in one go because the buffer size > changes in different stages and its not even multiples, it only > comes together in fixed size blocks at each end. When I check each > stage individually I'm failing to find any problem. I see I've > lost blocks overall but the stage I'm looking at has no breaks in > the pattern. They all seem to process the data correctly - and > most of the time the input and output keep pace. There is no code > in the path that could just junk blocks. > >> Usually in audio-processing you're going to run into issues in >> two main places; buffer starvation and codec corruption. >> Buffer starvation is pretty self-explanitory -- something in the >> chain cannot keep up, usually due to CPU load, lock contention or >> other data flow interruption. >> Codec corruption happens when the codec tries to use the new >> incoming packets against it's stored run-state. If >> something happens (out of sequence, null packets, etc) the codec >> will usually do one of two things: do its best and give you >> corrupted audio or just drop the packet on the floor as if it >> never got it. >> > This is akin to audio processing but the signal processing is > under my control so there are no unknowns in the chain. > >> One more caveat with audio: output packet size is not usually >> equal to input packet size. The most codecs do >> lossy-compression. Just one more thing to check. >> > I'm using 48k input and output rate for the test. >> -mox >> >> On Thu, Jun 2, 2011 at 12:09 PM, Bob Cowdery >> > wrote: >> >> Can anyone give me some hints on how to debug this problem. I >> know its a >> long-shot but maybe someone has experience of a similar problem. >> >> I have a real-time system that reads from a device using UDP >> and writes >> back to the same device using UDP. The data processing chain >> comprises 6 >> processes: >> >> Reader - Decoder - Input to linked-in-driver - output from >> linked-in-driver - Encoder - Writer >> >> With the parameters I'm using I should be writing the same >> number of >> blocks that I am reading. There are sequence numbers on both >> input and >> output blocks. For a while this is what happens, then I start >> to lose >> blocks. I believe in snatches rather than a regular pattern. >> I can >> sometimes hear the disruption in the output and if I stop at >> that point >> I've lost blocks, as many as 40 in one case. However it can >> run on way >> longer than that with no lost blocks. >> >> I've checked every stage of the code with debug statements but >> everything seems to pass through correctly. I've also >> checked the >> message queue length at each stage and it never builds up on any >> process. I have no catch-all message handlers. The messages >> are all >> large binary. >> >> The symptoms look like it just can't keep up but If a process was >> getting behind I would expect the message q to build up. CPU >> is 5-7%. >> >> Any ideas would be very welcome. >> >> Bob >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Fri Jun 3 10:03:49 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Fri, 03 Jun 2011 09:03:49 +0100 Subject: [erlang-questions] net_adm:ping/1 hostname In-Reply-To: References: Message-ID: <4DE89565.30400@llaisdy.com> Dear Marutha How are you starting the shell? If I start with an sname I get the same error as you: $ erl -sname bla Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) (bla@REDACTED)1> net_adm:ping('tcpserver@REDACTED'). pang (bla@REDACTED)2> =ERROR REPORT==== 3-Jun-2011::08:58:03 === ** System NOT running to use fully qualified hostnames ** ** Hostname 192.168.10.125 is illegal ** (bla@REDACTED)2> To use fully qualified hostnames you should start the shell with -name instead of -sname: $ erl -name bla Erlang R13B01 (erts-5.7.2) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.7.2 (abort with ^G) (bla@REDACTED)1> net_adm:ping('tcpserver@REDACTED'). pang (bla@REDACTED)2> You can use IP addresses or names in the remote node term. To get "pong" instead of "pang" you need to make sure the nodes are allowed to connect. See the documentation, or I recently blogged about it here: http://llaisdy.wordpress.com/2011/05/17/connecting-erlang-nodes/ Best wishes Ivan On 03/06/2011 05:24, Maruthavanan Subbarayan wrote: > Hi, > > I wanted to run a distributed system and I tried to do the below and got > a error > > 1>net_adm:ping('tcpserver@REDACTED'). > pang > > =ERROR REPORT==== 3-Jun-2011::09:48:45 === > ** System NOT running to use fully qualified hostnames ** > ** Hostname 192.168.10.125 is illegal ** > > Do I need to use only hostname and I can't use ip addresses? I would > like to know why because few of the networks which I came through did > not have a DNS among LAN? One way I could find to solve this is add the > ip against hostname in system file and proceed. (but this mostly > requires admin /root privileges). Are there any other workarounds? > > Thanks, > Marutha > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From bob@REDACTED Fri Jun 3 10:05:40 2011 From: bob@REDACTED (Bob Cowdery) Date: Fri, 03 Jun 2011 09:05:40 +0100 Subject: [erlang-questions] Losing data In-Reply-To: References: <4DE7DFF4.3020809@bobcowdery.plus.com> Message-ID: <4DE895D4.4040202@bobcowdery.plus.com> On 03/06/2011 01:08, Edmond Begumisa wrote: > On Fri, 03 Jun 2011 05:09:40 +1000, Bob Cowdery > wrote: > >> Can anyone give me some hints on how to debug this problem. I know its a >> long-shot but maybe someone has experience of a similar problem. >> >> I have a real-time system that reads from a device using UDP and writes >> back to the same device using UDP. The data processing chain comprises 6 >> processes: >> >> Reader - Decoder - Input to linked-in-driver - output from >> linked-in-driver - Encoder - Writer >> > > Wondering: is there any particular reason why the reader and decoder > are separate processes, and the encoder plus writer are also separate > processes? My initial though was the reader and writer are pretty simple processes and if there was a throughput problem I wanted it to show up as a message queue building and not as dropped packets. I think its structure over efficiency at the moment. > > If the reader+decoder were one single process (reader module calling > decoder module) and the encoder+writer another single process, you > could reduce the amount of message-passing and get better performance. > Probably won't solve your immediate problem since you're not > witnessing message-bottlenecks (yet) but worth looking into. If I can solve this problem then it will get pushed much harder as I ramp up the sample rate and I might then hit a wall and I will have to optimise. > > - Edmond - > > >> With the parameters I'm using I should be writing the same number of >> blocks that I am reading. There are sequence numbers on both input and >> output blocks. For a while this is what happens, then I start to lose >> blocks. I believe in snatches rather than a regular pattern. I can >> sometimes hear the disruption in the output and if I stop at that point >> I've lost blocks, as many as 40 in one case. However it can run on way >> longer than that with no lost blocks. >> >> I've checked every stage of the code with debug statements but >> everything seems to pass through correctly. I've also checked the >> message queue length at each stage and it never builds up on any >> process. I have no catch-all message handlers. The messages are all >> large binary. >> >> The symptoms look like it just can't keep up but If a process was >> getting behind I would expect the message q to build up. CPU is 5-7%. >> >> Any ideas would be very welcome. >> >> Bob >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From systemio@REDACTED Fri Jun 3 10:53:12 2011 From: systemio@REDACTED (systemio systemio) Date: Fri, 03 Jun 2011 12:53:12 +0400 Subject: [erlang-questions] =?utf-8?q?there_is_a_blocking_in_the_process_e?= =?utf-8?q?valuatingthe_huge_factorial?= In-Reply-To: References: Message-ID: but the same behavior if i make this below (io:format doesn't matter). Start S3() wait for a 3 seconds and try to type something. the typing will appear after factorial will be evaluated. BUT sometimes i can typing. It's very strange. (PS. i don't know if this answer will be posted right, because erlang-questions forum is strange for me, i didn't find ?any buttons like Reply or Answer, i've only made Reply All in my mail web tool.) 5> S3 = fun()-> spawn(Fac) end. #Fun 6> S3(). <0.46.0> ?7>ughuihuihiuh Thu, 2 Jun 2011 11:54:43 -0700 ?????? ?? Mike Oxford : As a guess; ?io:format sends the message to the group leader for output. If your group leader were to change based on the spawn() call, you may have your io:format message in the run queue behind the fac call. More information may be found here...?http://www.trapexit.org/forum/viewtopic.php?p=43855 Again, just a guess. :) G'luck! -mox On Thu, Jun 2, 2011 at 7:27 AM, systemio systemio wrote: Hi all, My Erlang is (Erlang R13B03 (erts-5.7.4) [smp:2:2] [rq:2] [async-threads:0]). WinXP SP1. I've found the strange behavior when i spawn the process to evaluate the huge factorial. Here is the factorial module: -module(test). -compile(export_all). fac(0) -> 1; fac(N) -> N * fac(N-1). Here is the Erlang Shell code: 1> Fac = fun() -> test:fac(100000) end. #Fun 2> S1 = fun()-> spawn(Fac), timer:sleep(3000), io:format("Will I am", []), spawn(Fac) end. #Fun 3> S2 = fun()-> spawn(Fac), io:format("Will I am", []), spawn(Fac) end. #Fun 4> S1(). Will I am<0.38.0> 5> S2(). Will I am<0.42.0> In my PC the fun Fac evaluates about 11 sec. If i start S1() i will see the message "Will I am" after 14 sec and it is strange as i expect it after 3 seconds. After message appeares i type "i()." and hit enter. This command is blocked and result appeares after 11 sec until second process evaluates. if i start S2() i will see the message "Will i am" immediately but i will not be able to type anything in the shell and my typing will appear after 22 sec. Also i've tested it with spawn(Node, ...) and the result is IAE same. Are these behaviors valid or do i do smth wrong? ~ Alex _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mazen.harake@REDACTED Fri Jun 3 11:21:53 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Fri, 3 Jun 2011 11:21:53 +0200 Subject: [erlang-questions] Understanding supervisor / start_link behaviour In-Reply-To: References: <0FC76AEFE97646D39F6975C82BBF0FB7@srstrong.com> Message-ID: True. This is a very valid point. Personally I have very rarely used the live upgrade tools of a node (relup/appup/release_handler etc) so I don't really know the bad side of not putting everything under a supervision tree. But then again I simply don't think the fuzz of specifying every single thing to reload/change is worth the "uptime" mark. The strategy I prefer is to have an architecture which enables me to; take down a node gracefully (detaching itself from the cluster), manually install a release (I.e. untar the release and changing start_erl.data to point to it), and start up the node again. This should not affect the system which should still be operational (say you have 10 nodes and you do this upgrade one by one). Should the new release not work or something unexpected turns up then just change the start_erl.data file to point to the old release and bounce the node (your version handling on your applications should support this meaning v1.32.424 in this release has *exactly* the same code as v1.32.424 in the previous release). This way of working has been proven very successful to me (and the systems I took part in building). Specifying relups and appups for this kind of work is, in my opinion, tedious but some seem to think it is worth the effort. However you do have a very important point to consider when not hanging everything under a supervisor tree. If I had only 2 nodes to consider maybe I'd want them up at all time but then again they would be built in a way to handle if one goes down (E.g. when I upgrade them). 2011/6/2 Fr?d?ric Trottier-H?bert : > There are disadvantages to *not* putting workers under the supervision tree, though. Namely, you'll be losing the ability to have the release handlers walk down the supervision trees to find which processes to suspend/update, and you'll then need to find a different way of doing things. > > This is a serious point to consider if you ever plan on going the way of releases/appups if the workers you use are to be long-lived (you don't want them to be killed during a purge). I'm not saying you didn't know this, but I felt I should point it out for the sake of having the arguments clear on the mailing list. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > On 2011-06-02, at 05:53 AM, Mazen Harake wrote: > >> Steve, >> >> I wouldn't say that you are wrong. I think that you are reasoning good >> about not putting the gen_event module under a supervisor because >> *that is what links are for*. Just because you have a supervisor >> doesn't mean the you shove everything underneath there! If the >> gen_server and the gen_event are truly linked (meaning: gen_server >> doesn't act as a "supervisor" keeping track of its gen_event process >> and restarts it all the time but rather that they really are linked >> and they crash together) then your approach, in my opinion, is good. >> >> There are great benefits in doing it in that way. Many will claim that >> it is best practice to put *everything* under a supervisor but this is >> simply not true. 90% of cases it *is* the best thing to do and many >> times it is more about how you designed your application rather than >> where to put the supervisors and their children but doing it the way >> you did is not necessarily wrong. >> >> The only problem I see with your approach is that you have registered >> the gen_event process which clearly isn't useful (since only the >> gen_server should know about it, after all, it started it). Other than >> that, this approach is extremely helpful and a nice way to clean up >> things after they die/shutdown (Again: assuming truly linked). >> >> There is a big misconception in the community that everything >> should/must look like the supervisor-tree model which shows how >> gen_servers are put under supervisors and more supervisors under the >> "top" supervisor but that is not enforced and the design principles >> doesn't take many cases into account where this setup actually brings >> more headache to the table than to just exit and clean up using linked >> processes (because they do exist). >> >> /M >> >> On 1 June 2011 21:26, Steve Strong wrote: >>> Hi, >>> >>> I've got some strange behaviour with gen_event within a supervision tree >>> which I don't fully understand. ?Consider the following supervisor >>> (completely standard, feel free to skip over): >>> >>> -module(sup). >>> -behaviour(supervisor). >>> -export([start_link/0, init/1]). >>> -define(SERVER, ?MODULE). >>> start_link() -> >>> ? ? supervisor:start_link({local, ?SERVER}, ?MODULE, []). >>> init([]) -> >>> ? ? Child1 = {child, {child, start_link, []}, permanent, 2000, worker, >>> [child]}, >>> ? ? {ok, {{one_for_all, 1000, 3600}, [Child1]}}. >>> >>> and corresponding gen_server (interesting code in bold): >>> >>> -module(child). >>> -behaviour(gen_server). >>> -export([start_link/0, init/1, handle_call/3, handle_cast/2, >>> handle_info/2, terminate/2, code_change/3]). >>> start_link() -> >>> ? ? gen_server:start_link({local, child}, child, [], []). >>> init([]) -> >>> ? ? io:format("about to start gen_event~n"), >>> ? ? X = gen_event:start_link({local, my_gen_event}), >>> ? ? io:format("gen_event started with ~p~n", [X]), >>> ? ? {ok, _Pid} = X, >>> ? ? {ok, {}, 2000}. >>> handle_call(_Request, _From, State) -> >>> ? ? {reply, ok, State}. >>> handle_cast(_Msg, State) -> >>> ? ? {noreply, State}. >>> handle_info(_Info, State) -> >>> ? ? io:format("about to crash...~n"), >>> ? ? 1 = 2, >>> ? ? {noreply, State}. >>> terminate(_Reason, _State) -> >>> ? ? ok. >>> code_change(_OldVsn, State, _Extra) -> >>> ? ? {ok, State}. >>> >>> If I run this from an erl shell like this: >>> >>> --> erl >>> Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] >>> [async-threads:0] [hipe] [kernel-poll:false] >>> Eshell V5.8.2 ?(abort with ^G) >>> 1> application:start(sasl), supervisor:start_link(sup, []). >>> >>> >>> Then the supervisor & server start as expected. ?After 2 seconds the server >>> gets a timeout message and crashes itself; the supervisor obviously spots >>> this and restarts it. ?Within the init of the gen_server, it also does a >>> start_link on a gen_event process. ?By my understanding, whenever the >>> gen_server process exits, the gen_event will also be terminated. >>> However, every now and then I see the following output (a ton of sasl trace >>> omitted for clarity!): >>> >>> about to crash... >>> about to start gen_event >>> gen_event started with {error,{already_started,<0.79.0>}} >>> about to start gen_event >>> gen_event started with {error,{already_started,<0.79.0>}} >>> about to start gen_event >>> >>> What is happening is that the gen_server is crashing but on its restart the >>> gen_event process is still running - hence the gen_server fails in its init >>> and gets restarted again. ?Sometimes this loop clears after a few >>> iterations, other times it can continue until the parent supervisor gives >>> up, packs its bags and goes home. >>> So, my question is whether this is expected behaviour or not. ?I assume that >>> the termination of the linked child is happening asynchronously, and that >>> the supervisor is hence restarting its children before things have cleaned >>> up correctly - is that correct? >>> I can fix this particular scenario by trapping exits within the gen_server, >>> and then calling gen_event:stop within the terminate. ?Is this type of >>> processing necessary whenever a process is start_link'ed within a supervisor >>> tree, or is what I'm doing considered bad practice? >>> Thanks for your time, >>> Steve >>> -- >>> Steve Strong, Director, id3as >>> twitter.com/srstrong >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From mihai@REDACTED Fri Jun 3 14:37:13 2011 From: mihai@REDACTED (Mihai Balea) Date: Fri, 3 Jun 2011 08:37:13 -0400 Subject: [erlang-questions] Losing data In-Reply-To: <4DE89197.6090601@bobcowdery.plus.com> References: <4DE7DFF4.3020809@bobcowdery.plus.com> <4DE7FA6A.80406@bobcowdery.plus.com> <4DE89197.6090601@bobcowdery.plus.com> Message-ID: <2EEE7782-0684-48A8-9E67-0048BB4483C9@hates.ms> On Jun 3, 2011, at 3:47 AM, Bob Cowdery wrote: >> > I've used Wireshark but the disruption in output is not seen because its missing data but the sequence is correct. It's not a silent pause. Just to clarify: on the wire you see packets that are correctly (continuously) sequenced, but the data in those packets has gaps? Mihai -------------- next part -------------- An HTML attachment was scrubbed... URL: From emmiller@REDACTED Fri Jun 3 17:06:34 2011 From: emmiller@REDACTED (Evan Miller) Date: Fri, 3 Jun 2011 10:06:34 -0500 Subject: [erlang-questions] Three useful libraries (do they exist?) Message-ID: I am looking for three kinds of functionality in Erlang. Does anyone know of a library, or the beginnings of a library, for any of these? 1. Image cropping and scaling - I know that erl_img can read and write various image formats, but I am looking for something to perform a resize (which usually involves some fancy math in the resample stage). 2. Rich-text reading and writing - I am looking for an RTF parser. Ideally it would parse many rich-text formats into a single abstract format which could then be converted to any of the other formats (e.g. HTML, Markdown, BBCode, WikiCreole) 3. Sender Policy Framework implementation - The Sender Policy Framework is emerging as the standard way to verify that email hosts are who they say they are. I can't imagine writing a serious application that receives email without it. Thanks, Evan -- Evan Miller http://www.evanmiller.org/ From andrew@REDACTED Fri Jun 3 18:00:35 2011 From: andrew@REDACTED (Andrew Thompson) Date: Fri, 3 Jun 2011 12:00:35 -0400 Subject: [erlang-questions] Three useful libraries (do they exist?) In-Reply-To: References: Message-ID: <20110603160035.GE18098@hijacked.us> On Fri, Jun 03, 2011 at 10:06:34AM -0500, Evan Miller wrote: > I am looking for three kinds of functionality in Erlang. Does anyone > know of a library, or the beginnings of a library, for any of these? > > 3. Sender Policy Framework implementation - The Sender Policy > Framework is emerging as the standard way to verify that email hosts > are who they say they are. I can't imagine writing a serious > application that receives email without it. > I've considered writing something along these lines, but I looked a lot at SPF/SenderID and wasn't convinced it was worth it. I'm open to being convinced otherwise, though. Andrew From comptekki@REDACTED Fri Jun 3 20:38:48 2011 From: comptekki@REDACTED (Wes James) Date: Fri, 3 Jun 2011 12:38:48 -0600 Subject: [erlang-questions] Three useful libraries (do they exist?) In-Reply-To: References: Message-ID: On Fri, Jun 3, 2011 at 9:06 AM, Evan Miller wrote: > I am looking for three kinds of functionality in Erlang. Does anyone > know of a library, or the beginnings of a library, for any of these? > > 1. Image cropping and scaling - I know that erl_img can read and write > various image formats, but I am looking for something to perform a > resize (which usually involves some fancy math in the resample stage). You may want to take a peek at the zotonic code for image manipulation. In the installation instructions: https://github.com/AlainODea/zotonic/blob/master/doc/INSTALL it talks about using imagemagick. See zotonic here: http://zotonic.com/ > 2. Rich-text reading and writing - I am looking for an RTF parser. > Ideally it would parse many rich-text formats into a single abstract > format which could then be converted to any of the other formats (e.g. > HTML, Markdown, BBCode, WikiCreole) I didn't see any erlang rtf libs. > 3. Sender Policy Framework implementation - The Sender Policy > Framework is emerging as the standard way to verify that email hosts > are who they say they are. I can't imagine writing a serious > application that receives email without it. There is a reference to spf here: http://www.erlang.org/doc/man/inet_res.html not sure it's what you need or not. > > Thanks, > > Evan > > -- > Evan Miller > http://www.evanmiller.org/ -wes From vladdu55@REDACTED Sat Jun 4 14:25:20 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Sat, 4 Jun 2011 14:25:20 +0200 Subject: [erlang-questions] is CEAN maintained? Message-ID: Hi! We got a question today about distributing the Erlang runtime with erlide and since we can't do that for all platforms, I came to thing about CEAN. Installing from that repository would be just as well from a user's point of view. Now the problem is that CEAN distributions exist only for R11 and R12... Are there any plans to update those? Is the reason for the lack of maintenance that the interest from the community was low? How much work is required to package an OTP release the CEAN way? best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Sat Jun 4 15:31:50 2011 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 4 Jun 2011 13:31:50 +0000 (GMT) Subject: [erlang-questions] Losing data In-Reply-To: <4DE895D4.4040202@bobcowdery.plus.com> Message-ID: <1176089866.258951307194310540.JavaMail.root@zimbra> My first instinct reply was the same as jlouis, check if you are losing packets in UDP. Because to cite the bible, er Wikipedia: "UDP uses a simple transmission model without implicit handshaking dialogues for providing reliability, ordering, or data integrity. Thus, UDP provides an unreliable service and datagrams may arrive out of order, appear duplicated, or go missing without notice." Robert ----- "Bob Cowdery" wrote: > On 03/06/2011 01:08, Edmond Begumisa wrote: > > On Fri, 03 Jun 2011 05:09:40 +1000, Bob Cowdery > > wrote: > > > >> Can anyone give me some hints on how to debug this problem. I know > its a > >> long-shot but maybe someone has experience of a similar problem. > >> > >> I have a real-time system that reads from a device using UDP and > writes > >> back to the same device using UDP. The data processing chain > comprises 6 > >> processes: > >> > >> Reader - Decoder - Input to linked-in-driver - output from > >> linked-in-driver - Encoder - Writer > >> > > > > Wondering: is there any particular reason why the reader and > decoder > > are separate processes, and the encoder plus writer are also > separate > > processes? > My initial though was the reader and writer are pretty simple > processes > and if there was a throughput problem I wanted it to show up as a > message queue building and not as dropped packets. I think its > structure > over efficiency at the moment. > > > > If the reader+decoder were one single process (reader module > calling > > decoder module) and the encoder+writer another single process, you > > could reduce the amount of message-passing and get better > performance. > > Probably won't solve your immediate problem since you're not > > witnessing message-bottlenecks (yet) but worth looking into. > If I can solve this problem then it will get pushed much harder as I > ramp up the sample rate and I might then hit a wall and I will have > to > optimise. > > > > - Edmond - > > > > > >> With the parameters I'm using I should be writing the same number > of > >> blocks that I am reading. There are sequence numbers on both input > and > >> output blocks. For a while this is what happens, then I start to > lose > >> blocks. I believe in snatches rather than a regular pattern. I can > >> sometimes hear the disruption in the output and if I stop at that > point > >> I've lost blocks, as many as 40 in one case. However it can run on > way > >> longer than that with no lost blocks. > >> > >> I've checked every stage of the code with debug statements but > >> everything seems to pass through correctly. I've also checked the > >> message queue length at each stage and it never builds up on any > >> process. I have no catch-all message handlers. The messages are > all > >> large binary. > >> > >> The symptoms look like it just can't keep up but If a process was > >> getting behind I would expect the message q to build up. CPU is > 5-7%. > >> > >> Any ideas would be very welcome. > >> > >> Bob > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From wmacgyver@REDACTED Sat Jun 4 17:50:45 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Sat, 4 Jun 2011 11:50:45 -0400 Subject: [erlang-questions] Handling user sessions in Misultin or any http server? Message-ID: I looked around but didn't see any obvious ways to handle user sessions in Misultin? I looked around for mochiweb too and all I found was some example codes about setting cookies manually. The only place I've found anything that deal with session handling is in nitrogen. Am I missing something obvious? or is this something I need to roll my own? Thanks -- Omnem crede diem tibi diluxisse supremum. From michael.eugene.turner@REDACTED Sat Jun 4 18:11:42 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sun, 5 Jun 2011 01:11:42 +0900 Subject: [erlang-questions] Questions about seq_trace Message-ID: seq_trace looks useful, but I'm a little confused by the documentation. "The trace token contains a component which is called serial. It consists of two integers Previous and Current. The purpose is to uniquely identify each traced event within a trace sequence and to order the messages chronologically and in the different branches if any." The term "branch" is left undefined. It's not really clear how Previous and Current uniquely identify events. Do you need both, to uniquely identify events? Previous *what*? Current *what*? Dangling reference: "A trace token contains a label, and a set of flags. Both the label and the flags are set in 1 and 2 above." There's no "1 and 2 above". There are two bulleted lines just after "The current token of a process is set in two ways, either ...." Are those the referents of "1" and "2"? Ambiguous: "Let the serial of the trace token be tprev and tcurr. curr_cnt := curr_cnt + 1 tprev := prev_cnt tcurr := curr_cnt The trace token with tprev and tcurr is then passed along with the message." I take it the "Let" means something like "define" and not "assign". Does this mean that prev_cnt doesn't get updated by a Send? If these are counts, one a Previous count, the other a Current count, what exactly is it that's being counted? Current seems to be the number of traces sent to the tracer process from the process in question. But this seems contradicted later. "The curr_cnt of a process is incremented each time the process is involved in a sequential trace." What does it mean for a process to be "involved in a sequential trace"? Is that when a message gets sent to the tracer process? Ambiguous and/or contradictory: in "When a message is received and contains a nonempty trace token": "The process trace token is set to the trace token from the message. Let the serial of the trace token be tprev and tcurr. if (curr_cnt < tcurr ) curr_cnt := tcurr prev_cnt := tcurr" Does this mean that the process trace token is set to the trace token from the message, *except* for a Serial, {prev_cnt, curr_cnt}? I'm really having trouble following this. Earlier, before "What is Sequential Tracing", there's this explanation: "Serial is a tuple {PreviousSerial, ThisSerial}, where the first integer PreviousSerial denotes the serial counter passed in the last received message which carried a trace token." "The serial counter"? But there are *two* serial counters being passed in each Serial, right? "If the process is the first one in a new sequential trace, PreviousSerial is set to the value of the process internal "trace clock". The second integer ThisSerial is the serial counter that a process sets on outgoing messages and it is based on the process internal "trace clock" which is incremented by one before it is attached to the trace token in the message." But this just introduces a new undefined term, "trace clock". Is that "trace clock" the same as "curr_cnt"? Does "trace clock" mean something more like "trace count"? Does the "sets" in "the serial counter that a process sets on outgoing message" mean "initialize" or "puts"? Does "it is based on the process internal clock" mean that it has the same value as that internal clock, or is it derived from it, and if so, how? I'm starting to play around with seq_trace, maybe it will become clearer in the process. -michael turner -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Sat Jun 4 19:05:14 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Sat, 4 Jun 2011 13:05:14 -0400 Subject: [erlang-questions] Handling user sessions in Misultin or any http server? In-Reply-To: References: Message-ID: On Sat, Jun 4, 2011 at 11:50 AM, Wilson MacGyver wrote: > I looked around but didn't see any obvious ways to handle user > sessions in Misultin? I looked > around for mochiweb too and all I found was some example codes about > setting cookies manually. > > The only place I've found anything that deal with session handling is > in nitrogen. > > Am I missing something obvious? or is this something I need to roll my own? Yaws? http://yaws.hyber.org/session.yaws --steve From max.lapshin@REDACTED Sat Jun 4 19:44:00 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 4 Jun 2011 21:44:00 +0400 Subject: [erlang-questions] Handling user sessions in Misultin or any http server? In-Reply-To: References: Message-ID: Why do you want to store session at server? It is really a bad idea, which should be avoided if possible. Rails has switched to cookie stored sessions long time ago and this method brings excelent scalability and convenience. Server storage for sessions is a very complicated thing. From prouleau001@REDACTED Sat Jun 4 19:47:25 2011 From: prouleau001@REDACTED (Pierre Rouleau) Date: Sat, 4 Jun 2011 13:47:25 -0400 Subject: [erlang-questions] Bug in escript when running under Windows? Message-ID: Hi all, I am new to Erlang. While learning it, I started writing Erlang scripts running them with escript. One of the very nice feature of Erlang is that it is cross platform. My escript files run fine in *nix boxes. Unfortunately the implementation under Windows seems buggy: the ".exe" extension must be placed after escript for the script to work. That prevents using Windows PATHEXT capability which would allow launching an Erlang escript file by simply giving its base name (like I can do it for Python scripts on Windows for example). On a Windows computer I added .erl to PATHEXT to get CMD to search for the .erl file. I also associated the .erl file to the escript.exe file ( on that computer it's c:/pg/erl/bin.escript.exe). Here's a copy of my session, using the escript reference manual example: C:\tmp>set PATHEXT PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.py;.tcl;.PSC1;.erl C:\tmp>factorial 5 usage: factorial integer C:\tmp>cat factorial.erl #!/usr/bin/env escript %% -*- erlang -*- %%! -smp enable -sname factorial -mnesia debug verbose main([String]) -> try N = list_to_integer(String), F = fac(N), io:format("factorial ~w = ~w\n", [N,F]) catch _:_ -> usage() end; main(_) -> usage(). usage() -> io:format("usage: factorial integer\n"), halt(1). fac(0) -> 1; fac(N) -> N * fac(N-1). C:\tmp>escript factorial 5 escript: Failed to open file: c:\pg\erl\bin\escript.escript C:\tmp> C:\tmp>escript.exe factorial.erl 5 factorial 5 = 120 C:\tmp>which escript.exe c:/pg/erl/bin/escript.exe As you can see, the first invocation, "factorial 5", is able to run the script, but the arguments are not passed properly to the script. The second invocation, "escript factorial 5" seems to indicate a bug in escript as it is looking for the file c:\pg\erl\bin\escript.escript The only way I can get it to work is to invoke it like this: "escript.exe factorial 5" If there is one thing I like on Windows, it's the ability to invoke a script by type the script name without the extension and the interpreter name. Is this behaviour intentional or am I missing something? Thanks! -- Pierre Rouleau -------------- next part -------------- An HTML attachment was scrubbed... URL: From wmacgyver@REDACTED Sat Jun 4 20:15:11 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Sat, 4 Jun 2011 14:15:11 -0400 Subject: [erlang-questions] Handling user sessions in Misultin or any http server? In-Reply-To: References: Message-ID: storing sessions on server has been a very common practice, esp due to cookie has a limit of 4K on the browser side. Not to mention most cookie store I've seen, store data in plain text. There is also the overhead of cookie read from browser esp if you are working on high traffic site. thus, the session hash id set it cookies that points to server data is the common practice. http://wonko.com/post/why-you-probably-shouldnt-use-cookies-to-store-session-data but regardless of where is stored, the question is really about are there any modules that exists for handling user sessions. Steve, Thanks for the pointer to yaws. I'll look at that direction. Thanks On Sat, Jun 4, 2011 at 1:44 PM, Max Lapshin wrote: > Why do you want to store session at server? > It is really a bad idea, which should be avoided if possible. ?Rails > has switched to cookie stored sessions long time ago and this method > brings > excelent scalability and convenience. > > Server storage for sessions is a very complicated thing. > -- Omnem crede diem tibi diluxisse supremum. From max.lapshin@REDACTED Sat Jun 4 20:39:23 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 4 Jun 2011 22:39:23 +0400 Subject: [erlang-questions] Handling user sessions in Misultin or any http server? In-Reply-To: References: Message-ID: On Sat, Jun 4, 2011 at 10:15 PM, Wilson MacGyver wrote: > storing sessions on server has been a very common practice, esp due to cookie > has a limit of 4K on the browser side. Not to mention most cookie > store I've seen, > store data in plain text. There is also the overhead of cookie read > from browser esp if you > are working on high traffic site. thus, the session hash id set it > cookies that points to server > data is the common practice. > > http://wonko.com/post/why-you-probably-shouldnt-use-cookies-to-store-session-data > > but regardless of where is stored, the question is really about are > there any modules > that exists for handling user sessions. > I haven't found anything serious in that text. Just "don't do it". 1) In 99% times you need to store only user_id in session. Nothing more. There is no secret in this information 2) It is absoulutely ok to send data plaintext in cookies. 3) Cookies are signed, so server just verify if cookie is not broken. 4) It doesn't differ: either client send you some magic "user_id" or "session_id". Difference is only in storing these sessions. User records doesn't expire, but sessions expire and you _have_ to track them. Worsest problem in server session storage is that you need to expire them and delete. Second problem is that you need to share them across servers. Server session store doesn't differ from database and still User record _must_ be fetched on each request or you will be required to maintain coherence between database and session storage. So, server storage: 1) has no benefits 2) is a very complicated mechanism 3) add one more database select (because session storage is a database) on each request. 4) make scaling harder Without any profits. What for do you advise it? From bob@REDACTED Sat Jun 4 20:50:24 2011 From: bob@REDACTED (Bob Cowdery) Date: Sat, 04 Jun 2011 19:50:24 +0100 Subject: [erlang-questions] Losing data In-Reply-To: <1176089866.258951307194310540.JavaMail.root@zimbra> References: <1176089866.258951307194310540.JavaMail.root@zimbra> Message-ID: <4DEA7E70.4010209@bobcowdery.plus.com> I have to apologise for my inadequate testing. I was miscounting the input sequence and also didn't run tests for long enough. When I corrected a few errors and did a more controlled test I was indeed losing incoming and outgoing packets. The incoming side seems to be fixed by putting the buffer up to 100kB. The output side is still showing an occasional sequence error. I'm definitely allocating sequential numbers and the hardware device is very fast FPGA based and has been tested to far greater rates. I am suspecting my 16 port switch at the moment. Thanks to everybody for your help on the subject. Bob On 04/06/2011 14:31, Robert Virding wrote: > My first instinct reply was the same as jlouis, check if you are losing packets in UDP. Because to cite the bible, er Wikipedia: > > "UDP uses a simple transmission model without implicit handshaking dialogues for providing reliability, ordering, or data integrity. Thus, UDP provides an unreliable service and datagrams may arrive out of order, appear duplicated, or go missing without notice." > > Robert > > ----- "Bob Cowdery" wrote: > >> On 03/06/2011 01:08, Edmond Begumisa wrote: >>> On Fri, 03 Jun 2011 05:09:40 +1000, Bob Cowdery >>> wrote: >>> >>>> Can anyone give me some hints on how to debug this problem. I know >> its a >>>> long-shot but maybe someone has experience of a similar problem. >>>> >>>> I have a real-time system that reads from a device using UDP and >> writes >>>> back to the same device using UDP. The data processing chain >> comprises 6 >>>> processes: >>>> >>>> Reader - Decoder - Input to linked-in-driver - output from >>>> linked-in-driver - Encoder - Writer >>>> >>> Wondering: is there any particular reason why the reader and >> decoder >>> are separate processes, and the encoder plus writer are also >> separate >>> processes? >> My initial though was the reader and writer are pretty simple >> processes >> and if there was a throughput problem I wanted it to show up as a >> message queue building and not as dropped packets. I think its >> structure >> over efficiency at the moment. >>> If the reader+decoder were one single process (reader module >> calling >>> decoder module) and the encoder+writer another single process, you >>> could reduce the amount of message-passing and get better >> performance. >>> Probably won't solve your immediate problem since you're not >>> witnessing message-bottlenecks (yet) but worth looking into. >> If I can solve this problem then it will get pushed much harder as I >> ramp up the sample rate and I might then hit a wall and I will have >> to >> optimise. >>> - Edmond - >>> >>> >>>> With the parameters I'm using I should be writing the same number >> of >>>> blocks that I am reading. There are sequence numbers on both input >> and >>>> output blocks. For a while this is what happens, then I start to >> lose >>>> blocks. I believe in snatches rather than a regular pattern. I can >>>> sometimes hear the disruption in the output and if I stop at that >> point >>>> I've lost blocks, as many as 40 in one case. However it can run on >> way >>>> longer than that with no lost blocks. >>>> >>>> I've checked every stage of the code with debug statements but >>>> everything seems to pass through correctly. I've also checked the >>>> message queue length at each stage and it never builds up on any >>>> process. I have no catch-all message handlers. The messages are >> all >>>> large binary. >>>> >>>> The symptoms look like it just can't keep up but If a process was >>>> getting behind I would expect the message q to build up. CPU is >> 5-7%. >>>> Any ideas would be very welcome. >>>> >>>> Bob >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From wmacgyver@REDACTED Sat Jun 4 22:07:33 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Sat, 4 Jun 2011 16:07:33 -0400 Subject: [erlang-questions] Handling user sessions in Misultin or any http server? In-Reply-To: References: Message-ID: <1A996300-DD76-43BC-8D5C-84132365667C@gmail.com> For the type of system I work on " a 1000-byte cookie adds 16ms to the response time of a single request made over a broadband DSL connection" That alone could be a deal breaker. Your assertion of 99% time people only put userid in cookie store hasn't been my experience. And serverside session need not be in db. Memcached and redis are common solutions in high traffic site. Redis even supports autoexpire. It may just be we have very different usage case. But I think claiming cookiestore is the solution 100% of the time is a over statement. On Jun 4, 2011, at 2:39 PM, Max Lapshin wrote: > On Sat, Jun 4, 2011 at 10:15 PM, Wilson MacGyver wrote: >> storing sessions on server has been a very common practice, esp due to cookie >> has a limit of 4K on the browser side. Not to mention most cookie >> store I've seen, >> store data in plain text. There is also the overhead of cookie read >> from browser esp if you >> are working on high traffic site. thus, the session hash id set it >> cookies that points to server >> data is the common practice. >> >> http://wonko.com/post/why-you-probably-shouldnt-use-cookies-to-store-session-data >> >> but regardless of where is stored, the question is really about are >> there any modules >> that exists for handling user sessions. >> > > I haven't found anything serious in that text. Just "don't do it". > > 1) In 99% times you need to store only user_id in session. Nothing > more. There is no secret in this information > 2) It is absoulutely ok to send data plaintext in cookies. > 3) Cookies are signed, so server just verify if cookie is not broken. > 4) It doesn't differ: either client send you some magic "user_id" or > "session_id". Difference is only in storing these sessions. > User records doesn't expire, but sessions expire and you _have_ to track them. > > > Worsest problem in server session storage is that you need to expire > them and delete. > Second problem is that you need to share them across servers. > Server session store doesn't differ from database and still User > record _must_ be fetched on each request or you will be required to > maintain coherence between database and session storage. > > > So, server storage: > 1) has no benefits > 2) is a very complicated mechanism > 3) add one more database select (because session storage is a > database) on each request. > 4) make scaling harder > > Without any profits. What for do you advise it? -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom@REDACTED Sat Jun 4 22:45:49 2011 From: tom@REDACTED (Tom Samplonius) Date: Sat, 04 Jun 2011 13:45:49 -0700 (PDT) Subject: [erlang-questions] Three useful libraries (do they exist?) In-Reply-To: Message-ID: > I am looking for three kinds of functionality in Erlang. Does anyone > know of a library, or the beginnings of a library, for any of these? > 1. Image cropping and scaling - I know that erl_img can read and > write > various image formats, but I am looking for something to perform a > resize (which usually involves some fancy math in the resample > stage). As someone already said, ImageMagick is the standard go-to solution for image manipulation. An Erlang port driver would be nice, but as far as I know, none exists, but you can use the script capability (http://imagemagick.com/script/index.php) > 2. Rich-text reading and writing - I am looking for an RTF parser. > Ideally it would parse many rich-text formats into a single abstract > format which could then be converted to any of the other formats > (e.g. > HTML, Markdown, BBCode, WikiCreole) Some of those formats are not exactly comparable though. For instance, RTF is intended as a printable format, and has no semantics. So sematic markup like "blockquote" from BBCode do not have an RTF equivalent. I think the best part of RTF, is that Microsoft has announce they won't be releasing any more versions of the RTF standard. > 3. Sender Policy Framework implementation - The Sender Policy > Framework is emerging as the standard way to verify that email hosts > are who they say they are. I can't imagine writing a serious > application that receives email without it. SPF finished emerging about 5 years ago, and is already in decline. DKIM is much more useful, and is now widely implemented (Yahoo and Gmail). SPF just defines what IPs are supposed to be sending email for a given domain, but most sites that have published a SPF, specify that email can come from anywhere. This means the receiving server has to come up with its own interpretation of what to do a with the policy. DKIM is must better, since the sending domain signs the email with a private key. The public key is in DNS, so receiving sites can easily verify the signature. If the signature check fails, you can discard the message. The only decision that you need to make, is what to do with unsigned emails. SPF is normally handled by the MTA, and you didn't ask for an SMTP server library for Erlang. If your MTA is not running in Erlang, it is probably better to leave DKIM and SPF up to your MTA., > Thanks, > Evan > -- > Evan Miller > http://www.evanmiller.org/ Tom From tobias.rodaebel@REDACTED Sat Jun 4 22:47:05 2011 From: tobias.rodaebel@REDACTED (Tobias) Date: Sat, 4 Jun 2011 13:47:05 -0700 (PDT) Subject: [erlang-questions] Handling user sessions in Misultin or any http server? In-Reply-To: <1A996300-DD76-43BC-8D5C-84132365667C@gmail.com> References: <1A996300-DD76-43BC-8D5C-84132365667C@gmail.com> Message-ID: <5e5aa06c-91e0-4b55-b06a-57ea6292a6a4@dr5g2000vbb.googlegroups.com> Hi Wilson, Jacob Vorreuter from Heroku has a straightforward implementation of an Erlang Redis client. See http://github.com/JacobVorreuter/redo for further details. And this http://github.com/JacobVorreuter/redis_pool is another one. I would go for storing session IDs in cookies and keep all session data in Redis with an appropriate expiration time. Kind regards, Tobias On Jun 4, 10:07?pm, Wilson MacGyver wrote: > For the type of system I work on > " a 1000-byte cookie adds 16ms to the response time of a single request made over a broadband DSL connection" > > That alone could be a deal breaker. > > Your assertion of 99% time people only put userid in > cookie store hasn't been my experience. > > And serverside session need not be in db. Memcached > and redis are common solutions in high traffic site. > Redis even supports autoexpire. > > It may just be we have very different usage case. But I think > claiming cookiestore is the solution 100% of the time is a > over statement. > > On Jun 4, 2011, at 2:39 PM, Max Lapshin wrote: > > > > > On Sat, Jun 4, 2011 at 10:15 PM, Wilson MacGyver wrote: > >> storing sessions on server has been a very common practice, esp due to cookie > >> has a limit of 4K on the browser side. Not to mention most cookie > >> store I've seen, > >> store data in plain text. There is also the overhead of cookie read > >> from browser esp if you > >> are working on high traffic site. thus, the session hash id set it > >> cookies that points to server > >> data is the common practice. > > >>http://wonko.com/post/why-you-probably-shouldnt-use-cookies-to-store-... > > >> but regardless of where is stored, the question is really about are > >> there any modules > >> that exists for handling user sessions. > > > I haven't found anything serious in that text. Just "don't do it". > > > 1) In 99% times you need to store only user_id in session. Nothing > > more. There is no secret in this information > > 2) It is absoulutely ok to send data plaintext in cookies. > > 3) Cookies are signed, so server just verify if cookie is not broken. > > 4) It doesn't differ: either client send you some magic "user_id" or > > "session_id". Difference is only in storing these sessions. > > User records doesn't expire, but sessions expire and you _have_ to track them. > > > Worsest problem in server session storage is that you need to expire > > them and delete. > > Second problem is that you need to share them across servers. > > Server session store doesn't differ from database and still User > > record _must_ be fetched on each request or you will be required to > > maintain coherence between database and session storage. > > > So, server storage: > > 1) has no benefits > > 2) is a very complicated mechanism > > 3) add one more database select (because session storage is a > > database) on each request. > > 4) make scaling harder > > > Without any profits. What for do you advise it? > > _______________________________________________ > erlang-questions mailing list > erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions From wmacgyver@REDACTED Sat Jun 4 22:52:30 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Sat, 4 Jun 2011 16:52:30 -0400 Subject: [erlang-questions] Handling user sessions in Misultin or any http server? In-Reply-To: <5e5aa06c-91e0-4b55-b06a-57ea6292a6a4@dr5g2000vbb.googlegroups.com> References: <1A996300-DD76-43BC-8D5C-84132365667C@gmail.com> <5e5aa06c-91e0-4b55-b06a-57ea6292a6a4@dr5g2000vbb.googlegroups.com> Message-ID: Perfect! Thank you very much. On Jun 4, 2011, at 4:47 PM, Tobias wrote: > Hi Wilson, > > Jacob Vorreuter from Heroku has a straightforward implementation of an > Erlang Redis client. See http://github.com/JacobVorreuter/redo for > further details. And this http://github.com/JacobVorreuter/redis_pool > is another one. > > I would go for storing session IDs in cookies and keep all session > data in Redis with an appropriate expiration time. > > Kind regards, > Tobias > > On Jun 4, 10:07 pm, Wilson MacGyver wrote: >> For the type of system I work on >> " a 1000-byte cookie adds 16ms to the response time of a single request made over a broadband DSL connection" >> >> That alone could be a deal breaker. >> >> Your assertion of 99% time people only put userid in >> cookie store hasn't been my experience. >> >> And serverside session need not be in db. Memcached >> and redis are common solutions in high traffic site. >> Redis even supports autoexpire. >> >> It may just be we have very different usage case. But I think >> claiming cookiestore is the solution 100% of the time is a >> over statement. >> >> On Jun 4, 2011, at 2:39 PM, Max Lapshin wrote: >> >> >> >>> On Sat, Jun 4, 2011 at 10:15 PM, Wilson MacGyver wrote: >>>> storing sessions on server has been a very common practice, esp due to cookie >>>> has a limit of 4K on the browser side. Not to mention most cookie >>>> store I've seen, >>>> store data in plain text. There is also the overhead of cookie read >>>> from browser esp if you >>>> are working on high traffic site. thus, the session hash id set it >>>> cookies that points to server >>>> data is the common practice. >> >>>> http://wonko.com/post/why-you-probably-shouldnt-use-cookies-to-store-... >> >>>> but regardless of where is stored, the question is really about are >>>> there any modules >>>> that exists for handling user sessions. >> >>> I haven't found anything serious in that text. Just "don't do it". >> >>> 1) In 99% times you need to store only user_id in session. Nothing >>> more. There is no secret in this information >>> 2) It is absoulutely ok to send data plaintext in cookies. >>> 3) Cookies are signed, so server just verify if cookie is not broken. >>> 4) It doesn't differ: either client send you some magic "user_id" or >>> "session_id". Difference is only in storing these sessions. >>> User records doesn't expire, but sessions expire and you _have_ to track them. >> >>> Worsest problem in server session storage is that you need to expire >>> them and delete. >>> Second problem is that you need to share them across servers. >>> Server session store doesn't differ from database and still User >>> record _must_ be fetched on each request or you will be required to >>> maintain coherence between database and session storage. >> >>> So, server storage: >>> 1) has no benefits >>> 2) is a very complicated mechanism >>> 3) add one more database select (because session storage is a >>> database) on each request. >>> 4) make scaling harder >> >>> Without any profits. What for do you advise it? >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From prouleau001@REDACTED Sun Jun 5 00:33:46 2011 From: prouleau001@REDACTED (Pierre Rouleau) Date: Sat, 4 Jun 2011 18:33:46 -0400 Subject: [erlang-questions] A potential fix for escript on Windows Message-ID: OK, I think I know why escript is not accepting scrips unless it's invoked as escript.exe. I took a look at erts/etc/common/escript.c which I believe is the source for escript. I think one of the problem under Windows is the fact that when escript's main() is looking for the escript basename, it does not consider that, under Windows, it could very well be escript (just as much as escript.exe). If the invocation line is "escript the_scrip_file.erl arguments", then escript thinks the argv[0] is the name of the script (instead of escript executable) and appends ".escript" to it. One way to solve this problem under Windows, from what I can tell so far, would be to check for escript.exe as well as escript under Windows, as the following patch to escript does (from the R14B03 source taken at erlang.org ): @@ -377,7 +377,8 @@ last_opt = argv; #ifdef __WIN32__ - if (_stricmp(basename, "escript.exe") == 0) { + if ( (_stricmp(basename, "escript.exe") == 0) + ||(_stricmp(basename, "escript") == 0)) { #else if (strcmp(basename, "escript") == 0) { #endif I have not yet tested that patch (I am still downloading Cygwin's development package to be able to build Erlang from source on the Windows box). If I get it working, should I (and where should I) submit a patch to escript.c? Also, is there a bug tracker for Erlang where one can browse through bug reports, submit one and identify patches or is everything done through the mailing lists? I did not see links to a bug tracker in the erlang.org site except for a mention about submitting patches via github. And finally, is there a documented process for development? Thanks -- Pierre Rouleau -------------- next part -------------- An HTML attachment was scrubbed... URL: From fred.hebert@REDACTED Sun Jun 5 00:36:38 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Sat, 4 Jun 2011 18:36:38 -0400 Subject: [erlang-questions] A potential fix for escript on Windows In-Reply-To: References: Message-ID: All steps of development, how to run tests, etc. are described in the Erlang github wiki: https://github.com/erlang/otp/wiki You can see the submitting patches page (https://github.com/erlang/otp/wiki/Submitting-patches) for more details on the exact topic you're looking for. Hope this helps. -- Fred H?bert http://www.erlang-solutions.com On 2011-06-04, at 18:33 PM, Pierre Rouleau wrote: > OK, I think I know why escript is not accepting scrips unless it's invoked as escript.exe. > > I took a look at erts/etc/common/escript.c which I believe is the source for escript. > > I think one of the problem under Windows is the fact that when escript's main() is looking for the escript basename, it does not consider that, under Windows, it could very well be escript (just as much as escript.exe). If the invocation line is "escript the_scrip_file.erl arguments", then escript thinks the argv[0] is the name of the script (instead of escript executable) and appends ".escript" to it. > > One way to solve this problem under Windows, from what I can tell so far, would be to check for escript.exe as well as escript under Windows, as the following patch to escript does (from the R14B03 source taken at erlang.org): > > > @@ -377,7 +377,8 @@ > last_opt = argv; > > #ifdef __WIN32__ > - if (_stricmp(basename, "escript.exe") == 0) { > + if ( (_stricmp(basename, "escript.exe") == 0) > + ||(_stricmp(basename, "escript") == 0)) { > #else > if (strcmp(basename, "escript") == 0) { > #endif > > > I have not yet tested that patch (I am still downloading Cygwin's development package to be able to build Erlang from source on the Windows box). > > If I get it working, should I (and where should I) submit a patch to escript.c? > > Also, is there a bug tracker for Erlang where one can browse through bug reports, submit one and identify patches or is everything done through the mailing lists? I did not see links to a bug tracker in the erlang.org site except for a mention about submitting patches via github. > > And finally, is there a documented process for development? > > Thanks > > -- Pierre Rouleau > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From prouleau001@REDACTED Sun Jun 5 00:43:56 2011 From: prouleau001@REDACTED (Pierre Rouleau) Date: Sat, 4 Jun 2011 18:43:56 -0400 Subject: [erlang-questions] A potential fix for escript on Windows In-Reply-To: References: Message-ID: Thanks Fr?d?ric! You answered before I completed the cygwin installation :-) Pierre 2011/6/4 Fr?d?ric Trottier-H?bert > All steps of development, how to run tests, etc. are described in the > Erlang github wiki: https://github.com/erlang/otp/wiki > > You can see the submitting patches page ( > https://github.com/erlang/otp/wiki/Submitting-patches) for more details on > the exact topic you're looking for. > > Hope this helps. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-06-04, at 18:33 PM, Pierre Rouleau wrote: > > OK, I think I know why escript is not accepting scrips unless it's invoked > as escript.exe. > > I took a look at erts/etc/common/escript.c which I believe is the source > for escript. > > I think one of the problem under Windows is the fact that when escript's > main() is looking for the escript basename, it does not consider that, under > Windows, it could very well be escript (just as much as escript.exe). If > the invocation line is "escript the_scrip_file.erl arguments", then escript > thinks the argv[0] is the name of the script (instead of escript executable) > and appends ".escript" to it. > > One way to solve this problem under Windows, from what I can tell so far, > would be to check for escript.exe as well as escript under Windows, as the > following patch to escript does (from the R14B03 source taken at > erlang.org): > > > @@ -377,7 +377,8 @@ > last_opt = argv; > > #ifdef __WIN32__ > - if (_stricmp(basename, "escript.exe") == 0) { > + if ( (_stricmp(basename, "escript.exe") == 0) > + ||(_stricmp(basename, "escript") == 0)) { > #else > if (strcmp(basename, "escript") == 0) { > #endif > > > I have not yet tested that patch (I am still downloading Cygwin's > development package to be able to build Erlang from source on the Windows > box). > > If I get it working, should I (and where should I) submit a patch to > escript.c? > > Also, is there a bug tracker for Erlang where one can browse through bug > reports, submit one and identify patches or is everything done through the > mailing lists? I did not see links to a bug tracker in the erlang.orgsite except for a mention about submitting patches via github. > > And finally, is there a documented process for development? > > Thanks > > -- Pierre Rouleau > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alain.odea@REDACTED Sun Jun 5 01:34:22 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sat, 4 Jun 2011 21:04:22 -0230 Subject: [erlang-questions] Zotonic now has a live Github mirror Message-ID: <139FCFD9-41F9-4091-8EF2-1CDE345FB3A3@gmail.com> Hi folks: Zotonic is a Erlang web framework based on Webmachine and Nitrogen with a great CMS component. For anyone using or interested in Zotonic we have moved the official Github mirror to http://github.com/zotonic/zotonic. This mirror is kept automatically within one hour so you can rely on it to be fresh. To get started with the latest Zotonic code (what we use) head to: http://zotonic.com/documentation/858/install-on-ubuntu-or-debian-squeeze Cheers, Alain From tobias.rodaebel@REDACTED Sun Jun 5 02:31:46 2011 From: tobias.rodaebel@REDACTED (Tobias) Date: Sat, 4 Jun 2011 17:31:46 -0700 (PDT) Subject: [erlang-questions] Zotonic now has a live Github mirror In-Reply-To: <139FCFD9-41F9-4091-8EF2-1CDE345FB3A3@gmail.com> References: <139FCFD9-41F9-4091-8EF2-1CDE345FB3A3@gmail.com> Message-ID: Thanks for sharing your awesome work with us! Regards, Tobias On Jun 5, 1:34?am, Alain O'Dea wrote: > Hi folks: > > Zotonic is a Erlang web framework based on Webmachine and Nitrogen with a great CMS component. > > For anyone using or interested in Zotonic we have moved the official Github mirror tohttp://github.com/zotonic/zotonic. > > This mirror is kept automatically within one hour so you can rely on it to be fresh. > > To get started with the latest Zotonic code (what we use) head to:http://zotonic.com/documentation/858/install-on-ubuntu-or-debian-squeeze > > Cheers, > Alain > > _______________________________________________ > erlang-questions mailing list > erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions From anthonym@REDACTED Sun Jun 5 03:24:58 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Sat, 4 Jun 2011 18:24:58 -0700 Subject: [erlang-questions] garbage collection: when? In-Reply-To: References: Message-ID: <859FD04E-BDD1-481B-8831-764AFC50EFDD@alumni.caltech.edu> On Jun 1, 2011, at 5:09 AM, Jesper Louis Andersen wrote: > * Use ETS to share data common to your processes. I thought data was copied out of ETS each time you access it (unless it references a constant or large binary I assume at which point you probably get a pointer). So while you are sharing you're sharing copies if two processes both access an entry, and not a pointer to the data, correct? -Anthony -------------- next part -------------- An HTML attachment was scrubbed... URL: From muhammad.yousaf@REDACTED Sun Jun 5 10:43:02 2011 From: muhammad.yousaf@REDACTED (Muhammad Yousaf) Date: Sun, 5 Jun 2011 14:43:02 +0600 Subject: [erlang-questions] Tool for performace evaluation In-Reply-To: References: , <315503444.211231306153617444.JavaMail.root@zimbra>, , , , Message-ID: Hi , I am getting the follow error while running percept:profile (client@REDACTED)3> percept:profile("test.dat", {sorter, go, [5, 2000, 15]}, [procs]). Starting profiling. ok (client@REDACTED)4> percept:analyze("test.dat"). Parsing: "test.dat" Consolidating... Parsed 443 entries in 0.015602 s. ** exception error: bad argument in function length/1 called as length({'EXIT',<0.56.0>,normal}) in call from percept:parse_and_insert_loop/5 and i am running Erlang R14B any idea how can i fix it and make it run? Thanks Regards, Muhammad Yousaf From: wsongcn@REDACTED Date: Thu, 2 Jun 2011 21:27:19 +0800 To: spawn.think@REDACTED CC: erlang-questions@REDACTED Subject: Re: [erlang-questions] Tool for performace evaluation Great. Thanks. 2011/6/2 Ahmed Omar As a note, you can start etop in text modeetop:start([{output, text}]). 2011/6/2 Hynek Vychodil Try use -X option to your ssh command. Don't forget that X11 protocol forwarding is also permitted in sshd configuration (X11Forwarding option in sshd_config). On Tue, May 31, 2011 at 8:32 AM, Andy W. Song wrote: > I run Erlang on a remote machine through SSH. When I fire up etop it get: > 1> etop:start(). > Application initialization failed: no display name and no $DISPLAY > environment variable > Error in startup script: no display name and no $DISPLAY environment > variable > while executing > "load /usr/lib/libtk8.4.so.0 Tk" > ("package ifneeded" script) > invoked from within > "package require Tk 8.3" > (file "/usr/lib/erlang/lib/gs-1.5.12/priv/gstk.tcl" line 7) > How can I use text mode? > Thanks > Andy > On Mon, May 23, 2011 at 8:26 PM, Gustav Simonsson > wrote: >> >> etop:start(). >> >> gives information similar to 'top' in UNIX: >> http://www.erlang.org/doc/man/etop.html >> >> Regards, >> Gustav Simonsson >> >> ----- Original Message ----- >> From: "Muhammad Yousaf" >> To: erlang-questions@REDACTED >> Sent: Monday, May 23, 2011 1:00:32 PM GMT +01:00 Amsterdam / Berlin / Bern >> / Rome / Stockholm / Vienna >> Subject: [erlang-questions] Tool for performace evaluation >> >> >> Hi, >> >> Is there any tools to evaluate the performance of a module in Erlang that >> can show graphical interface for time elapse, memory usage, performance >> benchmarks etc ?? >> >> or is there any alternative to do this in Erlang >> >> Thanks in advance >> >> Regards, >> >> >> Muhammad Yousaf >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss. Be a data hero! Try GoodData now for free: www.gooddata.com _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -- Best Regards, - Ahmed Omarhttp://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think -- ---------------------------------------------------------------???????????????????????????????????????? _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sun Jun 5 12:00:57 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 5 Jun 2011 12:00:57 +0200 Subject: [erlang-questions] Handling user sessions in Misultin or any http server? In-Reply-To: References: Message-ID: 2011/6/4 Wilson MacGyver > I looked around but didn't see any obvious ways to handle user > sessions in Misultin? I looked > around for mochiweb too and all I found was some example codes about > setting cookies manually. > > The only place I've found anything that deal with session handling is > in nitrogen. > > Am I missing something obvious? or is this something I need to roll my own? > > Thanks > there are several arguments for and against server session variables. let's however not forget that http is a stateless protocol, and that this is yet-again-another-http-hack, together with comet applications and such. the reason why i've not added those in misultin is because there are several good ways to deal with them, like memcached for instance. i just believe that sessions are not something http-related, and thus not within a lightweight http library. just my opinion, of course. :) r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zambal@REDACTED Sun Jun 5 13:36:20 2011 From: zambal@REDACTED (zambal) Date: Sun, 5 Jun 2011 04:36:20 -0700 (PDT) Subject: [erlang-questions] Handling user sessions in Misultin or any http server? In-Reply-To: <1A996300-DD76-43BC-8D5C-84132365667C@gmail.com> References: <1A996300-DD76-43BC-8D5C-84132365667C@gmail.com> Message-ID: <25ef4e3f-bf75-4b55-a73c-a0c011afe44c@b1g2000yql.googlegroups.com> Depending on your requirements, a nice solution could be spawning a process per session and store the data in an Erlang process. Gr, Vincent On Jun 4, 10:07?pm, Wilson MacGyver wrote: > For the type of system I work on > " a 1000-byte cookie adds 16ms to the response time of a single request made over a broadband DSL connection" > > That alone could be a deal breaker. > > Your assertion of 99% time people only put userid in > cookie store hasn't been my experience. > > And serverside session need not be in db. Memcached > and redis are common solutions in high traffic site. > Redis even supports autoexpire. > > It may just be we have very different usage case. But I think > claiming cookiestore is the solution 100% of the time is a > over statement. > > On Jun 4, 2011, at 2:39 PM, Max Lapshin wrote: > > > > > > > > > On Sat, Jun 4, 2011 at 10:15 PM, Wilson MacGyver wrote: > >> storing sessions on server has been a very common practice, esp due to cookie > >> has a limit of 4K on the browser side. Not to mention most cookie > >> store I've seen, > >> store data in plain text. There is also the overhead of cookie read > >> from browser esp if you > >> are working on high traffic site. thus, the session hash id set it > >> cookies that points to server > >> data is the common practice. > > >>http://wonko.com/post/why-you-probably-shouldnt-use-cookies-to-store-... > > >> but regardless of where is stored, the question is really about are > >> there any modules > >> that exists for handling user sessions. > > > I haven't found anything serious in that text. Just "don't do it". > > > 1) In 99% times you need to store only user_id in session. Nothing > > more. There is no secret in this information > > 2) It is absoulutely ok to send data plaintext in cookies. > > 3) Cookies are signed, so server just verify if cookie is not broken. > > 4) It doesn't differ: either client send you some magic "user_id" or > > "session_id". Difference is only in storing these sessions. > > User records doesn't expire, but sessions expire and you _have_ to track them. > > > Worsest problem in server session storage is that you need to expire > > them and delete. > > Second problem is that you need to share them across servers. > > Server session store doesn't differ from database and still User > > record _must_ be fetched on each request or you will be required to > > maintain coherence between database and session storage. > > > So, server storage: > > 1) has no benefits > > 2) is a very complicated mechanism > > 3) add one more database select (because session storage is a > > database) on each request. > > 4) make scaling harder > > > Without any profits. What for do you advise it? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions From spawn.think@REDACTED Sun Jun 5 13:54:43 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Sun, 5 Jun 2011 13:54:43 +0200 Subject: [erlang-questions] Tool for performace evaluation In-Reply-To: References: <315503444.211231306153617444.JavaMail.root@zimbra> Message-ID: .does your sorter:go involves creating a link? (using start_link, spawn_link, ... etc) if yes, I believe this might be a bug in percept, as it doesn't tag the result messages and doesn't use a selective receive as a workaround, try spawn(percept, analyze, ["test.dat"]). 2011/6/5 Muhammad Yousaf > Hi , > > I am getting the follow error while running percept:profile > > > (client@REDACTED)3> percept:profile("test.dat", {sorter, go, [5, 2000, > 15]}, [procs]). > Starting profiling. > ok > > (client@REDACTED)4> percept:analyze("test.dat"). > Parsing: "test.dat" > Consolidating... > Parsed 443 entries in 0.015602 s. > ** exception error: bad argument > in function length/1 > called as length({'EXIT',<0.56.0>,normal}) > in call from percept:parse_and_insert_loop/5 > > > and i am running Erlang R14B any idea how can i fix it and make it run? > > Thanks > > > > Regards, > > Muhammad Yousaf > > > > > ------------------------------ > From: wsongcn@REDACTED > Date: Thu, 2 Jun 2011 21:27:19 +0800 > To: spawn.think@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Tool for performace evaluation > > > Great. Thanks. > > 2011/6/2 Ahmed Omar > > As a note, you can start etop in text mode > etop:start([{output, text}]). > > > 2011/6/2 Hynek Vychodil > > Try use -X option to your ssh command. Don't forget that X11 protocol > forwarding is also permitted in sshd configuration (X11Forwarding > option in sshd_config). > > On Tue, May 31, 2011 at 8:32 AM, Andy W. Song wrote: > > I run Erlang on a remote machine through SSH. When I fire up etop it get: > > 1> etop:start(). > > Application initialization failed: no display name and no $DISPLAY > > environment variable > > Error in startup script: no display name and no $DISPLAY environment > > variable > > while executing > > "load /usr/lib/libtk8.4.so.0 Tk" > > ("package ifneeded" script) > > invoked from within > > "package require Tk 8.3" > > (file "/usr/lib/erlang/lib/gs-1.5.12/priv/gstk.tcl" line 7) > > How can I use text mode? > > Thanks > > Andy > > On Mon, May 23, 2011 at 8:26 PM, Gustav Simonsson > > wrote: > >> > >> etop:start(). > >> > >> gives information similar to 'top' in UNIX: > >> http://www.erlang.org/doc/man/etop.html > >> > >> Regards, > >> Gustav Simonsson > >> > >> ----- Original Message ----- > >> From: "Muhammad Yousaf" > >> To: erlang-questions@REDACTED > >> Sent: Monday, May 23, 2011 1:00:32 PM GMT +01:00 Amsterdam / Berlin / > Bern > >> / Rome / Stockholm / Vienna > >> Subject: [erlang-questions] Tool for performace evaluation > >> > >> > >> Hi, > >> > >> Is there any tools to evaluate the performance of a module in Erlang > that > >> can show graphical interface for time elapse, memory usage, performance > >> benchmarks etc ?? > >> > >> or is there any alternative to do this in Erlang > >> > >> Thanks in advance > >> > >> Regards, > >> > >> > >> Muhammad Yousaf > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > > --------------------------------------------------------------- > > ???????????????????? > > ???????????????????? > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try GoodData now for free: www.gooddata.com > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > > _______________________________________________ erlang-questions mailing > list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think -------------- next part -------------- An HTML attachment was scrubbed... URL: From muhammad.yousaf@REDACTED Sun Jun 5 15:06:37 2011 From: muhammad.yousaf@REDACTED (Muhammad Yousaf) Date: Sun, 5 Jun 2011 19:06:37 +0600 Subject: [erlang-questions] Tool for performace evaluation In-Reply-To: References: , <315503444.211231306153617444.JavaMail.root@zimbra>, , , , , , , Message-ID: My sorter:go have spawn only i.e go(I,N,M) -> spawn(?MODULE, main, [I,N,M,self()]), receive done -> ok end. main(I,N,M,Parent) -> Pids = lists:foldl( fun(_,Ps) -> [ spawn(?MODULE,loop, []) | Ps] anyway by doing spawn(percept, analyze, ["test.dat"]). it works thanks Regards, Muhammad Yousaf Date: Sun, 5 Jun 2011 13:54:43 +0200 Subject: Re: [erlang-questions] Tool for performace evaluation From: spawn.think@REDACTED To: muhammad.yousaf@REDACTED CC: erlang-questions@REDACTED .does your sorter:go involves creating a link? (using start_link, spawn_link, ... etc)if yes, I believe this might be a bug in percept, as it doesn't tag the result messages and doesn't use a selective receive as a workaround, tryspawn(percept, analyze, ["test.dat"]).2011/6/5 Muhammad Yousaf Hi , I am getting the follow error while running percept:profile (client@REDACTED)3> percept:profile("test.dat", {sorter, go, [5, 2000, 15]}, [procs]). Starting profiling. ok (client@REDACTED)4> percept:analyze("test.dat"). Parsing: "test.dat" Consolidating... Parsed 443 entries in 0.015602 s. ** exception error: bad argument in function length/1 called as length({'EXIT',<0.56.0>,normal}) in call from percept:parse_and_insert_loop/5 and i am running Erlang R14B any idea how can i fix it and make it run? Thanks Regards, Muhammad Yousaf From: wsongcn@REDACTED Date: Thu, 2 Jun 2011 21:27:19 +0800 To: spawn.think@REDACTED CC: erlang-questions@REDACTED Subject: Re: [erlang-questions] Tool for performace evaluation Great. Thanks. 2011/6/2 Ahmed Omar As a note, you can start etop in text modeetop:start([{output, text}]). 2011/6/2 Hynek Vychodil Try use -X option to your ssh command. Don't forget that X11 protocol forwarding is also permitted in sshd configuration (X11Forwarding option in sshd_config). On Tue, May 31, 2011 at 8:32 AM, Andy W. Song wrote: > I run Erlang on a remote machine through SSH. When I fire up etop it get: > 1> etop:start(). > Application initialization failed: no display name and no $DISPLAY > environment variable > Error in startup script: no display name and no $DISPLAY environment > variable > while executing > "load /usr/lib/libtk8.4.so.0 Tk" > ("package ifneeded" script) > invoked from within > "package require Tk 8.3" > (file "/usr/lib/erlang/lib/gs-1.5.12/priv/gstk.tcl" line 7) > How can I use text mode? > Thanks > Andy > On Mon, May 23, 2011 at 8:26 PM, Gustav Simonsson > wrote: >> >> etop:start(). >> >> gives information similar to 'top' in UNIX: >> http://www.erlang.org/doc/man/etop.html >> >> Regards, >> Gustav Simonsson >> >> ----- Original Message ----- >> From: "Muhammad Yousaf" >> To: erlang-questions@REDACTED >> Sent: Monday, May 23, 2011 1:00:32 PM GMT +01:00 Amsterdam / Berlin / Bern >> / Rome / Stockholm / Vienna >> Subject: [erlang-questions] Tool for performace evaluation >> >> >> Hi, >> >> Is there any tools to evaluate the performance of a module in Erlang that >> can show graphical interface for time elapse, memory usage, performance >> benchmarks etc ?? >> >> or is there any alternative to do this in Erlang >> >> Thanks in advance >> >> Regards, >> >> >> Muhammad Yousaf >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss. Be a data hero! Try GoodData now for free: www.gooddata.com _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -- Best Regards, - Ahmed Omarhttp://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think -- ---------------------------------------------------------------???????????????????????????????????????? _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -- Best Regards, - Ahmed Omarhttp://nl.linkedin.com/in/adiaaFollow me on twitter @spawn_think -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sun Jun 5 16:21:54 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 5 Jun 2011 16:21:54 +0200 Subject: [erlang-questions] garbage collection: when? In-Reply-To: <4DE629D2.4090100@gmail.com> References: <4DE629D2.4090100@gmail.com> Message-ID: 2011/6/1 Richard Carlsson > Garbage collection is not triggered by any particular event (except an > explicit call to garbage_collect()), but rather, when the code tries to do > something that requires more memory, e.g., to create a tuple or cons cell, > than what is currently easily available on the heap. It then calls the > garbage collector to try to get some more free space from the newest > generation - this moves the used memory to one end and all the free memory > to the other end. If this creates enough contiguous space, the code can > continue with the allocation. Otherwise, the system will try to garbage > collect the next older generation, and so on. If all generations have been > garbage collected and there's still not enough memory for the allocation, > the Erlang runtime system will enlarge the process' heap (by allocating more > memory from the operating system). > > Thus, how often garbage collection is triggered depends on how quickly you > create tuples and other data structures, and the size of the process heap > depends on whether or not it allocates new data faster than it releases old > data. If it releases data at the same speed or faster, then it will stay at > the same size (or even shrink), because garbage collection will always be > able to reclaim enough space from the existing heap. > > In your example above, the original list has no more references to it after > the call to lists:keyreplace(), so it might get collected at that point, or > at any later point, depending on whether your program needs to allocate more > data structures and how much space is currently free on the process heap. A > process that does not try to allocate more data does not waste time doing > garbage collection either. > > /Richard > thank you richard and jesper for these very helpful insights. r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sun Jun 5 17:23:24 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 5 Jun 2011 17:23:24 +0200 Subject: [erlang-questions] garbage collection: when? In-Reply-To: <859FD04E-BDD1-481B-8831-764AFC50EFDD@alumni.caltech.edu> References: <859FD04E-BDD1-481B-8831-764AFC50EFDD@alumni.caltech.edu> Message-ID: On Sun, Jun 5, 2011 at 03:24, Anthony Molinaro wrote: > * Use ETS to share data common to your processes. > > I thought data was copied out of ETS each time you access it (unless it > references a constant or large binary I assume at which point you probably > get a pointer). ?So while you are sharing you're sharing copies if two > processes both access an entry, and not a pointer to the data, correct? This is correct. The data will be copied to the process. But if you let go of the data it will be garbage collected on the next collection. Note that 'free()' is essentially at no cost in a two-space collector which is the default for a process. What you can't do is to share large amounts of data that way since the copy will then get too expensive. -- J. From wmacgyver@REDACTED Sun Jun 5 18:19:07 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Sun, 5 Jun 2011 12:19:07 -0400 Subject: [erlang-questions] Handling user sessions in Misultin or any http server? In-Reply-To: References: Message-ID: On Sun, Jun 5, 2011 at 6:00 AM, Roberto Ostinelli wrote: > there are several arguments for and against server session variables. let's > however not forget that http is a stateless protocol, and that this is > yet-again-another-http-hack, together with comet applications and such. Yup, the industry spent 15 years putting state into a stateless protocol :) -- Omnem crede diem tibi diluxisse supremum. From wmacgyver@REDACTED Sun Jun 5 18:19:50 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Sun, 5 Jun 2011 12:19:50 -0400 Subject: [erlang-questions] Handling user sessions in Misultin or any http server? In-Reply-To: <25ef4e3f-bf75-4b55-a73c-a0c011afe44c@b1g2000yql.googlegroups.com> References: <1A996300-DD76-43BC-8D5C-84132365667C@gmail.com> <25ef4e3f-bf75-4b55-a73c-a0c011afe44c@b1g2000yql.googlegroups.com> Message-ID: Thanks for the suggestion, that is worth exploring too. On Sun, Jun 5, 2011 at 7:36 AM, zambal wrote: > Depending on your requirements, a nice solution could be spawning a > process per session and store the data in an Erlang process. > > > Gr, Vincent > -- Omnem crede diem tibi diluxisse supremum. From roberto@REDACTED Sun Jun 5 18:47:55 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 5 Jun 2011 18:47:55 +0200 Subject: [erlang-questions] Handling user sessions in Misultin or any http server? In-Reply-To: References: Message-ID: 2011/6/5 Wilson MacGyver > On Sun, Jun 5, 2011 at 6:00 AM, Roberto Ostinelli > wrote: > > there are several arguments for and against server session variables. > let's > > however not forget that http is a stateless protocol, and that this is > > yet-again-another-http-hack, together with comet applications and such. > > Yup, the industry spent 15 years putting state into a stateless protocol :) > > indeed :) my point was merely related to why there are not in misultin itself. it might get, though, who knows :) r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben@REDACTED Mon Jun 6 00:34:56 2011 From: ben@REDACTED (Ben Godfrey) Date: Sun, 5 Jun 2011 23:34:56 +0100 Subject: [erlang-questions] Running a release as a Windows service with erlsrv.exe and start_erl.exe Message-ID: Hello all, I've been trying to deploy an Erlang application as a Windows service and not having much luck. I built a release package using rebar. I'm registering the service as follows: c:\erl5.8.3\erts-5.8.3\bin\erlsrv.exe add Rolf -c "Collects system data for monitoring." -w c:\rolf -m c:\erl5.8.3\erts-5.8.3\bin\start_erl.exe -debugtype reuse -args "-setcookie xyz123 ++ -reldir c:\rolf\releases" This fails with the error "Failed to query BinDir of release." in the .debug file. Looking at the code for start_erl.exe, this is related to registry keys. I can't see any sign of any keys which match the pattern described in the source, but curiously the preceding call to open a registry key succeeds [2]. Any one have any idea about any of this stuff? Server is Windows Server 2008. Ben [1] https://github.com/erlang/otp/blob/dev/erts/etc/win32/start_erl.c#L364 [2] https://github.com/erlang/otp/blob/dev/erts/etc/win32/start_erl.c#L342 -- Ben Godfrey, Wandering Hacker http://aftnn.org | http://twitter.com/afternoon | http://www.linkedin.com/in/bengodfrey From bernie@REDACTED Mon Jun 6 00:58:15 2011 From: bernie@REDACTED (Bernie Duggan) Date: Mon, 6 Jun 2011 08:58:15 +1000 Subject: [erlang-questions] Three useful libraries (do they exist?) In-Reply-To: References: Message-ID: On 4 June 2011 01:06, Evan Miller wrote: > 1. Image cropping and scaling - I know that erl_img can read and write > various image formats, but I am looking for something to perform a > resize (which usually involves some fancy math in the resample stage). > Depending on the format you want to manipulate, erlycairo ( http://code.google.com/p/erlycairo/) might do what you want (Cairo itself only supports PNG and some vector formats, so obviously the erlycairo wrapper can't do more than that). If that's useful and you need a hand with it, let me know. Cheers, Bernard -------------- next part -------------- An HTML attachment was scrubbed... URL: From emmiller@REDACTED Mon Jun 6 05:46:04 2011 From: emmiller@REDACTED (Evan Miller) Date: Sun, 5 Jun 2011 22:46:04 -0500 Subject: [erlang-questions] Three useful libraries (do they exist?) In-Reply-To: References: Message-ID: (Apologies if this is a double-post, posting via Gmane yesterday did not appear to work) On Sat, Jun 4, 2011 at 3:45 PM, Tom Samplonius wrote: > >> I am looking for three kinds of functionality in Erlang. Does anyone >> know of a library, or the beginnings of a library, for any of these? > >> 1. Image cropping and scaling - I know that erl_img can read and >> write >> various image formats, but I am looking for something to perform a >> resize (which usually involves some fancy math in the resample >> stage). > > ?As someone already said, ImageMagick is the standard go-to solution for image manipulation. ?An Erlang port driver would be nice, but as far as I know, none exists, but you can use the script capability (http://imagemagick.com/script/index.php) I am looking for a pure-Erlang solution. For what it's worth I went ahead and added crop and scale functions to erl_img this weekend (along with PNG write support): https://github.com/evanmiller/erl_img > > >> 2. Rich-text reading and writing - I am looking for an RTF parser. >> Ideally it would parse many rich-text formats into a single abstract >> format which could then be converted to any of the other formats >> (e.g. >> HTML, Markdown, BBCode, WikiCreole) > > ?Some of those formats are not exactly comparable though. ?For instance, RTF is intended as a printable format, and has no semantics. ?So sematic markup like "blockquote" from BBCode do not have an RTF equivalent. Right, but I'd like to convert among rich-text formats as easily as converting among image formats. I'd love it if I could email an RTF file to a server application and have it display on web page as HTML, then make minor edits via Markdown. > > ?I think the best part of RTF, is that Microsoft has announce they won't be releasing any more versions of the RTF standard. > > >> 3. Sender Policy Framework implementation - The Sender Policy >> Framework is emerging as the standard way to verify that email hosts >> are who they say they are. I can't imagine writing a serious >> application that receives email without it. > > ?SPF finished emerging about 5 years ago, and is already in decline. ?DKIM is much more useful, and is now widely implemented (Yahoo and Gmail). ?SPF just defines what IPs are supposed to be sending email for a given domain, but most sites that have published a SPF, specify that email can come from anywhere. ?This means the receiving server has to come up with its own interpretation of what to do a with the policy. > > ?DKIM is must better, since the sending domain signs the email with a private key. ?The public key is in DNS, so receiving sites can easily verify the signature. ?If the signature check fails, you can discard the message. ?The only decision that you need to make, is what to do with unsigned emails. > > ?SPF is normally handled by the MTA, and you didn't ask for an SMTP server library for Erlang. ?If your MTA is not running in Erlang, it is probably better to leave DKIM and SPF up to your MTA., Interesting, I will take a look at DKIM. I am using the excellent gen_smtp library to build some basic MTA functionality: https://github.com/Vagabond/gen_smtp I would like to have an email interface to my application without worrying too much about unauthorized users. I am hoping that SPF or DKIM will do the trick. > >> Thanks, > >> Evan > >> -- >> Evan Miller >> http://www.evanmiller.org/ > > > Tom > -- Evan Miller http://www.evanmiller.org/ From soroker@REDACTED Mon Jun 6 08:56:09 2011 From: soroker@REDACTED (Andrei Soroker) Date: Sun, 5 Jun 2011 23:56:09 -0700 Subject: [erlang-questions] ssl_manager errors Message-ID: Hello folks, While test-driving R14B03, I saw a ton of errors such as {undef,[{ssl_session_cache,delete,[{{{74,125,95,27},25},<<32 bytes>>}]}]} Looks like it's a missing parameter issue, and I think this fixes the problem: https://github.com/abs/otp/commit/2b965860207b33a5816fdcbc58833d5d85425afe Thanks, Andrei From christophe.romain@REDACTED Mon Jun 6 10:22:14 2011 From: christophe.romain@REDACTED (Christophe Romain) Date: Mon, 6 Jun 2011 10:22:14 +0200 Subject: [erlang-questions] is CEAN maintained? In-Reply-To: References: Message-ID: <20110606082214.GE7546@localhost> Hello Vlad the CEAN framework as been rewritten to be much more easy to deploy I use it daily now to handle R12 B13 and R14 at the same time. it supports full OTP and compiles 250 other packages (500 declared). i need to free 1 or 2 days to finish some tasks on it and publish it. it has been tested on Mac Linux(x86/amd64/arm) NetBSD FreeBSD OpenBSD. it allows to deploy erlang installation (minimalist/custom/full) in few seconds, or starting from scratch with full compilation. I can publish it quickly now, and also provide you a full R14B03 binary repository. CEAN 2.0 will run the same way old CEAN using erlang command line and also provides a shell interface. it also allow you to generate your own custom erlang automatic installer (self extractable shell script) so you can deploy a minimalist erlang VM with your custom code on any system you need, without any dependency, in few seconds. we're in discussion with Yurii Rashkovskii (erlagner.org) and Martin Logan (erlware.org) to provide community a common packaging environment by the way. we'll try to merge our efforts to finally end up with this packaging issue. BR From spawn.think@REDACTED Mon Jun 6 10:46:18 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Mon, 6 Jun 2011 10:46:18 +0200 Subject: [erlang-questions] Tool for performace evaluation In-Reply-To: References: <315503444.211231306153617444.JavaMail.root@zimbra> Message-ID: I committed a fix for messages handling in percept_db, it might be useful for your case https://github.com/spawnthink/otp/commit/5ba7bf2acdf59cf36dd626cc16c8de72b59b3946 2011/6/5 Muhammad Yousaf > > > My sorter:go have spawn only i.e > > go(I,N,M) -> > spawn(?MODULE, main, [I,N,M,self()]), > receive done -> ok end. > main(I,N,M,Parent) -> > Pids = lists:foldl( > fun(_,Ps) -> > [ spawn(?MODULE,loop, []) | Ps] > > > anyway by doing spawn(percept, analyze, ["test.dat"]). > > it works thanks > > > Regards, > > Muhammad Yousaf > > > > > ------------------------------ > Date: Sun, 5 Jun 2011 13:54:43 +0200 > Subject: Re: [erlang-questions] Tool for performace evaluation > From: spawn.think@REDACTED > To: muhammad.yousaf@REDACTED > CC: erlang-questions@REDACTED > > .does your sorter:go involves creating a link? (using start_link, > spawn_link, ... etc) > if yes, I believe this might be a bug in percept, as it doesn't tag the > result messages and doesn't use a selective receive > as a workaround, try > spawn(percept, analyze, ["test.dat"]). > 2011/6/5 Muhammad Yousaf > > Hi , > > I am getting the follow error while running percept:profile > > > (client@REDACTED)3> percept:profile("test.dat", {sorter, go, [5, 2000, > 15]}, [procs]). > Starting profiling. > ok > > (client@REDACTED)4> percept:analyze("test.dat"). > Parsing: "test.dat" > Consolidating... > Parsed 443 entries in 0.015602 s. > ** exception error: bad argument > in function length/1 > called as length({'EXIT',<0.56.0>,normal}) > in call from percept:parse_and_insert_loop/5 > > > and i am running Erlang R14B any idea how can i fix it and make it run? > > Thanks > > > > Regards, > > Muhammad Yousaf > > > > > ------------------------------ > From: wsongcn@REDACTED > Date: Thu, 2 Jun 2011 21:27:19 +0800 > To: spawn.think@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Tool for performace evaluation > > > Great. Thanks. > > 2011/6/2 Ahmed Omar > > As a note, you can start etop in text mode > etop:start([{output, text}]). > > > 2011/6/2 Hynek Vychodil > > Try use -X option to your ssh command. Don't forget that X11 protocol > forwarding is also permitted in sshd configuration (X11Forwarding > option in sshd_config). > > On Tue, May 31, 2011 at 8:32 AM, Andy W. Song wrote: > > I run Erlang on a remote machine through SSH. When I fire up etop it get: > > 1> etop:start(). > > Application initialization failed: no display name and no $DISPLAY > > environment variable > > Error in startup script: no display name and no $DISPLAY environment > > variable > > while executing > > "load /usr/lib/libtk8.4.so.0 Tk" > > ("package ifneeded" script) > > invoked from within > > "package require Tk 8.3" > > (file "/usr/lib/erlang/lib/gs-1.5.12/priv/gstk.tcl" line 7) > > How can I use text mode? > > Thanks > > Andy > > On Mon, May 23, 2011 at 8:26 PM, Gustav Simonsson > > wrote: > >> > >> etop:start(). > >> > >> gives information similar to 'top' in UNIX: > >> http://www.erlang.org/doc/man/etop.html > >> > >> Regards, > >> Gustav Simonsson > >> > >> ----- Original Message ----- > >> From: "Muhammad Yousaf" > >> To: erlang-questions@REDACTED > >> Sent: Monday, May 23, 2011 1:00:32 PM GMT +01:00 Amsterdam / Berlin / > Bern > >> / Rome / Stockholm / Vienna > >> Subject: [erlang-questions] Tool for performace evaluation > >> > >> > >> Hi, > >> > >> Is there any tools to evaluate the performance of a module in Erlang > that > >> can show graphical interface for time elapse, memory usage, performance > >> benchmarks etc ?? > >> > >> or is there any alternative to do this in Erlang > >> > >> Thanks in advance > >> > >> Regards, > >> > >> > >> Muhammad Yousaf > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > > --------------------------------------------------------------- > > ???????????????????? > > ???????????????????? > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try GoodData now for free: www.gooddata.com > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > > _______________________________________________ erlang-questions mailing > list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Mon Jun 6 11:29:45 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 6 Jun 2011 11:29:45 +0200 Subject: [erlang-questions] is CEAN maintained? In-Reply-To: <20110606082214.GE7546@localhost> References: <20110606082214.GE7546@localhost> Message-ID: Hi Christophe, On Mon, Jun 6, 2011 at 10:22, Christophe Romain < christophe.romain@REDACTED> wrote: > the CEAN framework as been rewritten to be much more easy to deploy > I use it daily now to handle R12 B13 and R14 at the same time. it > supports full OTP and compiles 250 other packages (500 declared). > i need to free 1 or 2 days to finish some tasks on it and publish it. > it has been tested on Mac Linux(x86/amd64/arm) NetBSD FreeBSD OpenBSD. > it allows to deploy erlang installation (minimalist/custom/full) in few > seconds, or starting from scratch with full compilation. > > I can publish it quickly now, and also provide you a full R14B03 binary > repository. CEAN 2.0 will run the same way old CEAN using erlang command > line and also provides a shell interface. > it also allow you to generate your own custom erlang automatic installer > (self extractable shell script) so you can deploy a minimalist erlang VM > with your custom code on any system you need, without any dependency, in > few seconds. > > That's great, this is an important step to lower the threshold for starting up with Erlang! Please don't hurry for my sake, it's not something I will do right away. best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Mon Jun 6 11:44:49 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 6 Jun 2011 11:44:49 +0200 Subject: [erlang-questions] is CEAN maintained? In-Reply-To: <20110606082214.GE7546@localhost> References: <20110606082214.GE7546@localhost> Message-ID: On Mon, Jun 6, 2011 at 10:22, Christophe Romain wrote: > we're in discussion with Yurii Rashkovskii (erlagner.org) and Martin > Logan (erlware.org) to provide community a common packaging environment > by the way. we'll try to merge our efforts to finally end up with this > packaging issue. Please include Jacob Vorreuter to the discussion. His EPM is a pretty handy thing too: https://github.com/JacobVorreuter/epm also it would be nice for rebar to support that common packaging environment and/or make it support specific package manager as sort of a plugin. Just my $0.02 From sumasai.shivaprasad@REDACTED Mon Jun 6 11:45:12 2011 From: sumasai.shivaprasad@REDACTED (Suma Shivaprasad) Date: Mon, 6 Jun 2011 15:15:12 +0530 Subject: [erlang-questions] Http client sending http response to wrong handler Message-ID: Our app makes a lot of HTTP requests and we are facing this issue with both inets-5.5.1 and 5.3.2. Basically our receive clause for the response is trying to match the request id returned in httpc:request call and we see that the match always fails with the wrong Request Id . We gave seen this mismatch in all 3 receive clauses for stream_start, stream and stream_end What we observed after a lot of trial and error is that if the same pid makes the http requests , the responses get kind of muddled up but if we spawn a separate process for the httpc:request , it is better . But here too we have seen some occurrences of this faulty behaviour. Has anyone faced this issue ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon Jun 6 13:22:34 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 6 Jun 2011 12:22:34 +0100 Subject: [erlang-questions] is CEAN maintained? In-Reply-To: References: <20110606082214.GE7546@localhost> Message-ID: On 6 June 2011 10:44, Gleb Peregud wrote: > > Please include Jacob Vorreuter to the discussion. His EPM is a pretty > handy thing too: > > https://github.com/JacobVorreuter/epm > +1. I use epm a lot, preferring the "look for it on github" approach to the homebrew style "check for registered stuff" of agner and sutro (the latter also by Jacob). From watson.timothy@REDACTED Mon Jun 6 13:25:26 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 6 Jun 2011 12:25:26 +0100 Subject: [erlang-questions] is CEAN maintained? In-Reply-To: <20110606082214.GE7546@localhost> References: <20110606082214.GE7546@localhost> Message-ID: On 6 June 2011 09:22, Christophe Romain wrote: > it also allow you to generate your own custom erlang automatic installer > (self extractable shell script) so you can deploy a minimalist erlang VM > with your custom code on any system you need, without any dependency, in > few seconds. Very cool - looking forward to this. > > we're in discussion with Yurii Rashkovskii (erlagner.org) and Martin > Logan (erlware.org) to provide community a common packaging environment > by the way. we'll try to merge our efforts to finally end up with this > packaging issue. > I think this is a really important thing for the community. Something that was discussed recently on this list was improving the "discovery" of packages, applications and libraries for use in projects. In particular, the idea of searching for "something" based on type specs and/or other metadata was raised, which I thought a very good idea. Erlware and CEAN 1.0 already provide search capabilities, so perhaps this is something that can be considered for the mix. From dsc@REDACTED Mon Jun 6 17:10:09 2011 From: dsc@REDACTED (Dave Challis) Date: Mon, 06 Jun 2011 16:10:09 +0100 Subject: [erlang-questions] Passing binary strings to c port References: <4DECEDD1.6000206@ecs.soton.ac.uk> Message-ID: I've got an erlang program which uses binary strings (e.g. <<"abc">>), and a c port with some functions which operate on char arrays. Is it safe to pass this binary string directly to c (using ei_decode_binary) for use in char arrays? Or should I always use binary_to_list(<<"abc">>) before passing it to c, then decode using ei_decode_string? My main worry is whether binary strings in erlang could contain the null character somewhere (e.g. passing <<"abc\0def">> to c port and decoding as a binary results in c seeing the string "abc"). Thanks, -- Dave Challis dsc@REDACTED From michael.santos@REDACTED Mon Jun 6 17:32:15 2011 From: michael.santos@REDACTED (Michael Santos) Date: Mon, 6 Jun 2011 11:32:15 -0400 Subject: [erlang-questions] Passing binary strings to c port In-Reply-To: References: <4DECEDD1.6000206@ecs.soton.ac.uk> Message-ID: <20110606153215.GA2400@ecn.lan> On Mon, Jun 06, 2011 at 04:10:09PM +0100, Dave Challis wrote: > I've got an erlang program which uses binary strings (e.g. > <<"abc">>), and a c port with some functions which operate on char > arrays. > > Is it safe to pass this binary string directly to c (using > ei_decode_binary) for use in char arrays? > > Or should I always use binary_to_list(<<"abc">>) before passing it > to c, then decode using ei_decode_string? > > My main worry is whether binary strings in erlang could contain the > null character somewhere (e.g. passing <<"abc\0def">> to c port and > decoding as a binary results in c seeing the string "abc"). Yes, it's safe for NULLs. The external term format puts the length of the binary in the header: 1> term_to_binary(<<"abc\0def">>). <<131,109,0,0,0,7,97,98,99,0,100,101,102>> If you're encoding atoms or strings on the C side and they may contain NULLs, use the ei_encode_string_len/ei_encode_atom_len functions. From zsolt.erl@REDACTED Mon Jun 6 19:23:06 2011 From: zsolt.erl@REDACTED (Zsolt Keszthelyi) Date: Mon, 6 Jun 2011 13:23:06 -0400 Subject: [erlang-questions] Three useful libraries (do they exist?) In-Reply-To: References: Message-ID: > > SPF is normally handled by the MTA, and you didn't ask for an SMTP > server library for Erlang. If your MTA is not running in Erlang, it is > probably better to leave DKIM and SPF up to your MTA., > > Interesting, I will take a look at DKIM. I am using the excellent > gen_smtp library to build some basic MTA functionality: > > https://github.com/Vagabond/gen_smtp > > I would like to have an email interface to my application without > worrying too much about unauthorized users. I am hoping that SPF or > DKIM will do the trick. > > > -- > Evan Miller > http://www.evanmiller.org/ > > Hi Evan, Not sure how far away with your MTA, but I'm building a full MTA using gen_smtp (with connection pooling, per domain throttling, several queues optionally running on a cluster, web gui, etc.). Doing some testing in a production env, right now then I'll put it up on github ( https://github.com/zsolt-erl). Maybe you can just add SPF handling to it. I'm not sure if SPF is that much useful but I'll add dkim in the future (for now I'm just using dkimproxy ). Zsolt Keszthelyi -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela@REDACTED Mon Jun 6 21:52:15 2011 From: ingela@REDACTED (Ingela Andin) Date: Mon, 6 Jun 2011 21:52:15 +0200 Subject: [erlang-questions] ssl_manager errors In-Reply-To: References: Message-ID: Hi! There will be a slightyl diffrent fix for this problem at github in the next couple of days, as the ets table is protected just adding the parameter will alas not fix the whole problem. This problem has already been reported to erlang-bugs and was fixed last week with a special test case and all, but has not made it to gihub yet due to swedish holidays. Regards Ingela Erlang/OTP team - Ericsson AB 2011/6/6 Andrei Soroker : > Hello folks, > > While test-driving R14B03, I saw a ton of errors such as > > {undef,[{ssl_session_cache,delete,[{{{74,125,95,27},25},<<32 bytes>>}]}]} > > Looks like it's a missing parameter issue, and I think this fixes the problem: > > https://github.com/abs/otp/commit/2b965860207b33a5816fdcbc58833d5d85425afe > > Thanks, > Andrei > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From vladdu55@REDACTED Mon Jun 6 22:25:21 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 6 Jun 2011 22:25:21 +0200 Subject: [erlang-questions] is CEAN maintained? In-Reply-To: <1551690769.265931307373982371.JavaMail.root@zimbra> References: <21921812.265761307373258615.JavaMail.root@zimbra> <1551690769.265931307373982371.JavaMail.root@zimbra> Message-ID: Hi! On Mon, Jun 6, 2011 at 17:26, Aniko Nagyne Vig < aniko.nagyne.vig@REDACTED> wrote: > We are working on building the Erlang/OTP binary packages for different > platforms (Mac OS X, CentOS, Debian, Fedora, openSUSE, Ubuntu, Windows, > embedded platforms): > > http://www.erlang-solutions.com/section/88/packages-by-os > > Currently we provide the full Erlang/OTP, together with the full test cycle > results so people can check if the applications they are interested in are > not skipped, passing all the tests. > > We are working on making it more generic, integrating our existing solution > to Moebius (our continuous integration system), to enable us to include > third-party applications and customise the OTP package being able to pick > the applications needed, and providing a continuous integration framework. > We can also provide support for them, please see: > > http://www.erlang-solutions.com/section/13/support > > If you would like we can build and test erlang packages with erlide for the > above platforms, and provide support if required. > That's great, I had forgotten that you were doing this at Erlang Solutions too. Compared to CEAN, though, I miss having a self-executable archive. All in all, it looks doable to install Erlang from inside Eclipse, it's mostly a matter of defining how it's going to be done (and also there's a question if one can run the installation program as root...). A related feature would also be a front-end for the package repositories. best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From radek.bulat@REDACTED Mon Jun 6 22:35:50 2011 From: radek.bulat@REDACTED (=?ISO-8859-2?Q?Rados=B3aw_Bu=B3at?=) Date: Mon, 6 Jun 2011 22:35:50 +0200 Subject: [erlang-questions] NIF + HiPE - can it works together? Message-ID: I'm playing with some erlang stuffs (still learning). Lately I wanted to compile erlang module with HiPE and also with NIF extension. But when I try to load nif (erlang:load_nif function) I get error: {"init terminating in do_boot",{{badmatch,{error,{bad_lib,"Library module name 'gim_crypto' does not match calling module 'lists'"}}},[{gim_crypto,load_nif,0},{gim_benchmark,bm,1},{lists,flatten,1}]}} When compiling without HiPE everything works fine. Am I doing something wrong or HiPE doesn't work with NIF? -- Pozdrawiam Rados?aw Bu?at http://radarek.jogger.pl - m?j blog From soroker@REDACTED Mon Jun 6 22:38:49 2011 From: soroker@REDACTED (Andrei Soroker) Date: Mon, 6 Jun 2011 13:38:49 -0700 Subject: [erlang-questions] ssl_manager errors In-Reply-To: References: Message-ID: On Mon, Jun 6, 2011 at 12:52 PM, Ingela Andin wrote: > Hi! > > There will be a slightyl diffrent fix for this problem at github in > the next couple of days, Great! > as the ets table is protected just adding the parameter will alas not > fix the whole problem. Ah yes, I realized that this morning. > This problem has already been reported to erlang-bugs and was fixed > last week with a special test case and all, but has not made it to > gihub yet ?due to swedish holidays. > > Regards Ingela Erlang/OTP team - Ericsson AB > > 2011/6/6 Andrei Soroker : >> Hello folks, >> >> While test-driving R14B03, I saw a ton of errors such as >> >> {undef,[{ssl_session_cache,delete,[{{{74,125,95,27},25},<<32 bytes>>}]}]} >> >> Looks like it's a missing parameter issue, and I think this fixes the problem: >> >> https://github.com/abs/otp/commit/2b965860207b33a5816fdcbc58833d5d85425afe >> >> Thanks, >> Andrei >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From mikpe@REDACTED Mon Jun 6 23:11:04 2011 From: mikpe@REDACTED (Mikael Pettersson) Date: Mon, 6 Jun 2011 23:11:04 +0200 Subject: [erlang-questions] NIF + HiPE - can it works together? In-Reply-To: References: Message-ID: <19949.17000.27340.404061@pilspetsen.it.uu.se> On Mon, 6 Jun 2011 22:35:50 +0200, Rados?aw Bu?at wrote: > I'm playing with some erlang stuffs (still learning). Lately I wanted > to compile erlang module with HiPE and also with NIF extension. > But when I try to load nif (erlang:load_nif function) I get error: > > {"init terminating in do_boot",{{badmatch,{error,{bad_lib,"Library > module name 'gim_crypto' does not match calling module > 'lists'"}}},[{gim_crypto,load_nif,0},{gim_benchmark,bm,1},{lists,flatten,1}]}} > > When compiling without HiPE everything works fine. Am I doing > something wrong or HiPE doesn't work with NIF? Please post a standalone and minimal test case. From anthonym@REDACTED Tue Jun 7 00:15:18 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Mon, 6 Jun 2011 15:15:18 -0700 Subject: [erlang-questions] Trap exit down? Message-ID: <20110606221518.GA14835@alumni.caltech.edu> I haven't been able to view pages since last night, and http://www.downforeveryoneorjustme.com/trapexit.org says it isn't just me. Anyone on here able to do anything about it? -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From radek.bulat@REDACTED Tue Jun 7 01:15:25 2011 From: radek.bulat@REDACTED (=?ISO-8859-2?Q?Rados=B3aw_Bu=B3at?=) Date: Tue, 7 Jun 2011 01:15:25 +0200 Subject: [erlang-questions] NIF + HiPE - can it works together? In-Reply-To: <19949.17000.27340.404061@pilspetsen.it.uu.se> References: <19949.17000.27340.404061@pilspetsen.it.uu.se> Message-ID: 2011/6/6 Mikael Pettersson : > Please post a standalone and minimal test case. > Yeah, sure. $ git clone https://github.com/radarek/nif_with_hipe $ cd nif_with_hipe $ less README $ cat rebar.config $ ./rebar compile # compile with hipe support (see rebar.config) $ erl -pa ../nif_with_hipe/ebin/ -s nif_with_hipe start -s init stop Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] {"init terminating in do_boot",{{badmatch,{error,{bad_lib,"Library module name 'nif_with_hipe' does not match calling module 'init'"}}},[{nif_with_hipe,load_nif,0},{nif_with_hipe,start,0},{code,lib_dir,1}]}} Crash dump was written to: erl_crash.dump init terminating in do_boot () $ # now remove rebar.config and run again $ mv rebar.config rebar.config.deleted $ ./rebar clean compile $ erl -pa ../nif_with_hipe/ebin/ -s nif_with_hipe start -s init stop 'atom from nif!' Eshell V5.8.4 (abort with ^G) 1> -- Pozdrawiam Rados?aw Bu?at http://radarek.jogger.pl - m?j blog From radek.bulat@REDACTED Tue Jun 7 01:17:35 2011 From: radek.bulat@REDACTED (=?ISO-8859-2?Q?Rados=B3aw_Bu=B3at?=) Date: Tue, 7 Jun 2011 01:17:35 +0200 Subject: [erlang-questions] NIF + HiPE - can it works together? In-Reply-To: References: <19949.17000.27340.404061@pilspetsen.it.uu.se> Message-ID: 2011/6/7 Rados?aw Bu?at : > 2011/6/6 Mikael Pettersson : >> Please post a standalone and minimal test case. >> > Yeah, sure. > One more thing. When module is compiled with HiPE and try to use on_load attribute it ends with error: EXITED with reason {'trans_fun/2',on_load} @hipe_beam_to_icode:1148 =ERROR REPORT==== 7-Jun-2011::01:18:36 === Error in process <0.50.0> with exit value: {{badmatch,{'EXIT',{{hipe_beam_to_icode,1148,{'trans_fun/2',on_load}},[{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode,trans_mfa_code,5},{hipe_beam_to_icode,trans_beam_function_chunk... src/nif_with_hipe.erl:none: internal error in native_compile; crash reason: {{badmatch, {'EXIT', {{hipe_beam_to_icode,1148,{'trans_fun/2',on_load}}, [{hipe_beam_to_icode,trans_fun,2}, {hipe_beam_to_icode,trans_fun,2}, {hipe_beam_to_icode,trans_mfa_code,5}, {hipe_beam_to_icode,trans_beam_function_chunk,2}, {hipe_beam_to_icode,'-module/2-lc$^1/1-1-',2}, {hipe,get_beam_icode,4}, {hipe,'-run_compiler_1/3-fun-0-',4}]}}}, [{hipe,get_beam_icode,4},{hipe,'-run_compiler_1/3-fun-0-',4}]} -- Pozdrawiam Rados?aw Bu?at http://radarek.jogger.pl - m?j blog From paul.joseph.davis@REDACTED Tue Jun 7 01:39:59 2011 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Mon, 6 Jun 2011 19:39:59 -0400 Subject: [erlang-questions] NIF + HiPE - can it works together? In-Reply-To: References: <19949.17000.27340.404061@pilspetsen.it.uu.se> Message-ID: I'm fairly certain that I saw somewhere that HiPE is incompatible with NIF's but I'm unable to actually Google a reference so I could be making that up based on that on_init error that I've seen before. If so, I'd probably just shove the code you want HiPE compiled into a lib module and have the nif stubs call out to that as necessary. HTH, Paul Davis 2011/6/6 Rados?aw Bu?at : > 2011/6/7 Rados?aw Bu?at : >> 2011/6/6 Mikael Pettersson : >>> Please post a standalone and minimal test case. >>> >> Yeah, sure. >> > > One more thing. When module is compiled with HiPE and try to use > on_load attribute it ends with error: > > EXITED with reason {'trans_fun/2',on_load} > @hipe_beam_to_icode:1148 > > =ERROR REPORT==== 7-Jun-2011::01:18:36 === > Error in process <0.50.0> with exit value: > {{badmatch,{'EXIT',{{hipe_beam_to_icode,1148,{'trans_fun/2',on_load}},[{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode,trans_mfa_code,5},{hipe_beam_to_icode,trans_beam_function_chunk... > > src/nif_with_hipe.erl:none: internal error in native_compile; > crash reason: {{badmatch, > ? ? ? ? ? ? ? ? ? {'EXIT', > ? ? ? ? ? ? ? ? ? ? ? {{hipe_beam_to_icode,1148,{'trans_fun/2',on_load}}, > ? ? ? ? ? ? ? ? ? ? ? ?[{hipe_beam_to_icode,trans_fun,2}, > ? ? ? ? ? ? ? ? ? ? ? ? {hipe_beam_to_icode,trans_fun,2}, > ? ? ? ? ? ? ? ? ? ? ? ? {hipe_beam_to_icode,trans_mfa_code,5}, > ? ? ? ? ? ? ? ? ? ? ? ? {hipe_beam_to_icode,trans_beam_function_chunk,2}, > ? ? ? ? ? ? ? ? ? ? ? ? {hipe_beam_to_icode,'-module/2-lc$^1/1-1-',2}, > ? ? ? ? ? ? ? ? ? ? ? ? {hipe,get_beam_icode,4}, > ? ? ? ? ? ? ? ? ? ? ? ? {hipe,'-run_compiler_1/3-fun-0-',4}]}}}, > ? ? ? ? ? ? ? [{hipe,get_beam_icode,4},{hipe,'-run_compiler_1/3-fun-0-',4}]} > > -- > Pozdrawiam > > Rados?aw Bu?at > http://radarek.jogger.pl - m?j blog > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From igorrs@REDACTED Tue Jun 7 08:19:36 2011 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Tue, 7 Jun 2011 03:19:36 -0300 Subject: [erlang-questions] All keys in the same slot (Mnesia/dets) Message-ID: Hello. I was running a pool with Erlang/OTP R13B04 until the beginning of the last week, when we upgraded to R14B02. The reason we upgraded is that everyday we were experiencing a lot of corruption in Mnesia's fragments (disc_only_copies). We had Erlang processes checking the fragments periodically and, in case of problems, we would delete the fragment (mnesia:del_table_copy) and clone it again from its replica. Given that Erlang/OTP R14B01 fixed a lot of concurrency issues with dets, I believe those bugs were affecting us because we perform a lot of dirty reads, what must cause more concurrent operations in dets tables. Anyway, since the upgrade to R14B02, no corrupted fragment has been detected. :-) But then I observed that, after the upgrade, most of the servers are performing much better (spending less time in I/O operations), while some of them almost didn't change in that respect. Taking a look at Mnesia's directory in each server, I noticed that the fragment files in one table (uc) are smaller in the servers that are performing better. Example of a fragment in a "slower" server: 59M uc_frag1004.DAT Example of a fragment in a "faster" server: 34M uc_frag598.DAT Since uc is a bag, I thought it could be because uc_frag598, for example, has less records than uc_frag1004. But I copied both to my box and saw I was wrong: 1> {ok, F1004} = dets:open_file("uc_frag1004.DAT"). {ok,#Ref<0.0.0.33>} 2> {ok, F598} = dets:open_file("uc_frag598.DAT"). {ok,#Ref<0.0.0.41>} 3> dets:info(F1004, no_objects). 280105 4> dets:info(F598, no_objects). 303074 5> dets:info(F1004, no_keys). 1404 6> dets:info(F598, no_keys). 1476 The only (very) relevant difference I found between them was in the slots: 7> dets:info(F1004, no_slots). {256,1536,2097152} 8> dets:info(F598, no_slots). {524288,524288,33554432} The weirdest difference being that all records in uc_frag1004 are in the same slot! 9> length([S || S <- lists:seq(0, element(2, dets:info(F1004, no_slots)) - 1), length(dets:slot(F1004, S)) > 0]). 1 10> length([S || S <- lists:seq(0, element(2, dets:info(F598, no_slots)) - 1), length(dets:slot(F598, S)) > 0]). 489 So... what may have happened and what can I do to fix it? Thank you. Igor. -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From igorrs@REDACTED Tue Jun 7 09:46:14 2011 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Tue, 7 Jun 2011 04:46:14 -0300 Subject: [erlang-questions] All keys in the same slot (Mnesia/dets) In-Reply-To: References: Message-ID: Hum... mnesia_frag_hash and dets both use phash2, so it makes sense that the keys are poorly distributed among the slots of each dets table that is a disc_only_copies fragment. :-( I guess we'll have to deal with that somehow. However, I still don't understand why one fragment has 1536 slots and the other has 524288, in the example below. Thank you. Igor. On Tue, Jun 7, 2011 at 3:19 AM, Igor Ribeiro Sucupira wrote: > Hello. > > I was running a pool with Erlang/OTP R13B04 until the beginning of the > last week, when we upgraded to R14B02. > > The reason we upgraded is that everyday we were experiencing a lot of > corruption in Mnesia's fragments (disc_only_copies). We had Erlang > processes checking the fragments periodically and, in case of > problems, we would delete the fragment (mnesia:del_table_copy) and > clone it again from its replica. > Given that Erlang/OTP R14B01 fixed a lot of concurrency issues with > dets, I believe those bugs were affecting us because we perform a lot > of dirty reads, what must cause more concurrent operations in dets > tables. > > Anyway, since the upgrade to R14B02, no corrupted fragment has been > detected. ?:-) > > But then I observed that, after the upgrade, most of the servers are > performing much better (spending less time in I/O operations), while > some of them almost didn't change in that respect. > > Taking a look at Mnesia's directory in each server, I noticed that the > fragment files in one table (uc) are smaller in the servers that are > performing better. > > Example of a fragment in a "slower" server: > 59M ? ? uc_frag1004.DAT > > Example of a fragment in a "faster" server: > 34M ? ? uc_frag598.DAT > > Since uc is a bag, I thought it could be because uc_frag598, for > example, has less records than uc_frag1004. But I copied both to my > box and saw I was wrong: > > 1> {ok, F1004} = dets:open_file("uc_frag1004.DAT"). > {ok,#Ref<0.0.0.33>} > 2> {ok, F598} = dets:open_file("uc_frag598.DAT"). > {ok,#Ref<0.0.0.41>} > 3> dets:info(F1004, no_objects). > 280105 > 4> dets:info(F598, no_objects). > 303074 > 5> dets:info(F1004, no_keys). > 1404 > 6> dets:info(F598, no_keys). > 1476 > > The only (very) relevant difference I found between them was in the slots: > > 7> dets:info(F1004, no_slots). > {256,1536,2097152} > 8> dets:info(F598, no_slots). > {524288,524288,33554432} > > The weirdest difference being that all records in uc_frag1004 are in > the same slot! > > 9> length([S || S <- lists:seq(0, element(2, dets:info(F1004, > no_slots)) - 1), length(dets:slot(F1004, S)) > 0]). > 1 > 10> length([S || S <- lists:seq(0, element(2, dets:info(F598, > no_slots)) - 1), length(dets:slot(F598, S)) > 0]). > 489 > > > So... what may have happened and what can I do to fix it? > > Thank you. > Igor. > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. From paul.james.barry@REDACTED Tue Jun 7 09:57:23 2011 From: paul.james.barry@REDACTED (Paul Barry) Date: Tue, 7 Jun 2011 08:57:23 +0100 Subject: [erlang-questions] Trap exit down? In-Reply-To: <20110606221518.GA14835@alumni.caltech.edu> References: <20110606221518.GA14835@alumni.caltech.edu> Message-ID: I've also noticed "Transaction failed" on the Erlang Factory site, here (for example): http://www.erlang-factory.com/conference/SFBay2011/speakers/RobertVirding I'd imagine someone has switched something off somewhere...? Paul. On 6 June 2011 23:15, Anthony Molinaro wrote: > I haven't been able to view pages since last night, and > > http://www.downforeveryoneorjustme.com/trapexit.org > > says it isn't just me. > > Anyone on here able to do anything about it? > > -Anthony > > -- > ------------------------------------------------------------------------ > Anthony Molinaro ? ? ? ? ? ? ? ? ? ? ? ? ? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Paul Barry, w: http://paulbarry.itcarlow.ie, e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From mjtruog@REDACTED Tue Jun 7 10:01:11 2011 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 07 Jun 2011 01:01:11 -0700 Subject: [erlang-questions] [erlang-bugs] All keys in the same slot (Mnesia/dets) In-Reply-To: References: Message-ID: <4DEDDAC7.4040304@gmail.com> Normally hash tables use prime numbers for the number of slots to avoid collisions. Could this be part of the problem here? On 06/07/2011 12:46 AM, Igor Ribeiro Sucupira wrote: > Hum... mnesia_frag_hash and dets both use phash2, so it makes sense > that the keys are poorly distributed among the slots of each dets > table that is a disc_only_copies fragment. :-( > I guess we'll have to deal with that somehow. > > However, I still don't understand why one fragment has 1536 slots and > the other has 524288, in the example below. > > Thank you. > Igor. > > On Tue, Jun 7, 2011 at 3:19 AM, Igor Ribeiro Sucupira wrote: >> Hello. >> >> I was running a pool with Erlang/OTP R13B04 until the beginning of the >> last week, when we upgraded to R14B02. >> >> The reason we upgraded is that everyday we were experiencing a lot of >> corruption in Mnesia's fragments (disc_only_copies). We had Erlang >> processes checking the fragments periodically and, in case of >> problems, we would delete the fragment (mnesia:del_table_copy) and >> clone it again from its replica. >> Given that Erlang/OTP R14B01 fixed a lot of concurrency issues with >> dets, I believe those bugs were affecting us because we perform a lot >> of dirty reads, what must cause more concurrent operations in dets >> tables. >> >> Anyway, since the upgrade to R14B02, no corrupted fragment has been >> detected. :-) >> >> But then I observed that, after the upgrade, most of the servers are >> performing much better (spending less time in I/O operations), while >> some of them almost didn't change in that respect. >> >> Taking a look at Mnesia's directory in each server, I noticed that the >> fragment files in one table (uc) are smaller in the servers that are >> performing better. >> >> Example of a fragment in a "slower" server: >> 59M uc_frag1004.DAT >> >> Example of a fragment in a "faster" server: >> 34M uc_frag598.DAT >> >> Since uc is a bag, I thought it could be because uc_frag598, for >> example, has less records than uc_frag1004. But I copied both to my >> box and saw I was wrong: >> >> 1> {ok, F1004} = dets:open_file("uc_frag1004.DAT"). >> {ok,#Ref<0.0.0.33>} >> 2> {ok, F598} = dets:open_file("uc_frag598.DAT"). >> {ok,#Ref<0.0.0.41>} >> 3> dets:info(F1004, no_objects). >> 280105 >> 4> dets:info(F598, no_objects). >> 303074 >> 5> dets:info(F1004, no_keys). >> 1404 >> 6> dets:info(F598, no_keys). >> 1476 >> >> The only (very) relevant difference I found between them was in the slots: >> >> 7> dets:info(F1004, no_slots). >> {256,1536,2097152} >> 8> dets:info(F598, no_slots). >> {524288,524288,33554432} >> >> The weirdest difference being that all records in uc_frag1004 are in >> the same slot! >> >> 9> length([S || S <- lists:seq(0, element(2, dets:info(F1004, >> no_slots)) - 1), length(dets:slot(F1004, S)) > 0]). >> 1 >> 10> length([S || S <- lists:seq(0, element(2, dets:info(F598, >> no_slots)) - 1), length(dets:slot(F598, S)) > 0]). >> 489 >> >> >> So... what may have happened and what can I do to fix it? >> >> Thank you. >> Igor. >> >> -- >> "The secret of joy in work is contained in one word - excellence. To >> know how to do something well is to enjoy it." - Pearl S. Buck. > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > From erlangsiri@REDACTED Tue Jun 7 10:11:38 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Tue, 7 Jun 2011 10:11:38 +0200 Subject: [erlang-questions] Running a release as a Windows service with erlsrv.exe and start_erl.exe In-Reply-To: References: Message-ID: Hi Ben! Unfortunately, the release handling for windows, including erlsrv.erl and start_erl.c, is broken. This was discovered not long ago, after refreshing the release_handler test suite where these tests were skipped on windows. We are currently working on this issue, and plan to release a correction as soon as possible. Regards siri@REDACTED 2011/6/6 Ben Godfrey > Hello all, > > I've been trying to deploy an Erlang application as a Windows service > and not having much luck. > > I built a release package using rebar. I'm registering the service as > follows: > > c:\erl5.8.3\erts-5.8.3\bin\erlsrv.exe add Rolf -c "Collects system > data for monitoring." -w c:\rolf -m > c:\erl5.8.3\erts-5.8.3\bin\start_erl.exe -debugtype reuse -args > "-setcookie xyz123 ++ -reldir c:\rolf\releases" > > This fails with the error "Failed to query BinDir of release." in the > .debug file. Looking at the code for start_erl.exe, this is related to > registry keys. I can't see any sign of any keys which match the > pattern described in the source, but curiously the preceding call to > open a registry key succeeds [2]. > > Any one have any idea about any of this stuff? > > Server is Windows Server 2008. > > Ben > > [1] https://github.com/erlang/otp/blob/dev/erts/etc/win32/start_erl.c#L364 > [2] https://github.com/erlang/otp/blob/dev/erts/etc/win32/start_erl.c#L342 > > -- > Ben Godfrey, Wandering Hacker > http://aftnn.org | http://twitter.com/afternoon | > http://www.linkedin.com/in/bengodfrey > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Tue Jun 7 10:41:44 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 7 Jun 2011 09:41:44 +0100 Subject: [erlang-questions] is CEAN maintained? In-Reply-To: References: <21921812.265761307373258615.JavaMail.root@zimbra> <1551690769.265931307373982371.JavaMail.root@zimbra> Message-ID: >> >> Currently we provide the full Erlang/OTP, together with the full test >> cycle results so people can check if the applications they are interested in >> are not skipped, passing all the tests. >> >> We are working on making it more generic, integrating our existing >> solution to Moebius (our continuous integration system), to enable us to >> include third-party applications and customise the OTP package being able to >> pick the applications needed, and providing a continuous integration >> framework. We can also provide support for them, please see: What are the chances of moebuis being open sourced (if it isn't already) some time? From jwilberding@REDACTED Tue Jun 7 12:51:46 2011 From: jwilberding@REDACTED (Jordan Wilberding) Date: Tue, 7 Jun 2011 06:51:46 -0400 Subject: [erlang-questions] Compiling erlang with static glibc for OTP releases In-Reply-To: References: Message-ID: Max, Were you able to come up with a fully statically compiled erlang? I am looking to doing the same thing. Thanks! JW On Mon, May 9, 2011 at 6:45 AM, Max Lapshin wrote: > I'm trying to migrate to OTP releases (using rebar) for erlyvideo. > Goal is to minify problems, build dependencies and package build farm. > > I've compiled libcrypt in a static manner into erlang so, that there > will be no problems on different versions (debian squeeze and debian > sid use different versions), > but problems go on: different versions of libc. > > I'm thinking about building erlang with all static dependencies so, > that packages from one build machine will be useable on all 64bit > linuxes. > > Is it a good idea to link erlang VM with glibc static? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Jun 7 13:08:55 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 7 Jun 2011 15:08:55 +0400 Subject: [erlang-questions] Compiling erlang with static glibc for OTP releases In-Reply-To: References: Message-ID: On Tue, Jun 7, 2011 at 2:51 PM, Jordan Wilberding wrote: > Max, > Were you able to come up with a fully statically compiled erlang? I am > looking to doing the same thing. > Thanks! > JW > Hi, not yet. So, problem looks so: if I compile erlang under debian squeeze, it is impossible to use the same release under Centos 5. I will try other systems, because Debian brings too many problem with wrong compiled libraries. From rich.claxton@REDACTED Tue Jun 7 14:22:39 2011 From: rich.claxton@REDACTED (richard claxton) Date: Tue, 7 Jun 2011 13:22:39 +0100 Subject: [erlang-questions] mochijson2 decode Message-ID: I am new to erlang and i am playing around with mochijson2, JSON parsing from a http post, I can decode the json http request if i escape all the quotes in the string \" but there must be a better solution then this? Cheer Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivier.boudeville@REDACTED Tue Jun 7 14:51:06 2011 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Tue, 7 Jun 2011 14:51:06 +0200 Subject: [erlang-questions] Get cookie time-out Message-ID: Hello, We are running a distributed application over a cluster and recently, during its deployment over 45 nodes, we have encountered a time-out apparently related to the inter-node cookie retrieval: "{timeout,{gen_server,call,[auth,{get_cookie,'SOME_NODE...". We are not that familiar with OTP, but apparently there could be a way of specifying a time-out for such an internal gen_server: we have indeed in lib/kernel/src/auth.erl: """ start_link() -> gen_server:start_link({local, auth}, auth, [], []). """ I suppose we could add a {timeout,Time} option there ? (there must be a default time-out, even if I do not know its value) Unless there is a less hackish way of waiting longer, maybe with some command-line option? More generally, maybe the default value for that time-out should be increased? Thanks in advance for any hint, Best regards, Olivier Boudeville. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Tue Jun 7 14:51:56 2011 From: bob@REDACTED (Bob Ippolito) Date: Tue, 7 Jun 2011 05:51:56 -0700 Subject: [erlang-questions] mochijson2 decode In-Reply-To: References: Message-ID: It sounds like your input isn't JSON, or maybe you've misunderstood something about Erlang syntax. If you post some code it may help pinpoint exactly what's going on. On Tue, Jun 7, 2011 at 5:22 AM, richard claxton wrote: > I am new to erlang and i am playing around with mochijson2, JSON parsing > from a http post, I can decode the json http request if i escape all the > quotes in the string \" but there must be a better solution then this? > Cheer Richard > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From rich.claxton@REDACTED Tue Jun 7 15:06:24 2011 From: rich.claxton@REDACTED (richard claxton) Date: Tue, 7 Jun 2011 14:06:24 +0100 Subject: [erlang-questions] mochijson2 decode In-Reply-To: References: Message-ID: I though i should be able to run this. mochijson2:decode( "{ "colour" : "red", "value" : "#f00" }" ). but get 2: syntax error before: colour Like you said i think its the syntax, but i cant see it. Cheers Richard On Tue, Jun 7, 2011 at 1:51 PM, Bob Ippolito wrote: > It sounds like your input isn't JSON, or maybe you've misunderstood > something about Erlang syntax. If you post some code it may help > pinpoint exactly what's going on. > > On Tue, Jun 7, 2011 at 5:22 AM, richard claxton > wrote: > > I am new to erlang and i am playing around with mochijson2, JSON parsing > > from a http post, I can decode the json http request if i escape all the > > quotes in the string \" but there must be a better solution then this? > > Cheer Richard > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Tue Jun 7 15:07:46 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 7 Jun 2011 14:07:46 +0100 Subject: [erlang-questions] is CEAN maintained? In-Reply-To: <1955034284.269291307448785499.JavaMail.root@zimbra> References: <1955034284.269291307448785499.JavaMail.root@zimbra> Message-ID: Hi Aniko, Thanks for letting me know. Good luck with the trails! Cheers, Tim On 7 June 2011 13:13, Aniko Nagyne Vig wrote: > Hi Tim, > > We will probably release some parts of it as > open source, but at the moment, we are focusing on getting the core > functionality in place and releasing it to some trial support customers. > For now we are providing the packages with the test results only. > > Best Regards, > ?Aniko Nagyne Vig > > Technical Coordinator of the London Office > Software Engineer > > Erlang Solutions Ltd. > 29 Fruit & Wool Exchange > London E1 6EU > > Direct: + 44 (0) 207 655 0344 > Mobile: + 44 (0) 75 0800 9002 > > Web: > > ----- Original Message ----- > From: "Tim Watson" > To: "Vlad Dumitrescu" > Cc: "Aniko Nagyne Vig" , "erlang-questions" > Sent: Tuesday, 7 June, 2011 09:41:44 GMT +00:00 GMT Britain, Ireland, Portugal > Subject: Re: [erlang-questions] is CEAN maintained? > >>> >>> Currently we provide the full Erlang/OTP, together with the full test >>> cycle results so people can check if the applications they are interested in >>> are not skipped, passing all the tests. >>> >>> We are working on making it more generic, integrating our existing >>> solution to Moebius (our continuous integration system), to enable us to >>> include third-party applications and customise the OTP package being able to >>> pick the applications needed, and providing a continuous integration >>> framework. We can also provide support for them, please see: > > What are the chances of moebuis being open sourced (if it isn't > already) some time? > From gleber.p@REDACTED Tue Jun 7 15:09:39 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Tue, 7 Jun 2011 15:09:39 +0200 Subject: [erlang-questions] mochijson2 decode In-Reply-To: References: Message-ID: Don't forget to escape quotes: (test@REDACTED)1> mochijson2:decode( "{ \"colour\" : \"red\", \"value\" : \"#f00\" }" ). {struct,[{<<"colour">>,<<"red">>},{<<"value">>,<<"#f00">>}]} On Tue, Jun 7, 2011 at 15:06, richard claxton wrote: > I though i should be able to run this. > mochijson2:decode( "{ "colour" : "red", "value" : "#f00" }" ). > but get??2: syntax error before: colour > Like you said i think its the syntax, but i cant see it. > Cheers Richard > > On Tue, Jun 7, 2011 at 1:51 PM, Bob Ippolito wrote: >> >> It sounds like your input isn't JSON, or maybe you've misunderstood >> something about Erlang syntax. If you post some code it may help >> pinpoint exactly what's going on. >> >> On Tue, Jun 7, 2011 at 5:22 AM, richard claxton >> wrote: >> > I am new to erlang and i am playing around with mochijson2, JSON parsing >> > from a http post, I can decode the json http request if i escape all the >> > quotes in the string \" but there must be a better solution then this? >> > Cheer Richard >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From bob@REDACTED Tue Jun 7 15:34:56 2011 From: bob@REDACTED (Bob Ippolito) Date: Tue, 7 Jun 2011 06:34:56 -0700 Subject: [erlang-questions] mochijson2 decode In-Reply-To: References: Message-ID: In a string you need to escape the quotes so that the parser can know when the string ends. This is required in essentially any programming language. In some languages you may be able to use a different quote character for your string, but in Erlang there is only one so embedding strings with lots of quotes does look noisy. In practice this isn't a problem because most of your JSON is going to be coming from the network or disk and not embedded in Erlang source strings. On Tue, Jun 7, 2011 at 6:06 AM, richard claxton wrote: > I though i should be able to run this. > mochijson2:decode( "{ "colour" : "red", "value" : "#f00" }" ). > but get??2: syntax error before: colour > Like you said i think its the syntax, but i cant see it. > Cheers Richard > > On Tue, Jun 7, 2011 at 1:51 PM, Bob Ippolito wrote: >> >> It sounds like your input isn't JSON, or maybe you've misunderstood >> something about Erlang syntax. If you post some code it may help >> pinpoint exactly what's going on. >> >> On Tue, Jun 7, 2011 at 5:22 AM, richard claxton >> wrote: >> > I am new to erlang and i am playing around with mochijson2, JSON parsing >> > from a http post, I can decode the json http request if i escape all the >> > quotes in the string \" but there must be a better solution then this? >> > Cheer Richard >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > > > From rich.claxton@REDACTED Tue Jun 7 15:36:09 2011 From: rich.claxton@REDACTED (richard claxton) Date: Tue, 7 Jun 2011 14:36:09 +0100 Subject: [erlang-questions] mochijson2 decode In-Reply-To: References: Message-ID: Ok Cheers for the feed back. Rich On Tue, Jun 7, 2011 at 2:34 PM, Bob Ippolito wrote: > In a string you need to escape the quotes so that the parser can know > when the string ends. This is required in essentially any programming > language. In some languages you may be able to use a different quote > character for your string, but in Erlang there is only one so > embedding strings with lots of quotes does look noisy. In practice > this isn't a problem because most of your JSON is going to be coming > from the network or disk and not embedded in Erlang source strings. > > On Tue, Jun 7, 2011 at 6:06 AM, richard claxton > wrote: > > I though i should be able to run this. > > mochijson2:decode( "{ "colour" : "red", "value" : "#f00" }" ). > > but get 2: syntax error before: colour > > Like you said i think its the syntax, but i cant see it. > > Cheers Richard > > > > On Tue, Jun 7, 2011 at 1:51 PM, Bob Ippolito wrote: > >> > >> It sounds like your input isn't JSON, or maybe you've misunderstood > >> something about Erlang syntax. If you post some code it may help > >> pinpoint exactly what's going on. > >> > >> On Tue, Jun 7, 2011 at 5:22 AM, richard claxton > > >> wrote: > >> > I am new to erlang and i am playing around with mochijson2, JSON > parsing > >> > from a http post, I can decode the json http request if i escape all > the > >> > quotes in the string \" but there must be a better solution then this? > >> > Cheer Richard > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > >> > > >> > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ali.sabil@REDACTED Tue Jun 7 16:21:56 2011 From: ali.sabil@REDACTED (Ali Sabil) Date: Tue, 7 Jun 2011 16:21:56 +0200 Subject: [erlang-questions] Get cookie time-out In-Reply-To: References: Message-ID: Hi, On Tue, Jun 7, 2011 at 2:51 PM, Olivier BOUDEVILLE wrote: > > Hello, > > We are running a distributed application over a cluster and recently, during > its deployment over 45 nodes, we have encountered a time-out apparently > related to the inter-node cookie retrieval: > "{timeout,{gen_server,call,[auth,{get_cookie,'SOME_NODE...". > > We are not that familiar with OTP, but apparently there could be a way of > specifying a time-out for such an internal gen_server: we have indeed in > lib/kernel/src/auth.erl: > > """ > start_link() -> > ? ? gen_server:start_link({local, auth}, auth, [], []). > """ > > I suppose we could add a {timeout,Time} option there ? (there must be a > default time-out, even if I do not know its value) The default timeout value for gen_server:call() is 5000 ms, maybe you could try replacing the gen_server:call/2 with a gen_server:call/3 in lib/kernel/src/auth.erl so that: """ get_cookie(Node) -> gen_server:call(auth, {get_cookie, Node}). """ becomes: """ get_cookie(Node) -> gen_server:call(auth, {get_cookie, Node}, infinity). """ > Unless there is a less hackish way of waiting longer, maybe with some > command-line option? > More generally, maybe the default value for that time-out should be > increased? > > Thanks in advance for any hint, > Best regards, > > Olivier Boudeville. > --------------------------- > Olivier Boudeville > > EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France > D?partement SINETICS, groupe ASICS (I2A), bureau B-226 > Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 > 27 13 > > > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis > ? l'intention exclusive des destinataires et les informations qui y figurent > sont strictement confidentielles. Toute utilisation de ce Message non > conforme ? sa destination, toute diffusion ou toute publication totale ou > partielle, est interdite sauf autorisation expresse. > > Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le > copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. > Si vous avez re?u ce Message par erreur, merci de le supprimer de votre > syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur > quelque support que ce soit. Nous vous remercions ?galement d'en avertir > imm?diatement l'exp?diteur par retour du message. > > Il est impossible de garantir que les communications par messagerie > ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute > erreur ou virus. > ____________________________________________________ > > This message and any attachments (the 'Message') are intended solely for the > addressees. The information contained in this Message is confidential. Any > use of information contained in this Message not in accord with its purpose, > any dissemination or disclosure, either whole or partial, is prohibited > except formal approval. > > If you are not the addressee, you may not copy, forward, disclose or use any > part of it. If you have received this message in error, please delete it and > all copies from your system and notify the sender immediately by return > message. > > E-mail communication cannot be guaranteed to be timely secure, error or > virus-free. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From antoine.koener@REDACTED Tue Jun 7 16:25:26 2011 From: antoine.koener@REDACTED (Antoine Koener) Date: Tue, 7 Jun 2011 16:25:26 +0200 Subject: [erlang-questions] Get cookie time-out In-Reply-To: References: Message-ID: On Jun 7, 2011, at 16:21 , Ali Sabil wrote: > Hi, > > On Tue, Jun 7, 2011 at 2:51 PM, Olivier BOUDEVILLE > wrote: >> >> Hello, >> >> We are running a distributed application over a cluster and >> recently, during >> its deployment over 45 nodes, we have encountered a time-out >> apparently >> related to the inter-node cookie retrieval: >> "{timeout,{gen_server,call,[auth,{get_cookie,'SOME_NODE...". >> >> We are not that familiar with OTP, but apparently there could be a >> way of >> specifying a time-out for such an internal gen_server: we have >> indeed in >> lib/kernel/src/auth.erl: >> >> """ >> start_link() -> >> gen_server:start_link({local, auth}, auth, [], []). >> """ >> >> I suppose we could add a {timeout,Time} option there ? (there must >> be a >> default time-out, even if I do not know its value) > > The default timeout value for gen_server:call() is 5000 ms, maybe you > could try replacing the gen_server:call/2 with a gen_server:call/3 in > lib/kernel/src/auth.erl so that: > """ > get_cookie(Node) -> > gen_server:call(auth, {get_cookie, Node}). > """ > > becomes: > """ > get_cookie(Node) -> > gen_server:call(auth, {get_cookie, Node}, infinity). > """ Take care that doing so means that the calling process will be blocked... I'm not sure that this will fix anything. Instead of infinity, may be increase the timeout, why not 60000 (1 minute) ? From pablo.platt@REDACTED Tue Jun 7 17:15:32 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 7 Jun 2011 08:15:32 -0700 (PDT) Subject: [erlang-questions] Http client sending http response to wrong handler In-Reply-To: References: Message-ID: <105283.64112.qm@web112611.mail.gq1.yahoo.com> I think I had the same problem. Try the following settings before you send the first request: httpc:set_options([{max_pipeline_length, 0}, {max_sessions, NRequests}, {max_keep_alive_length, 0}]) NRequests is the number of concurrent requests you are going to use. ________________________________ From: Suma Shivaprasad To: erlang-questions@REDACTED Sent: Monday, June 6, 2011 12:45 PM Subject: [erlang-questions] Http client sending http response to wrong handler Our app makes a lot of HTTP requests and we are facing this issue with both inets-5.5.1 and 5.3.2. Basically our receive clause for the response is trying to match the request id returned in httpc:request call and we see that the match always fails with the wrong Request Id . We gave seen this mismatch in all 3 receive clauses for stream_start, stream and stream_end What we observed after a lot of trial and error? is that if the same pid makes? the http requests , the responses get kind of muddled up but if we spawn a separate process for the httpc:request ,? it is better . But here too we have seen some occurrences of this faulty behaviour. Has anyone faced this issue ? _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivier.boudeville@REDACTED Tue Jun 7 17:21:51 2011 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Tue, 7 Jun 2011 17:21:51 +0200 Subject: [erlang-questions] Get cookie time-out In-Reply-To: Message-ID: Thanks Ali and Antoine for your answer! I guess we will indeed use a finite, yet increased, time-out for that. This is the first time we will have to patch our Erlang version though; hopefully, as the scale increases, other exchanges will not require hardcoded time-outs to be modified. As I see it, in some cases, time-outs have indeed to be set to actual values (not infinity, for example to overcome hardware faults), but then these values are arbitrary and context-dependent. In our user code we ended up defining profiles (development/production), as no single set of values could be used in all contexts. Depending on how many constant time-out values are lurking in the Erlang code itself, maybe it could be useful to do the same there? Not strictly related, but still in the proper distributed initialization of Erlang nodes, is there any feedback of the Erlang maintaining team about the code_server:call/2 suspected race condition ? (http://erlang.2086793.n4.nabble.com/code-server-call-2-problem-td3387884.html) Thanks again, Best regards, Olivier Boudeville. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 antoine.koener@REDACTED 07/06/2011 16:25 A ali.sabil@REDACTED cc olivier.boudeville@REDACTED, erlang-questions@REDACTED Objet Re: [erlang-questions] Get cookie time-out On Jun 7, 2011, at 16:21 , Ali Sabil wrote: > Hi, > > On Tue, Jun 7, 2011 at 2:51 PM, Olivier BOUDEVILLE > wrote: >> >> Hello, >> >> We are running a distributed application over a cluster and >> recently, during >> its deployment over 45 nodes, we have encountered a time-out >> apparently >> related to the inter-node cookie retrieval: >> "{timeout,{gen_server,call,[auth,{get_cookie,'SOME_NODE...". >> >> We are not that familiar with OTP, but apparently there could be a >> way of >> specifying a time-out for such an internal gen_server: we have >> indeed in >> lib/kernel/src/auth.erl: >> >> """ >> start_link() -> >> gen_server:start_link({local, auth}, auth, [], []). >> """ >> >> I suppose we could add a {timeout,Time} option there ? (there must >> be a >> default time-out, even if I do not know its value) > > The default timeout value for gen_server:call() is 5000 ms, maybe you > could try replacing the gen_server:call/2 with a gen_server:call/3 in > lib/kernel/src/auth.erl so that: > """ > get_cookie(Node) -> > gen_server:call(auth, {get_cookie, Node}). > """ > > becomes: > """ > get_cookie(Node) -> > gen_server:call(auth, {get_cookie, Node}, infinity). > """ Take care that doing so means that the calling process will be blocked... I'm not sure that this will fix anything. Instead of infinity, may be increase the timeout, why not 60000 (1 minute) ? Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Jun 7 17:33:55 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 7 Jun 2011 17:33:55 +0200 Subject: [erlang-questions] ets:match_delete Message-ID: dear list, i'm trying to match delete elements from an ETS table which have one of the tuple elements greater than 2. start() -> ets:new(test, [named_table, set, protected]), ets:insert(test, {a, 1, true, true}), ets:insert(test, {b, 2, true, true}), ets:insert(test, {c, 3, true, true}), ets:insert(test, {d, 2, true, true}), ets:insert(test, {e, 1, true, true}), ets:match_delete(test, [{['_', '$1', '_', '_'], [{'>', '$1', 2}], []}]), Result = ets:tab2list(test), ets:delete(test), Result. this however does return the whole table. any hints on what i'm doing wrong? r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobias.rodaebel@REDACTED Tue Jun 7 17:43:49 2011 From: tobias.rodaebel@REDACTED (Tobias) Date: Tue, 7 Jun 2011 17:43:49 +0200 Subject: [erlang-questions] ets:match_delete In-Reply-To: References: Message-ID: <197A6F9D-D015-426F-8879-1A6CDEC7FD95@googlemail.com> Hi Roberto, have you tried N = ets:select_delete(test, [{{'$1', '$2', '$3', '$4'}, [{'>', '$2', 2}], [true]}]), instead? Cheers, Tobias On Jun 7, 2011, at 5:33 PM, Roberto Ostinelli wrote: > dear list, > > i'm trying to match delete elements from an ETS table which have one of the tuple elements greater than 2. > > start() -> > ets:new(test, [named_table, set, protected]), > ets:insert(test, {a, 1, true, true}), > ets:insert(test, {b, 2, true, true}), > ets:insert(test, {c, 3, true, true}), > ets:insert(test, {d, 2, true, true}), > ets:insert(test, {e, 1, true, true}), > ets:match_delete(test, [{['_', '$1', '_', '_'], [{'>', '$1', 2}], []}]), > Result = ets:tab2list(test), > ets:delete(test), > Result. > > this however does return the whole table. > > any hints on what i'm doing wrong? > > r. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From igorrs@REDACTED Tue Jun 7 17:57:43 2011 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Tue, 7 Jun 2011 12:57:43 -0300 Subject: [erlang-questions] [erlang-bugs] All keys in the same slot (Mnesia/dets) Message-ID: Both mnesia_frag_hash and dets use linear hashing, so I don't believe there would be any improvement in this case. Thanks. Igor. Em 07/06/2011 05:01, "Michael Truog" escreveu: -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobias.rodaebel@REDACTED Tue Jun 7 18:01:03 2011 From: tobias.rodaebel@REDACTED (Tobias) Date: Tue, 7 Jun 2011 18:01:03 +0200 Subject: [erlang-questions] ets:match_delete In-Reply-To: <197A6F9D-D015-426F-8879-1A6CDEC7FD95@googlemail.com> References: <197A6F9D-D015-426F-8879-1A6CDEC7FD95@googlemail.com> Message-ID: <5437F5C8-909E-446A-BBBD-AA08149B7825@googlemail.com> I guess the problem here is, that ets:match_delete/2 takes a pattern as second argument while ets:select_delete/2 wants a match_spec(). But maybe I'm mixing things up myself... On Jun 7, 2011, at 5:43 PM, Tobias wrote: > Hi Roberto, > > have you tried > > N = ets:select_delete(test, [{{'$1', '$2', '$3', '$4'}, [{'>', '$2', 2}], [true]}]), > > instead? > > Cheers, > Tobias > > On Jun 7, 2011, at 5:33 PM, Roberto Ostinelli wrote: > >> dear list, >> >> i'm trying to match delete elements from an ETS table which have one of the tuple elements greater than 2. >> >> start() -> >> ets:new(test, [named_table, set, protected]), >> ets:insert(test, {a, 1, true, true}), >> ets:insert(test, {b, 2, true, true}), >> ets:insert(test, {c, 3, true, true}), >> ets:insert(test, {d, 2, true, true}), >> ets:insert(test, {e, 1, true, true}), >> ets:match_delete(test, [{['_', '$1', '_', '_'], [{'>', '$1', 2}], []}]), >> Result = ets:tab2list(test), >> ets:delete(test), >> Result. >> >> this however does return the whole table. >> >> any hints on what i'm doing wrong? >> >> r. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From roberto@REDACTED Tue Jun 7 18:07:51 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 7 Jun 2011 18:07:51 +0200 Subject: [erlang-questions] ets:match_delete In-Reply-To: <197A6F9D-D015-426F-8879-1A6CDEC7FD95@googlemail.com> References: <197A6F9D-D015-426F-8879-1A6CDEC7FD95@googlemail.com> Message-ID: 2011/6/7 Tobias > Hi Roberto, > > have you tried > > N = ets:select_delete(test, [{{'$1', '$2', '$3', '$4'}, [{'>', '$2', 2}], > [true]}]), > > instead? > > Cheers, > Tobias > > > i hadn't, and thank you since this works beautifully. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nox@REDACTED Tue Jun 7 18:52:34 2011 From: nox@REDACTED (Anthony Ramine) Date: Tue, 7 Jun 2011 18:52:34 +0200 Subject: [erlang-questions] ets:match_delete In-Reply-To: References: Message-ID: Le 7 juin 2011 ? 17:33, Roberto Ostinelli a ?crit : > dear list, > > i'm trying to match delete elements from an ETS table which have one of the tuple elements greater than 2. > > start() -> > ets:new(test, [named_table, set, protected]), > ets:insert(test, {a, 1, true, true}), > ets:insert(test, {b, 2, true, true}), > ets:insert(test, {c, 3, true, true}), > ets:insert(test, {d, 2, true, true}), > ets:insert(test, {e, 1, true, true}), > ets:match_delete(test, [{['_', '$1', '_', '_'], [{'>', '$1', 2}], []}]), > Result = ets:tab2list(test), > ets:delete(test), > Result. > > this however does return the whole table. > > any hints on what i'm doing wrong? > > r. [{{'_','$1','_','_'},[{'>','$1',2}],[]}] should work. You put a list instead of a tuple for the match argument. By the way I didn't read the doc to figure it out, I just did ets:fun2ms(fun ({_,X,_,_}) when X > 2 -> X end), which is pretty good for figuring out how to write matchspecs. Regards, -- Anthony Ramine Dev:Extend http://dev-extend.eu From roberto@REDACTED Tue Jun 7 19:15:34 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 7 Jun 2011 19:15:34 +0200 Subject: [erlang-questions] ets:match_delete In-Reply-To: References: Message-ID: 2011/6/7 Anthony Ramine > > [{{'_','$1','_','_'},[{'>','$1',2}],[]}] should work. You put a list > instead of a tuple for the match argument. > By the way I didn't read the doc to figure it out, I just did > ets:fun2ms(fun ({_,X,_,_}) when X > 2 -> X end), > which is pretty good for figuring out how to write matchspecs. > hi anthony, no this does not work. furthermore, i took the list format from the 2nd example provided in the docs: http://www.erlang.org/doc/apps/erts/match_spec.html#id71596 thank you anyway :) r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nox@REDACTED Tue Jun 7 19:26:15 2011 From: nox@REDACTED (Anthony Ramine) Date: Tue, 7 Jun 2011 19:26:15 +0200 Subject: [erlang-questions] ets:match_delete In-Reply-To: References: Message-ID: Le 7 juin 2011 ? 19:15, Roberto Ostinelli a ?crit : > 2011/6/7 Anthony Ramine > > [{{'_','$1','_','_'},[{'>','$1',2}],[]}] should work. You put a list instead of a tuple for the match argument. > By the way I didn't read the doc to figure it out, I just did ets:fun2ms(fun ({_,X,_,_}) when X > 2 -> X end), > which is pretty good for figuring out how to write matchspecs. > > hi anthony, > > no this does not work. > > furthermore, i took the list format from the 2nd example provided in the docs: > http://www.erlang.org/doc/apps/erts/match_spec.html#id71596 > > thank you anyway :) > > r. My bad then, ets:match_delete must behave differently than lookup then, because this works: ets:test_ms({a, 3, b, c}, [{{'_', '$1', '_', '_'},[{'>', '$1', 2}],['it works!']}]) -- Anthony Ramine Dev:Extend http://dev-extend.eu From n.oxyde@REDACTED Tue Jun 7 19:35:21 2011 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 7 Jun 2011 19:35:21 +0200 Subject: [erlang-questions] ets:match_delete In-Reply-To: References: Message-ID: <4B1BC3E9-49D5-4F43-ADBE-EF65C2425CC9@gmail.com> Le 7 juin 2011 ? 19:26, Anthony Ramine a ?crit : > Le 7 juin 2011 ? 19:15, Roberto Ostinelli a ?crit : > >> 2011/6/7 Anthony Ramine >> >> [{{'_','$1','_','_'},[{'>','$1',2}],[]}] should work. You put a list instead of a tuple for the match argument. >> By the way I didn't read the doc to figure it out, I just did ets:fun2ms(fun ({_,X,_,_}) when X > 2 -> X end), >> which is pretty good for figuring out how to write matchspecs. >> >> hi anthony, >> >> no this does not work. >> >> furthermore, i took the list format from the 2nd example provided in the docs: >> http://www.erlang.org/doc/apps/erts/match_spec.html#id71596 >> >> thank you anyway :) >> >> r. > > My bad then, ets:match_delete must behave differently than lookup then, because this works: > > ets:test_ms({a, 3, b, c}, [{{'_', '$1', '_', '_'},[{'>', '$1', 2}],['it works!']}]) > Oh ok I understand, I didn't know what to return in the fun from my ets:fun2ms/1 call. It should have returned true, thus making the last ms component as [true]. Then you end up with the same thing as Tobias :) -- Anthony Ramine Dev:Extend http://dev-extend.eu From attila.r.nohl@REDACTED Tue Jun 7 20:35:59 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 7 Jun 2011 19:35:59 +0100 Subject: [erlang-questions] Compiling erlang with static glibc for OTP releases In-Reply-To: References: Message-ID: As far as I remember, if you compile libc statically, the binary will be very much dependent on the kernel version, so the situation might not be better... 2011/6/7, Jordan Wilberding : > Max, > > Were you able to come up with a fully statically compiled erlang? I am > looking to doing the same thing. > > Thanks! > JW > > On Mon, May 9, 2011 at 6:45 AM, Max Lapshin wrote: > >> I'm trying to migrate to OTP releases (using rebar) for erlyvideo. >> Goal is to minify problems, build dependencies and package build farm. >> >> I've compiled libcrypt in a static manner into erlang so, that there >> will be no problems on different versions (debian squeeze and debian >> sid use different versions), >> but problems go on: different versions of libc. >> >> I'm thinking about building erlang with all static dependencies so, >> that packages from one build machine will be useable on all 64bit >> linuxes. >> >> Is it a good idea to link erlang VM with glibc static? >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From jameschurchman@REDACTED Tue Jun 7 22:19:40 2011 From: jameschurchman@REDACTED (James Churchman) Date: Tue, 7 Jun 2011 21:19:40 +0100 Subject: [erlang-questions] Specs Message-ID: Hi guys When you parse a file with the epp module it has no problems with specs However when using erl_scan then passing on to erl_parse, in order to parse a string not a file, it fails on -specs in the code Is there a spec aware version of the erl_parse module ? James From kostis@REDACTED Tue Jun 7 22:40:46 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 07 Jun 2011 23:40:46 +0300 Subject: [erlang-questions] Specs In-Reply-To: References: Message-ID: <4DEE8CCE.9070202@cs.ntua.gr> James Churchman wrote: > Hi guys > > When you parse a file with the epp module it has no problems with specs > > However when using erl_scan then passing on to erl_parse, in order to parse a string not a file, it fails on -specs in the code > > Is there a spec aware version of the erl_parse module ? I find this question a bit strange... the erl_parse module *is* spec aware. This is actually how specs are parsed in the system. Does the following do what you want? 42> SpecInString = "-spec my_function(some_type()) -> 42 | 'gazonk'.", 42> {ok, Toks, 1} = erl_scan:string(SpecInString), 42> erl_parse:parse_form(Toks). {ok,{attribute,1,spec, {{my_function,1}, [{type,1,'fun', [{type,1,product,[{type,1,some_type,[]}]}, {type,1,union,[{integer,1,42},{atom,1,gazonk}]}]}]}}} Kostis From michael.eugene.turner@REDACTED Wed Jun 8 09:09:15 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 8 Jun 2011 16:09:15 +0900 Subject: [erlang-questions] seq_trace Serial - a unique event ID? Message-ID: According to the manual: "The trace token contains a component which is called serial. It consists of two integers Previous and Current. The purpose is to uniquely identify each traced event within a trace sequence and to order the messages chronologically and in the different branches if any." http://www.erlang.org/doc/man/seq_trace.html Is that passage actually asserting that Serial can "uniquely identify" a seq_trace event? Or is it only implying that Serial *helps* to uniquely identify those events? If it's "to uniquely identify", is unique identification of seq_trace events actually a proven property of the Serial updating algorithm given? (I assume that "event" means "either a seq_trace:print() call or a traceable message between processes". Serial can't uniquely identify each trace message sent to the tracer process, because if you enable tracing of both sends and receives, you'll get the same Serial value at the tracer process when it records both a traced send and its corresponding (possibly much later) receive.) Also: if seq_trace is based on some classic method for handling "sequential tracing", is there a formal treatment of of this method somewhere? Searching on "sequential tracing" and some other likely terms just turns up ... the Erlang seq_trace manual page. -michael turner -------------- next part -------------- An HTML attachment was scrubbed... URL: From yrashk@REDACTED Wed Jun 8 10:13:54 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Wed, 8 Jun 2011 01:13:54 -0700 (PDT) Subject: [erlang-questions] is CEAN maintained? In-Reply-To: Message-ID: <30012253.358.1307520834207.JavaMail.geo-discussion-forums@yqnd17> To my best knowledge, Jacob is not planning much for neither epm nor sutro at this moment, if I got this right. On Monday, June 6, 2011 9:44:49 AM UTC, Gleb Peregud wrote: > > On Mon, Jun 6, 2011 at 10:22, Christophe Romain > wrote: > > we're in discussion with Yurii Rashkovskii (erlagner.org) and Martin > > Logan (erlware.org) to provide community a common packaging environment > > by the way. we'll try to merge our efforts to finally end up with this > > packaging issue. > > Please include Jacob Vorreuter to the discussion. His EPM is a pretty > handy thing too: > > https://github.com/JacobVorreuter/epm > > also it would be nice for rebar to support that common packaging > environment and/or make it support specific package manager as sort of > a plugin. > > Just my $0.02 > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker@REDACTED Wed Jun 8 15:55:35 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Wed, 8 Jun 2011 15:55:35 +0200 Subject: [erlang-questions] NIF + HiPE - can it works together? In-Reply-To: References: <19949.17000.27340.404061@pilspetsen.it.uu.se> Message-ID: <4DEF7F57.4090008@erix.ericsson.se> I don't know if it has been stated, but HiPE and NIFs do not play nice together *within the same module*. Using HiPE and NIFs on different modules within the same VM will hopefully work fine. There seem to be a number of problems here. 1. erlang:load_nif fails as reported by Rados?aw Bu?at. This seems to be a "simple" bug that can be fixed, but even if so... 2. The NIF overriding mechanism is not implemented for hipe and would probably yield unpredictable results. 3. Hipe compiler fails on on_load as reported. I will look into the first two. Not sure if #3 was expected not to work? (Mikael? Kostis?) /Sverker, Erlang/OTP Paul Davis wrote: > I'm fairly certain that I saw somewhere that HiPE is incompatible with > NIF's but I'm unable to actually Google a reference so I could be > making that up based on that on_init error that I've seen before. > > If so, I'd probably just shove the code you want HiPE compiled into a > lib module and have the nif stubs call out to that as necessary. > > HTH, > Paul Davis > > 2011/6/6 Rados?aw Bu?at : > >> 2011/6/7 Rados?aw Bu?at : >> >>> 2011/6/6 Mikael Pettersson : >>> >>>> Please post a standalone and minimal test case. >>>> >>>> >>> Yeah, sure. >>> >>> >> One more thing. When module is compiled with HiPE and try to use >> on_load attribute it ends with error: >> >> EXITED with reason {'trans_fun/2',on_load} >> @hipe_beam_to_icode:1148 >> >> =ERROR REPORT==== 7-Jun-2011::01:18:36 === >> Error in process <0.50.0> with exit value: >> {{badmatch,{'EXIT',{{hipe_beam_to_icode,1148,{'trans_fun/2',on_load}},[{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode,trans_mfa_code,5},{hipe_beam_to_icode,trans_beam_function_chunk... >> >> src/nif_with_hipe.erl:none: internal error in native_compile; >> crash reason: {{badmatch, >> {'EXIT', >> {{hipe_beam_to_icode,1148,{'trans_fun/2',on_load}}, >> [{hipe_beam_to_icode,trans_fun,2}, >> {hipe_beam_to_icode,trans_fun,2}, >> {hipe_beam_to_icode,trans_mfa_code,5}, >> {hipe_beam_to_icode,trans_beam_function_chunk,2}, >> {hipe_beam_to_icode,'-module/2-lc$^1/1-1-',2}, >> {hipe,get_beam_icode,4}, >> {hipe,'-run_compiler_1/3-fun-0-',4}]}}}, >> [{hipe,get_beam_icode,4},{hipe,'-run_compiler_1/3-fun-0-',4}]} >> >> -- >> Pozdrawiam >> >> Rados?aw Bu?at >> http://radarek.jogger.pl - m?j blog >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From mikpe@REDACTED Wed Jun 8 17:08:58 2011 From: mikpe@REDACTED (Mikael Pettersson) Date: Wed, 8 Jun 2011 17:08:58 +0200 Subject: [erlang-questions] NIF + HiPE - can it works together? In-Reply-To: <4DEF7F57.4090008@erix.ericsson.se> References: <19949.17000.27340.404061@pilspetsen.it.uu.se> <4DEF7F57.4090008@erix.ericsson.se> Message-ID: <19951.37002.43330.882893@pilspetsen.it.uu.se> On Wed, 8 Jun 2011 15:55:35 +0200, Sverker Eriksson wrote: > I don't know if it has been stated, but HiPE and NIFs do not play nice=20 > together *within the same module*. > Using HiPE and NIFs on different modules within the same VM will=20 > hopefully work fine. > > > There seem to be a number of problems here. > > 1. erlang:load_nif fails as reported by Rados=C5=82aw Bu=C5=82at. This se= > ems to be=20 > a "simple" bug that can be fixed, but even if so... > > 2. The NIF overriding mechanism is not implemented for hipe and would=20 > probably yield unpredictable results. > > 3. Hipe compiler fails on on_load as reported. > > I will look into the first two. Not sure if #3 was expected not to=20 > work? (Mikael? Kostis?) Last time I checked on_load was unsupportable by HiPE. The BEAM loader did some hacks behind the scenes that it could get away with, but which were unexpected by and incompatible with the HiPE loader. Keeping HiPE and NIFs separate seems to be the only viable solution currently. Alas there are effectively no compiler/vm developers left in the HiPE team; we've all left to pursue other career paths. /Mikael > > > /Sverker, Erlang/OTP > > Paul Davis wrote: > > I'm fairly certain that I saw somewhere that HiPE is incompatible with > > NIF's but I'm unable to actually Google a reference so I could be > > making that up based on that on_init error that I've seen before. > > > > If so, I'd probably just shove the code you want HiPE compiled into a > > lib module and have the nif stubs call out to that as necessary. > > > > HTH, > > Paul Davis > > > > 2011/6/6 Rados=C5=82aw Bu=C5=82at : > > =20 > >> 2011/6/7 Rados=C5=82aw Bu=C5=82at : > >> =20 > >>> 2011/6/6 Mikael Pettersson : > >>> =20 > >>>> Please post a standalone and minimal test case. > >>>> > >>>> =20 > >>> Yeah, sure. > >>> > >>> =20 > >> One more thing. When module is compiled with HiPE and try to use > >> on_load attribute it ends with error: > >> > >> EXITED with reason {'trans_fun/2',on_load} > >> @hipe_beam_to_icode:1148 > >> > >> =3DERROR REPORT=3D=3D=3D=3D 7-Jun-2011::01:18:36 =3D=3D=3D > >> Error in process <0.50.0> with exit value: > >> {{badmatch,{'EXIT',{{hipe_beam_to_icode,1148,{'trans_fun/2',on_load}},= > [{hipe_beam_to_icode,trans_fun,2},{hipe_beam_to_icode,trans_fun,2},{hipe_= > beam_to_icode,trans_mfa_code,5},{hipe_beam_to_icode,trans_beam_function_c= > hunk... > >> > >> src/nif_with_hipe.erl:none: internal error in native_compile; > >> crash reason: {{badmatch, > >> {'EXIT', > >> {{hipe_beam_to_icode,1148,{'trans_fun/2',on_load= > }}, > >> [{hipe_beam_to_icode,trans_fun,2}, > >> {hipe_beam_to_icode,trans_fun,2}, > >> {hipe_beam_to_icode,trans_mfa_code,5}, > >> {hipe_beam_to_icode,trans_beam_function_chunk,= > 2}, > >> {hipe_beam_to_icode,'-module/2-lc$^1/1-1-',2},= > > >> {hipe,get_beam_icode,4}, > >> {hipe,'-run_compiler_1/3-fun-0-',4}]}}}, > >> [{hipe,get_beam_icode,4},{hipe,'-run_compiler_1/3-fun-0-= > ',4}]} > >> > >> -- > >> Pozdrawiam > >> > >> Rados=C5=82aw Bu=C5=82at > >> http://radarek.jogger.pl - m=C3=B3j blog > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> =20 > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > =20 > > > > From igorrs@REDACTED Wed Jun 8 21:48:01 2011 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Wed, 8 Jun 2011 16:48:01 -0300 Subject: [erlang-questions] [erlang-bugs] All keys in the same slot (Mnesia/dets) In-Reply-To: References: Message-ID: Copying erlang-questions. BTW, I'm still trying to understand why some fragments ended up with many more slots after the upgrade I described in the original post. Is there a way to manually raise the number of slots in a dets table? I guess I'll have to do that now. Thanks. Igor. On Tue, Jun 7, 2011 at 1:12 PM, Igor Ribeiro Sucupira wrote: > > I know I can use another module instead of mnesia_frag_hash (and we have > tables that don't use it). > > But it's too late now. I don't see a way to do that without much pain. The > cheapest approach I can think of is to create other tables, change the code > to use both versions of each table and then migrate the data. It will take a > month. :-( > > Changing dets would be less painful, but I don't know if it's "doable". > > Is this pitfall with table fragmentation documented anywhere? > > Thanks. > Igor. > > Em 07/06/2011 05:01, "Erik S?e S?rensen" escreveu: > > Igor Ribeiro Sucupira wrote: > Hum... mnesia_frag_hash and dets both use phash2, so it makes sense > that the keys are poorly distributed among the slots of each dets > table that is a disc_only_copies fragment. :-( > I guess we'll have to deal with that somehow. > > > I needed two independent hash functions at some point, and noticed that phash2(X) and phash2({X}) appear to be independent. > Is that a good way to achieve independent hash functions? > And is it applicable in this situation? > > Ergards, > Erik S?e S?rensen From fan3@REDACTED Thu Jun 9 00:34:37 2011 From: fan3@REDACTED (Fan, Victor) Date: Wed, 8 Jun 2011 15:34:37 -0700 Subject: [erlang-questions] Compatability with VXWorks 5.4? Message-ID: <09C2EEC6C9ADD34DA12E89615542F7B70100CA55A866@NSPEXMBX-A.the-lab.llnl.gov> I'm a summer student with Lawrence Livermore National Laboratory evaluating a bunch of different embedded middleware solutions for some old VXWorks machines we have. One of the more ambitious proposals is a RabbitMQ + Erlang lashup, but I'm having trouble convincing Erlang/OTP to compile against a VXWorks 5.4 target. The provided VXWorks xconf files were written for the VXWorks 6.2 toolchain, and simply hacking out the paths and replacing them with the Wind 2.0 toolchain isn't working (./configure throws a fit because of its use of functions called "main" in its test suites, which doesn't mesh well with VXWorks). Does anyone have experience or (even better!) a guide about getting Erlang running on an old VXWorks machine? It seems that people have managed to get older releases of OTP compiled to a 5.2/5.4 target, but RabbitMQ lists as a requirement a "relatively recent" version of Erlang. Thanks, -Victor Fan -------------- next part -------------- An HTML attachment was scrubbed... URL: From techabc@REDACTED Thu Jun 9 05:20:45 2011 From: techabc@REDACTED (techabc) Date: Wed, 8 Jun 2011 20:20:45 -0700 (PDT) Subject: [erlang-questions] snmp vacm.conf erronous behaviour In-Reply-To: <94C5430AA547F24AA801BA5706F29D2E25CE56@srvkeyx01.KEYTRONIX.local> References: <94C5430AA547F24AA801BA5706F29D2E25CE56@srvkeyx01.KEYTRONIX.local> Message-ID: <1307589645777-3584328.post@n4.nabble.com> when I do NOT use v3, How can I config the vacm.conf to enable all mibs? I means disable the view based mib ACM. -- View this message in context: http://erlang.2086793.n4.nabble.com/snmp-vacm-conf-erronous-behaviour-tp2111375p3584328.html Sent from the Erlang Questions mailing list archive at Nabble.com. From ben@REDACTED Thu Jun 9 12:55:20 2011 From: ben@REDACTED (Ben Godfrey) Date: Thu, 9 Jun 2011 11:55:20 +0100 Subject: [erlang-questions] Running a release as a Windows service with erlsrv.exe and start_erl.exe In-Reply-To: References: Message-ID: Thanks Siri, Is there anything I can do to help? Ben On 7 June 2011 09:11, Siri Hansen wrote: > Hi Ben! > Unfortunately, the release handling for windows, including erlsrv.erl and > start_erl.c, is broken. This was discovered not long ago, after refreshing > the release_handler test suite where these tests were skipped on windows. We > are currently working on this issue, and plan to release a correction as > soon as possible. > Regards > siri@REDACTED > > 2011/6/6 Ben Godfrey >> >> Hello all, >> >> I've been trying to deploy an Erlang application as a Windows service >> and not having much luck. >> >> I built a release package using rebar. I'm registering the service as >> follows: >> >> ? ?c:\erl5.8.3\erts-5.8.3\bin\erlsrv.exe add Rolf -c "Collects system >> data for monitoring." -w c:\rolf -m >> c:\erl5.8.3\erts-5.8.3\bin\start_erl.exe -debugtype reuse -args >> "-setcookie xyz123 ++ -reldir c:\rolf\releases" >> >> This fails with the error "Failed to query BinDir of release." in the >> .debug file. Looking at the code for start_erl.exe, this is related to >> registry keys. I can't see any sign of any keys which match the >> pattern described in the source, but curiously the preceding call to >> open a registry key succeeds [2]. >> >> Any one have any idea about any of this stuff? >> >> Server is Windows Server 2008. >> >> Ben >> >> [1] https://github.com/erlang/otp/blob/dev/erts/etc/win32/start_erl.c#L364 >> [2] https://github.com/erlang/otp/blob/dev/erts/etc/win32/start_erl.c#L342 >> >> -- >> Ben Godfrey, Wandering Hacker >> http://aftnn.org | http://twitter.com/afternoon | >> http://www.linkedin.com/in/bengodfrey >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- Ben Godfrey, Wandering Hacker http://aftnn.org | http://twitter.com/afternoon | http://www.linkedin.com/in/bengodfrey From opensource@REDACTED Thu Jun 9 13:19:42 2011 From: opensource@REDACTED (Paul G Webster) Date: Thu, 09 Jun 2011 12:19:42 +0100 Subject: [erlang-questions] General help with unix sockets Message-ID: Good day, I was wondering if anyone could lend a hand; I am trying to use erlang to connect to a unix socket '/var/pexim.sock' I spied out 'procket' it looks good but I am a little to new to erlang to figure out howto actually use the toolkit to connect.. Would anyone be able to give me a hand; it does not even need to be procket .. just a simple connection script for a unix socket If you could help it would be excellent p.s. If this actually ends up on the list twice; my apologies :) -- Thank you for your time -- Paul G Webster -- Using Opera's revolutionary email client: http://www.opera.com/mail/ From michael.santos@REDACTED Thu Jun 9 15:03:43 2011 From: michael.santos@REDACTED (Michael Santos) Date: Thu, 9 Jun 2011 09:03:43 -0400 Subject: [erlang-questions] General help with unix sockets In-Reply-To: References: Message-ID: <20110609130343.GB3824@ecn.lan> On Thu, Jun 09, 2011 at 12:19:42PM +0100, Paul G Webster wrote: > Good day, > > I was wondering if anyone could lend a hand; I am trying to use > erlang to connect to a unix socket '/var/pexim.sock' I spied out > 'procket' it looks good but I am a little to new to erlang to figure > out howto actually use the toolkit to connect.. > > Would anyone be able to give me a hand; it does not even need to be > procket .. just a simple connection script for a unix socket > > If you could help it would be excellent > > p.s. If this actually ends up on the list twice; my apologies :) There are a few ways to connect to Unix sockets from Erlang. I responded to your other email privately but for the archives my blog has some info about it: http://listincomprehension.com/2010/12/unix-sockets.html Assuming that /var/pexim.sock is a stream socket (is that path right?) and you're using procket, the steps are similar to opening a socket in C. The layout of a struct sockaddr_un will depend on your OS. For example, it's defined in /usr/include/linux/un.h as: #define UNIX_PATH_MAX 108 struct sockaddr_un { sa_family_t sun_family; /* AF_UNIX */ char sun_path[UNIX_PATH_MAX]; /* pathname */ }; So the code to open the socket would look something like (untested): Path = <<"/var/pexim.sock">>, PF_LOCAL = 1, % Get a Unix stream socket fd {ok, Socket} = procket:socket(local, stream, 0), Len = 108 - byte_size(Path), % Linux sockaddr_un Sun = <>, ok = procket:connect(Socket, Sun), {ok, Buf} = procket:read(Socket, 16#FFFF), ok = procket:write(Socket, Buf), ok = procket:close(). I've left out the portability stuff for running on both BSD/Linux. Hope that helps! From joelr1@REDACTED Thu Jun 9 15:25:25 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 9 Jun 2011 06:25:25 -0700 (PDT) Subject: [erlang-questions] NIF + HiPE - can it works together? In-Reply-To: <19951.37002.43330.882893@pilspetsen.it.uu.se> Message-ID: <28150537.1296.1307625925806.JavaMail.geo-discussion-forums@vbyw6> On Wednesday, June 8, 2011 7:08:58 PM UTC+4, Mikael Pettersson wrote: > > Alas there are effectively no compiler/vm developers left in the HiPE team; > we've all left to pursue other career paths. > Does this mean that HiPE is up for adoption? It wasn't very cuddly last time I looked at it but... -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethstone@REDACTED Thu Jun 9 15:36:38 2011 From: kennethstone@REDACTED (Kenny Stone) Date: Thu, 9 Jun 2011 08:36:38 -0500 Subject: [erlang-questions] NIF + HiPE - can it works together? In-Reply-To: <28150537.1296.1307625925806.JavaMail.geo-discussion-forums@vbyw6> References: <19951.37002.43330.882893@pilspetsen.it.uu.se> <28150537.1296.1307625925806.JavaMail.geo-discussion-forums@vbyw6> Message-ID: Is HiPE generally kept up with beam? Is there a set of automated tests you can point to to say "HiPE has 98% coverage of BEAM"? On Thu, Jun 9, 2011 at 8:25 AM, Joel Reymont wrote: > > > On Wednesday, June 8, 2011 7:08:58 PM UTC+4, Mikael Pettersson wrote: >> >> Alas there are effectively no compiler/vm developers left in the HiPE >> team; >> we've all left to pursue other career paths. >> > > Does this mean that HiPE is up for adoption? > > It wasn't very cuddly last time I looked at it but... > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From opensource@REDACTED Thu Jun 9 15:50:57 2011 From: opensource@REDACTED (Paul G Webster) Date: Thu, 09 Jun 2011 14:50:57 +0100 Subject: [erlang-questions] General help with unix sockets In-Reply-To: <20110609130343.GB3824@ecn.lan> References: <20110609130343.GB3824@ecn.lan> Message-ID: Thank you again michael, I emailed you from paul@ earlier Thank you for the help once again ;-) I managed to get my erlang application working well with your excellent procket module On Thu, 09 Jun 2011 14:03:43 +0100, Michael Santos wrote: > On Thu, Jun 09, 2011 at 12:19:42PM +0100, Paul G Webster wrote: >> Good day, >> >> I was wondering if anyone could lend a hand; I am trying to use >> erlang to connect to a unix socket '/var/pexim.sock' I spied out >> 'procket' it looks good but I am a little to new to erlang to figure >> out howto actually use the toolkit to connect.. >> >> Would anyone be able to give me a hand; it does not even need to be >> procket .. just a simple connection script for a unix socket >> >> If you could help it would be excellent >> >> p.s. If this actually ends up on the list twice; my apologies :) > > There are a few ways to connect to Unix sockets from Erlang. I responded > to your other email privately but for the archives my blog has some info > about it: > > http://listincomprehension.com/2010/12/unix-sockets.html > > Assuming that /var/pexim.sock is a stream socket (is that path right?) > and > you're using procket, the steps are similar to opening a socket in C. > The layout of a struct sockaddr_un will depend on your OS. For example, > it's defined in /usr/include/linux/un.h as: > > #define UNIX_PATH_MAX 108 > struct sockaddr_un { > sa_family_t sun_family; /* AF_UNIX */ > char sun_path[UNIX_PATH_MAX]; /* pathname */ > }; > > So the code to open the socket would look something like (untested): > > Path = <<"/var/pexim.sock">>, > PF_LOCAL = 1, > > % Get a Unix stream socket fd > {ok, Socket} = procket:socket(local, stream, 0), > Len = 108 - byte_size(Path), > > % Linux sockaddr_un > Sun = < Path/binary, > 0:(Len*8)>>, > > ok = procket:connect(Socket, Sun), > > {ok, Buf} = procket:read(Socket, 16#FFFF), > ok = procket:write(Socket, Buf), > > ok = procket:close(). > > I've left out the portability stuff for running on both BSD/Linux. > > Hope that helps! -- Using Opera's revolutionary email client: http://www.opera.com/mail/ From joelr1@REDACTED Thu Jun 9 15:51:20 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 9 Jun 2011 17:51:20 +0400 Subject: [erlang-questions] google groups mirroring Message-ID: <2446BABC-C5A7-48B8-95E8-5AD9537F3E0A@gmail.com> Is the mirroring of the Erlang Google Group two-way? Can I post to the Google Groups without subscribing to the list and have the post sent to list subscribers? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From skruger@REDACTED Thu Jun 9 19:14:01 2011 From: skruger@REDACTED (Shaun Kruger) Date: Thu, 09 Jun 2011 11:14:01 -0600 Subject: [erlang-questions] certificate chain trouble in R13B04 {ssl_imp, new} Message-ID: <1307639641.2092.22.camel@quadratic> I am having some trouble getting some SSL certs to work. I can get a self signed cert working, but I can't get a CA issued cert with a certificate chain file working. My certs were working with the old ssl implementation, but my need for IPv6 support is forcing me to commit to the new ssl implementation. The certificate I am currently testing with is a StartCom.org certificate. I am able to ssl:listen() and ssl:transport_accept() without any problems. The problem comes when I call ssl:ssl_accept(). I catch the exit and it returns with this: {'EXIT',{{{case_clause,{{file,"/home/skruger/erlang/workspace/Surrogate/conf/fastinfra.key"}, [{rsa_private_key,<<48,130,4,164,2,1,0,2,130,1,1,0,180,94,233, ....... 109,208,55,111>>, not_encrypted}]}}, [{ssl_certificate,find_issuer,2}, {ssl_certificate,certificate_chain,4}, {ssl_handshake,certificate,3}, {ssl_connection,certify_server,1}, {ssl_connection,server_certify_and_key_exchange,1}, {ssl_connection,do_server_hello,2}, {lists,foldl,3}, {ssl_connection,handle_event,3}]}, {gen_fsm,sync_send_all_state_event,[<0.293.0>,started,infinity]}}} The same certificate and key work with the old SSL implementation on IPv4. I'm hoping to find a solution to this as I would hate to have a dependency on ssl certs that don't have chain certs. Shaun =================================================== Here is some of the error report: =ERROR REPORT==== 9-Jun-2011::11:01:57 === ** State machine <0.293.0> terminating ** Last event in was {ssl_tls,undefined,22, {3,1}, <<1,0,0,155,3,1,77,240,252,133,189,215,225,174, 183,30,11,36,77,161,45,64,140,24,102,112,20, 97,137,55,217,18,225,113,59,187,209,208,0,0, 74,0,255,192,10,192,20,0,136,0,135,0,56,192, 15,192,5,0,132,0,53,0,57,192,7,192,9,192,17, 192,19,0,69,0,68,0,51,0,50,192,12,192,14,192, 2,192,4,0,150,0,65,0,4,0,5,0,47,192,8,192,18, 0,22,0,19,192,13,192,3,254,255,0,10,0,9,1,0,0, 40,0,0,0,14,0,12,0,0,9,108,111,99,97,108,104, 111,115,116,0,10,0,8,0,6,0,23,0,24,0,25,0,11, 0,2,1,0,0,35,0,0>>} (for all states) ** When State == hello ** Data == {state,server, {#Ref<0.0.0.796>,<0.292.0>}, gen_tcp,tcp,tcp_closed,"localhost",8089,#Port<0.3642>, {ssl_options,[],verify_none,#Fun,false, false,1, "/home/skruger/erlang/workspace/Surrogate/conf/fastinfra.crt", "/home/skruger/erlang/workspace/Surrogate/conf/fastinfra.key", undefined,undefined,[], [<<0,10>>,<<0,47>>,<<0,5>>,<<0,4>>,<<0,9>>], #Fun,true,[]}, {socket_options,binary,0,0,0,false}, ..... From opensource@REDACTED Thu Jun 9 20:23:07 2011 From: opensource@REDACTED (Paul G Webster) Date: Thu, 09 Jun 2011 18:23:07 -0000 Subject: [erlang-questions] General help with unix sockets In-Reply-To: <20110609130343.GB3824@ecn.lan> References: <20110609130343.GB3824@ecn.lan> Message-ID: Out of interest, my exim socket is working wonderfully but is it normal for it to immediately disconnect: server app: > ./ss Accepted connection! at ./ss line 51. GOT: test at ./ss line 86. Server session encountered read error 0: Normal disconnection. erlang app: > erl -pa procket/ebin/ Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) 1> c(unix_stream). {ok,unix_stream} 2> unix_stream:client(<<"test\n">>). <<"tset\r\n">> telnet -u: > telnet -u /tmp/testsocket Trying /tmp/testsocket... Connected to /tmp/testsocket. Escape character is '^]'. test tset test2 2tset My would preffer the socket did not exit immediately after it got a result I am hoping that I can run a TCP server in exim that will accept multiple connections and kind of relay to the unix socket selective data from the multiple TCP connections. But obviously I will not be able to do that if ever request is automatically terminated :) well.. I bet I could but it would spam the unix sock On Thu, 09 Jun 2011 13:03:43 -0000, Michael Santos wrote: > On Thu, Jun 09, 2011 at 12:19:42PM +0100, Paul G Webster wrote: >> Good day, >> >> I was wondering if anyone could lend a hand; I am trying to use >> erlang to connect to a unix socket '/var/pexim.sock' I spied out >> 'procket' it looks good but I am a little to new to erlang to figure >> out howto actually use the toolkit to connect.. >> >> Would anyone be able to give me a hand; it does not even need to be >> procket .. just a simple connection script for a unix socket >> >> If you could help it would be excellent >> >> p.s. If this actually ends up on the list twice; my apologies :) > > There are a few ways to connect to Unix sockets from Erlang. I responded > to your other email privately but for the archives my blog has some info > about it: > > http://listincomprehension.com/2010/12/unix-sockets.html > > Assuming that /var/pexim.sock is a stream socket (is that path right?) > and > you're using procket, the steps are similar to opening a socket in C. > The layout of a struct sockaddr_un will depend on your OS. For example, > it's defined in /usr/include/linux/un.h as: > > #define UNIX_PATH_MAX 108 > struct sockaddr_un { > sa_family_t sun_family; /* AF_UNIX */ > char sun_path[UNIX_PATH_MAX]; /* pathname */ > }; > > So the code to open the socket would look something like (untested): > > Path = <<"/var/pexim.sock">>, > PF_LOCAL = 1, > > % Get a Unix stream socket fd > {ok, Socket} = procket:socket(local, stream, 0), > Len = 108 - byte_size(Path), > > % Linux sockaddr_un > Sun = < Path/binary, > 0:(Len*8)>>, > > ok = procket:connect(Socket, Sun), > > {ok, Buf} = procket:read(Socket, 16#FFFF), > ok = procket:write(Socket, Buf), > > ok = procket:close(). > > I've left out the portability stuff for running on both BSD/Linux. > > Hope that helps! -- Using Opera's revolutionary email client: http://www.opera.com/mail/ From igorrs@REDACTED Thu Jun 9 20:55:47 2011 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Thu, 9 Jun 2011 15:55:47 -0300 Subject: [erlang-questions] [erlang-bugs] All keys in the same slot (Mnesia/dets) In-Reply-To: <19952.46482.187947.878787@ornendil.otp.ericsson.se> References: <19952.46482.187947.878787@ornendil.otp.ericsson.se> Message-ID: Hello. Thank you very much (I mean it) for teaching me I can use min_no_slots together with repair. I always thought I could only use min_no_slots "when the table is created", as the docs say. Most mortals won't think that repair is such a case. :-) However, I also had to specify other non-default properties of the table ({type, bag}, for example). Otherwise, it would fail with type_mismatch. As for the other issues (I'm taking the liberty of copying Mr. Gudmundsson, who knows a lot about the internals of Mnesia)... These dets files with all keys in the same slot are not corrupt. They are just a consequence of the awful combination of table fragmentation with disc_only_copies (mnesia_frag_hash with dets, to be more precise). If a table is split in N fragments, at most 1/N of the slots in each dets fragment will be used. I have verified that this is the case for all disc_only_copies tables I have deployed. The only exception is the table that does not use mnesia_frag_hash (another module was specified as hash_module at table creation). What happened to my "uc" table (which is a bag) during the upgrade from R13B04 to R14B02 seems to have been unintended behavior from Mnesia, which ended up benefiting me. Each of the 1024 fragments in the uc table has about 300,000 objects, but only about 1,500 keys. Before the upgrade, I think (not sure) each fragment had around 1,500 slots, with all keys in the same slot (as expected). During the upgrade, for some reason, the fragments in most servers (and in their replicas) seem to have been recreated with around 500,000 slots each. This fact caused a performance improvement, since 1/1024 of the 500k slots in each fragment are being used now. I believe it was unintended, because (I think) Mnesia used the number of objects (instead of the the number of keys) as estimated_no_objects when it recreated the dets tables. I don't suppose it has bags in mind for this step. If I now pick one of the fragments that were left with around 1,500 slots and perform mnesia:change_table_copy_type to disc_copies and then back to disc_only_copies, the number of slots is increased to around 500k (Mnesia's source code gave me this tip). But I still don't understand how it happened during the restarts/repairs for the upgrade. Calling del_table_copy and add_table_copy does not increase the number of slots in the dets table (I think it's just copied from the replica "as is", the same way it happens during the restarts with all nodes in R14B02). Thanks! Igor. On Thu, Jun 9, 2011 at 8:59 AM, Hans Bolinder wrote: > Hi, > > [Igor Ribeiro Sucupira:] >> The weirdest difference being that all records in uc_frag1004 are in >> the same slot! >> >> 9> length([S || S <- lists:seq(0, element(2, dets:info(F1004, >> ? ?no_slots)) - 1), length(dets:slot(F1004, S)) > 0]). >> 1 > > If you still have the file available: How many objects are there in > the slot? How many objects does a traversal of the file return? Try > > ?length(dets:match_object(F1004,'_')). > > If the numbers differ then the file is corrupt. > >> Is there a way to manually raise the number of slots in a dets table? > > You can try closing the file, and then force a repair: > > ?dets:open_file(T, [{min_no_slots,N},{repair,force}]). > > where N is some suitable integer. > > Best regards, > > Hans Bolinder, Erlang/OTP team, Ericsson > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs From mikpe@REDACTED Thu Jun 9 21:07:58 2011 From: mikpe@REDACTED (Mikael Pettersson) Date: Thu, 9 Jun 2011 21:07:58 +0200 Subject: [erlang-questions] NIF + HiPE - can it works together? In-Reply-To: <28150537.1296.1307625925806.JavaMail.geo-discussion-forums@vbyw6> References: <19951.37002.43330.882893@pilspetsen.it.uu.se> <28150537.1296.1307625925806.JavaMail.geo-discussion-forums@vbyw6> Message-ID: <19953.6670.248502.684310@pilspetsen.it.uu.se> Joel Reymont writes: > > > On Wednesday, June 8, 2011 7:08:58 PM UTC+4, Mikael Pettersson wrote: > > > > Alas there are effectively no compiler/vm developers left in the HiPE team; > > we've all left to pursue other career paths. > > > > Does this mean that HiPE is up for adoption? Not quite. Kostis still has a claim on both the name and the code, and he usually oversees a couple of (undergraduate) student projects each year that do some experiments in the compiler parts. But if other people submit high-quality patches that fix bugs or add useful improvements then I'm sure those patches will eventually find their way into OTP. From mikpe@REDACTED Thu Jun 9 21:12:31 2011 From: mikpe@REDACTED (Mikael Pettersson) Date: Thu, 9 Jun 2011 21:12:31 +0200 Subject: [erlang-questions] NIF + HiPE - can it works together? In-Reply-To: References: <19951.37002.43330.882893@pilspetsen.it.uu.se> <28150537.1296.1307625925806.JavaMail.geo-discussion-forums@vbyw6> Message-ID: <19953.6943.684363.766656@pilspetsen.it.uu.se> Kenny Stone writes: > Is HiPE generally kept up with beam? Yes. I'm only aware of three omissions: - apply with {M,F} tuple; that's a legacy feature we don't like - on_load, because of unresolved implementation difficulties - stack dumps in exceptions look different; that's a WONTFIX > Is there a set of automated tests you > can point to to say "HiPE has 98% coverage of BEAM"? No. From sunwood360@REDACTED Thu Jun 9 22:24:02 2011 From: sunwood360@REDACTED (envelopes envelopes) Date: Thu, 9 Jun 2011 13:24:02 -0700 Subject: [erlang-questions] Erlang library supports google protocol buffer Message-ID: Is there any erlang library supports google protocol buffer? http://code.google.com/p/protobuf/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jared@REDACTED Thu Jun 9 22:35:55 2011 From: jared@REDACTED (Jared Morrow) Date: Thu, 9 Jun 2011 14:35:55 -0600 Subject: [erlang-questions] Erlang library supports google protocol buffer In-Reply-To: References: Message-ID: https://github.com/ngerakines/erlang_protobuffs On Thu, Jun 9, 2011 at 2:24 PM, envelopes envelopes wrote: > Is there any erlang library supports google protocol buffer? > > http://code.google.com/p/protobuf/ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew@REDACTED Fri Jun 10 04:07:30 2011 From: andrew@REDACTED (Andrew Thompson) Date: Thu, 9 Jun 2011 22:07:30 -0400 Subject: [erlang-questions] Erlang library supports google protocol buffer In-Reply-To: References: Message-ID: <20110610020730.GA1658@hijacked.us> On Thu, Jun 09, 2011 at 02:35:55PM -0600, Jared Morrow wrote: > https://github.com/ngerakines/erlang_protobuffs > Actually, the one we use at basho is a fork of that with some more updates: https://github.com/freke/erlang_protobuffs Andrew From z_axis@REDACTED Fri Jun 10 05:16:36 2011 From: z_axis@REDACTED (z_axis) Date: Thu, 9 Jun 2011 20:16:36 -0700 (PDT) Subject: [erlang-questions] Why cannot I visit YAWS remotely ? In-Reply-To: <1307665381459-3587117.post@n4.nabble.com> References: <1307665381459-3587117.post@n4.nabble.com> Message-ID: <1307675796881-3587417.post@n4.nabble.com> I have asked it on YAWS maillist. but get no answer :( ----- e^(?.i) + 1 = 0 -- View this message in context: http://erlang.2086793.n4.nabble.com/Why-cannot-I-visit-YAWS-remotely-tp3587117p3587417.html Sent from the Erlang Questions mailing list archive at Nabble.com. From thatpythonguy@REDACTED Fri Jun 10 05:18:12 2011 From: thatpythonguy@REDACTED (Ahmed Al-Saadi) Date: Thu, 09 Jun 2011 23:18:12 -0400 Subject: [erlang-questions] google groups mirroring In-Reply-To: <2446BABC-C5A7-48B8-95E8-5AD9537F3E0A@gmail.com> References: <2446BABC-C5A7-48B8-95E8-5AD9537F3E0A@gmail.com> Message-ID: <4DF18CF4.6060908@gmail.com> I don't think so. I tried it before and it didn't work. -signed(ahmed). On 6/9/11 9:51 AM, Joel Reymont wrote: > Is the mirroring of the Erlang Google Group two-way? > > Can I post to the Google Groups without subscribing to the list and have the post sent to list subscribers? > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From sunwood360@REDACTED Fri Jun 10 08:24:49 2011 From: sunwood360@REDACTED (envelopes envelopes) Date: Thu, 9 Jun 2011 23:24:49 -0700 Subject: [erlang-questions] Is there a web interface to erlang shell? Message-ID: So I can connect to the remote shell via http, because the erlang vm is running @ a remote box that I can only access by HTTP. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From icarus.alive@REDACTED Fri Jun 10 08:40:11 2011 From: icarus.alive@REDACTED (Icarus Alive) Date: Fri, 10 Jun 2011 12:10:11 +0530 Subject: [erlang-questions] Is there a web interface to erlang shell? In-Reply-To: References: Message-ID: Search GOOG or your favourite SE for something called "Ajax Terminal" (or hint: "ajaxterm") download, install and you are good to go. On Fri, Jun 10, 2011 at 11:54 AM, envelopes envelopes wrote: > So I can connect to the remote shell via http,? because the erlang vm is > running @ a remote box that I can only access by HTTP. > > Thanks > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From ingela@REDACTED Fri Jun 10 10:00:41 2011 From: ingela@REDACTED (Ingela Andin) Date: Fri, 10 Jun 2011 10:00:41 +0200 Subject: [erlang-questions] certificate chain trouble in R13B04 {ssl_imp, new} In-Reply-To: <1307639641.2092.22.camel@quadratic> References: <1307639641.2092.22.camel@quadratic> Message-ID: Hi! This is a very old bug and I can not remember in which version it was fixed. In R13 new ssl was still exprimental so I would very much suggest that you upgrade to R14 if you want to run new ssl. Regards Ingela Erlang/OTP team - Ericsson AB 2011/6/9 Shaun Kruger : > I am having some trouble getting some SSL certs to work. ?I can get a > self signed cert working, but I can't get a CA issued cert with a > certificate chain file working. ?My certs were working with the old ssl > implementation, but my need for IPv6 support is forcing me to commit to > the new ssl implementation. > > The certificate I am currently testing with is a StartCom.org > certificate. ?I am able to ssl:listen() and ssl:transport_accept() > without any problems. ?The problem comes when I call ssl:ssl_accept(). > I catch the exit and it returns with this: > > {'EXIT',{{{case_clause,{{file,"/home/skruger/erlang/workspace/Surrogate/conf/fastinfra.key"}, > [{rsa_private_key,<<48,130,4,164,2,1,0,2,130,1,1,0,180,94,233, > ....... > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 109,208,55,111>>, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?not_encrypted}]}}, > ?[{ssl_certificate,find_issuer,2}, > ? {ssl_certificate,certificate_chain,4}, > ? {ssl_handshake,certificate,3}, > ? {ssl_connection,certify_server,1}, > ? {ssl_connection,server_certify_and_key_exchange,1}, > ? {ssl_connection,do_server_hello,2}, > ? {lists,foldl,3}, > ? {ssl_connection,handle_event,3}]}, > ?{gen_fsm,sync_send_all_state_event,[<0.293.0>,started,infinity]}}} > > The same certificate and key work with the old SSL implementation on > IPv4. > > I'm hoping to find a solution to this as I would hate to have a > dependency on ssl certs that don't have chain certs. > > Shaun > > > =================================================== > Here is some of the error report: > > =ERROR REPORT==== 9-Jun-2011::11:01:57 === > ** State machine <0.293.0> terminating > ** Last event in was {ssl_tls,undefined,22, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{3,1}, > <<1,0,0,155,3,1,77,240,252,133,189,215,225,174, > 183,30,11,36,77,161,45,64,140,24,102,112,20, > 97,137,55,217,18,225,113,59,187,209,208,0,0, > 74,0,255,192,10,192,20,0,136,0,135,0,56,192, > 15,192,5,0,132,0,53,0,57,192,7,192,9,192,17, > 192,19,0,69,0,68,0,51,0,50,192,12,192,14,192, > 2,192,4,0,150,0,65,0,4,0,5,0,47,192,8,192,18, > 0,22,0,19,192,13,192,3,254,255,0,10,0,9,1,0,0, > 40,0,0,0,14,0,12,0,0,9,108,111,99,97,108,104, > 111,115,116,0,10,0,8,0,6,0,23,0,24,0,25,0,11, > 0,2,1,0,0,35,0,0>>} (for all states) > ** When State == hello > ** ? ? ?Data ?== {state,server, > ? ? ? ? ? ? ? ? ? ? {#Ref<0.0.0.796>,<0.292.0>}, > gen_tcp,tcp,tcp_closed,"localhost",8089,#Port<0.3642>, > {ssl_options,[],verify_none,#Fun,false, > false,1, > "/home/skruger/erlang/workspace/Surrogate/conf/fastinfra.crt", > "/home/skruger/erlang/workspace/Surrogate/conf/fastinfra.key", > undefined,undefined,[], > [<<0,10>>,<<0,47>>,<<0,5>>,<<0,4>>,<<0,9>>], > #Fun,true,[]}, > {socket_options,binary,0,0,0,false}, > ..... > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From vinoski@REDACTED Fri Jun 10 11:13:54 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Fri, 10 Jun 2011 05:13:54 -0400 Subject: [erlang-questions] Why cannot I visit YAWS remotely ? In-Reply-To: <1307675796881-3587417.post@n4.nabble.com> References: <1307665381459-3587117.post@n4.nabble.com> <1307675796881-3587417.post@n4.nabble.com> Message-ID: Actually I've answered both of your messages on the yaws mailing list, but you haven't answered my replies. You need to run yaws -i and send the output to the yaws mailing list. Sending your conf file would be helpful as well. --steve On Thu, Jun 9, 2011 at 11:16 PM, z_axis wrote: > I have asked it on YAWS maillist. but get no answer :( > > ----- > e^(?.i) + 1 = 0 > -- > View this message in context: http://erlang.2086793.n4.nabble.com/Why-cannot-I-visit-YAWS-remotely-tp3587117p3587417.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From francis.joanis@REDACTED Fri Jun 10 15:44:28 2011 From: francis.joanis@REDACTED (Francis Joanis) Date: Fri, 10 Jun 2011 09:44:28 -0400 Subject: [erlang-questions] Running a release as a Windows service with erlsrv.exe and start_erl.exe In-Reply-To: References: Message-ID: Hi, Do you know if this issue only affects Windows Server 2008 or does it also happen on Windows Server 2003? Thanks, Francis On Thu, Jun 9, 2011 at 6:55 AM, Ben Godfrey wrote: > Thanks Siri, > > Is there anything I can do to help? > > Ben > > On 7 June 2011 09:11, Siri Hansen wrote: > > Hi Ben! > > Unfortunately, the release handling for windows, including erlsrv.erl and > > start_erl.c, is broken. This was discovered not long ago, after > refreshing > > the release_handler test suite where these tests were skipped on windows. > We > > are currently working on this issue, and plan to release a correction as > > soon as possible. > > Regards > > siri@REDACTED > > > > 2011/6/6 Ben Godfrey > >> > >> Hello all, > >> > >> I've been trying to deploy an Erlang application as a Windows service > >> and not having much luck. > >> > >> I built a release package using rebar. I'm registering the service as > >> follows: > >> > >> c:\erl5.8.3\erts-5.8.3\bin\erlsrv.exe add Rolf -c "Collects system > >> data for monitoring." -w c:\rolf -m > >> c:\erl5.8.3\erts-5.8.3\bin\start_erl.exe -debugtype reuse -args > >> "-setcookie xyz123 ++ -reldir c:\rolf\releases" > >> > >> This fails with the error "Failed to query BinDir of release." in the > >> .debug file. Looking at the code for start_erl.exe, this is related to > >> registry keys. I can't see any sign of any keys which match the > >> pattern described in the source, but curiously the preceding call to > >> open a registry key succeeds [2]. > >> > >> Any one have any idea about any of this stuff? > >> > >> Server is Windows Server 2008. > >> > >> Ben > >> > >> [1] > https://github.com/erlang/otp/blob/dev/erts/etc/win32/start_erl.c#L364 > >> [2] > https://github.com/erlang/otp/blob/dev/erts/etc/win32/start_erl.c#L342 > >> > >> -- > >> Ben Godfrey, Wandering Hacker > >> http://aftnn.org | http://twitter.com/afternoon | > >> http://www.linkedin.com/in/bengodfrey > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > Ben Godfrey, Wandering Hacker > http://aftnn.org | http://twitter.com/afternoon | > http://www.linkedin.com/in/bengodfrey > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mevans@REDACTED Fri Jun 10 16:15:28 2011 From: mevans@REDACTED (Evans, Matthew) Date: Fri, 10 Jun 2011 10:15:28 -0400 Subject: [erlang-questions] Is there a web interface to erlang shell? In-Reply-To: References: Message-ID: There is this: http://www.erlang.org/doc/man/webtool.html You can add your own tools to it. There is also this utility: http://code.google.com/p/erlwsh/ (you will need mochiweb too, I'm sure it could be extended to support yaws too). I played with it a while ago, and it does need some work (some command responses are sent to the Erlang shell and not your webpage). I'm sure it's hackable though ;) Matt ________________________________ From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of envelopes envelopes Sent: Friday, June 10, 2011 2:25 AM To: erlang-questions@REDACTED Subject: [erlang-questions] Is there a web interface to erlang shell? So I can connect to the remote shell via http, because the erlang vm is running @ a remote box that I can only access by HTTP. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From bpgodfrey@REDACTED Fri Jun 10 16:31:45 2011 From: bpgodfrey@REDACTED (Ben Godfrey) Date: Fri, 10 Jun 2011 15:31:45 +0100 Subject: [erlang-questions] Running a release as a Windows service with erlsrv.exe and start_erl.exe In-Reply-To: References: Message-ID: <-2309599951819072559@unknownmsgid> I have only tested on 2008. I could build a 2003 box and test if that's helpful. Ben On 10 Jun 2011, at 14:44, Francis Joanis wrote: Hi, Do you know if this issue only affects Windows Server 2008 or does it also happen on Windows Server 2003? Thanks, Francis On Thu, Jun 9, 2011 at 6:55 AM, Ben Godfrey wrote: > Thanks Siri, > > Is there anything I can do to help? > > Ben > > On 7 June 2011 09:11, Siri Hansen wrote: > > Hi Ben! > > Unfortunately, the release handling for windows, including erlsrv.erl and > > start_erl.c, is broken. This was discovered not long ago, after > refreshing > > the release_handler test suite where these tests were skipped on windows. > We > > are currently working on this issue, and plan to release a correction as > > soon as possible. > > Regards > > siri@REDACTED > > > > 2011/6/6 Ben Godfrey > >> > >> Hello all, > >> > >> I've been trying to deploy an Erlang application as a Windows service > >> and not having much luck. > >> > >> I built a release package using rebar. I'm registering the service as > >> follows: > >> > >> c:\erl5.8.3\erts-5.8.3\bin\erlsrv.exe add Rolf -c "Collects system > >> data for monitoring." -w c:\rolf -m > >> c:\erl5.8.3\erts-5.8.3\bin\start_erl.exe -debugtype reuse -args > >> "-setcookie xyz123 ++ -reldir c:\rolf\releases" > >> > >> This fails with the error "Failed to query BinDir of release." in the > >> .debug file. Looking at the code for start_erl.exe, this is related to > >> registry keys. I can't see any sign of any keys which match the > >> pattern described in the source, but curiously the preceding call to > >> open a registry key succeeds [2]. > >> > >> Any one have any idea about any of this stuff? > >> > >> Server is Windows Server 2008. > >> > >> Ben > >> > >> [1] > https://github.com/erlang/otp/blob/dev/erts/etc/win32/start_erl.c#L364 > >> [2] > https://github.com/erlang/otp/blob/dev/erts/etc/win32/start_erl.c#L342 > >> > >> -- > >> Ben Godfrey, Wandering Hacker > >> http://aftnn.org | http://twitter.com/afternoon | > >> http://www.linkedin.com/in/bengodfrey > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > Ben Godfrey, Wandering Hacker > http://aftnn.org | http://twitter.com/afternoon | > http://www.linkedin.com/in/bengodfrey > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From skruger@REDACTED Fri Jun 10 17:17:35 2011 From: skruger@REDACTED (Shaun Kruger) Date: Fri, 10 Jun 2011 09:17:35 -0600 Subject: [erlang-questions] certificate chain trouble in R13B04 {ssl_imp, new} In-Reply-To: References: <1307639641.2092.22.camel@quadratic> Message-ID: <1307719055.2092.24.camel@quadratic> I can live with that. I suppose that's why old ssl was the default in R13. I'll test it with the latest R14 and merely state that R14 is a dependency for doing IPv6 SSL termination. Thanks, Shaun On Fri, 2011-06-10 at 10:00 +0200, Ingela Andin wrote: > Hi! > > This is a very old bug and I can not remember in which version it was > fixed. In R13 new ssl was > still exprimental so I would very much suggest that you upgrade to R14 > if you want to run new ssl. > > Regards Ingela Erlang/OTP team - Ericsson AB > > 2011/6/9 Shaun Kruger : > > I am having some trouble getting some SSL certs to work. I can get a > > self signed cert working, but I can't get a CA issued cert with a > > certificate chain file working. My certs were working with the old ssl > > implementation, but my need for IPv6 support is forcing me to commit to > > the new ssl implementation. > > > > The certificate I am currently testing with is a StartCom.org > > certificate. I am able to ssl:listen() and ssl:transport_accept() > > without any problems. The problem comes when I call ssl:ssl_accept(). > > I catch the exit and it returns with this: > > > > {'EXIT',{{{case_clause,{{file,"/home/skruger/erlang/workspace/Surrogate/conf/fastinfra.key"}, > > [{rsa_private_key,<<48,130,4,164,2,1,0,2,130,1,1,0,180,94,233, > > ....... > > 109,208,55,111>>, > > not_encrypted}]}}, > > [{ssl_certificate,find_issuer,2}, > > {ssl_certificate,certificate_chain,4}, > > {ssl_handshake,certificate,3}, > > {ssl_connection,certify_server,1}, > > {ssl_connection,server_certify_and_key_exchange,1}, > > {ssl_connection,do_server_hello,2}, > > {lists,foldl,3}, > > {ssl_connection,handle_event,3}]}, > > {gen_fsm,sync_send_all_state_event,[<0.293.0>,started,infinity]}}} > > > > The same certificate and key work with the old SSL implementation on > > IPv4. > > > > I'm hoping to find a solution to this as I would hate to have a > > dependency on ssl certs that don't have chain certs. > > > > Shaun > > > > > > =================================================== > > Here is some of the error report: > > > > =ERROR REPORT==== 9-Jun-2011::11:01:57 === > > ** State machine <0.293.0> terminating > > ** Last event in was {ssl_tls,undefined,22, > > {3,1}, > > <<1,0,0,155,3,1,77,240,252,133,189,215,225,174, > > 183,30,11,36,77,161,45,64,140,24,102,112,20, > > 97,137,55,217,18,225,113,59,187,209,208,0,0, > > 74,0,255,192,10,192,20,0,136,0,135,0,56,192, > > 15,192,5,0,132,0,53,0,57,192,7,192,9,192,17, > > 192,19,0,69,0,68,0,51,0,50,192,12,192,14,192, > > 2,192,4,0,150,0,65,0,4,0,5,0,47,192,8,192,18, > > 0,22,0,19,192,13,192,3,254,255,0,10,0,9,1,0,0, > > 40,0,0,0,14,0,12,0,0,9,108,111,99,97,108,104, > > 111,115,116,0,10,0,8,0,6,0,23,0,24,0,25,0,11, > > 0,2,1,0,0,35,0,0>>} (for all states) > > ** When State == hello > > ** Data == {state,server, > > {#Ref<0.0.0.796>,<0.292.0>}, > > gen_tcp,tcp,tcp_closed,"localhost",8089,#Port<0.3642>, > > {ssl_options,[],verify_none,#Fun,false, > > false,1, > > "/home/skruger/erlang/workspace/Surrogate/conf/fastinfra.crt", > > "/home/skruger/erlang/workspace/Surrogate/conf/fastinfra.key", > > undefined,undefined,[], > > [<<0,10>>,<<0,47>>,<<0,5>>,<<0,4>>,<<0,9>>], > > #Fun,true,[]}, > > {socket_options,binary,0,0,0,false}, > > ..... > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > From robert.virding@REDACTED Fri Jun 10 18:26:57 2011 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 10 Jun 2011 16:26:57 +0000 (GMT) Subject: [erlang-questions] NIF + HiPE - can it works together? In-Reply-To: <19953.6943.684363.766656@pilspetsen.it.uu.se> Message-ID: <1828799266.285291307723217919.JavaMail.root@zimbra> ----- "Mikael Pettersson" wrote: > Kenny Stone writes: > > Is HiPE generally kept up with beam? > > Yes. I'm only aware of three omissions: > - apply with {M,F} tuple; that's a legacy feature we don't like Please don't support this feature, please! It should have dies an ignominious death long ago. > - on_load, because of unresolved implementation difficulties > - stack dumps in exceptions look different; that's a WONTFIX > > > Is there a set of automated tests you > > can point to to say "HiPE has 98% coverage of BEAM"? > > No. Robert From skruger@REDACTED Fri Jun 10 18:44:38 2011 From: skruger@REDACTED (Shaun Kruger) Date: Fri, 10 Jun 2011 10:44:38 -0600 Subject: [erlang-questions] certificate chain trouble in R13B04 {ssl_imp, new} In-Reply-To: References: <1307639641.2092.22.camel@quadratic> Message-ID: <1307724278.2092.48.camel@quadratic> Thanks again. I just finished verifying functionality with R14B03 and IPv6 SSL termination is working just fine. Shaun On Fri, 2011-06-10 at 10:00 +0200, Ingela Andin wrote: > Hi! > > This is a very old bug and I can not remember in which version it was > fixed. In R13 new ssl was > still exprimental so I would very much suggest that you upgrade to R14 > if you want to run new ssl. > > Regards Ingela Erlang/OTP team - Ericsson AB > > 2011/6/9 Shaun Kruger : > > I am having some trouble getting some SSL certs to work. I can get a > > self signed cert working, but I can't get a CA issued cert with a > > certificate chain file working. My certs were working with the old ssl > > implementation, but my need for IPv6 support is forcing me to commit to > > the new ssl implementation. > > > > The certificate I am currently testing with is a StartCom.org > > certificate. I am able to ssl:listen() and ssl:transport_accept() > > without any problems. The problem comes when I call ssl:ssl_accept(). > > I catch the exit and it returns with this: > > > > {'EXIT',{{{case_clause,{{file,"/home/skruger/erlang/workspace/Surrogate/conf/fastinfra.key"}, > > [{rsa_private_key,<<48,130,4,164,2,1,0,2,130,1,1,0,180,94,233, > > ....... > > 109,208,55,111>>, > > not_encrypted}]}}, > > [{ssl_certificate,find_issuer,2}, > > {ssl_certificate,certificate_chain,4}, > > {ssl_handshake,certificate,3}, > > {ssl_connection,certify_server,1}, > > {ssl_connection,server_certify_and_key_exchange,1}, > > {ssl_connection,do_server_hello,2}, > > {lists,foldl,3}, > > {ssl_connection,handle_event,3}]}, > > {gen_fsm,sync_send_all_state_event,[<0.293.0>,started,infinity]}}} > > > > The same certificate and key work with the old SSL implementation on > > IPv4. > > > > I'm hoping to find a solution to this as I would hate to have a > > dependency on ssl certs that don't have chain certs. > > > > Shaun > > > > > > =================================================== > > Here is some of the error report: > > > > =ERROR REPORT==== 9-Jun-2011::11:01:57 === > > ** State machine <0.293.0> terminating > > ** Last event in was {ssl_tls,undefined,22, > > {3,1}, > > <<1,0,0,155,3,1,77,240,252,133,189,215,225,174, > > 183,30,11,36,77,161,45,64,140,24,102,112,20, > > 97,137,55,217,18,225,113,59,187,209,208,0,0, > > 74,0,255,192,10,192,20,0,136,0,135,0,56,192, > > 15,192,5,0,132,0,53,0,57,192,7,192,9,192,17, > > 192,19,0,69,0,68,0,51,0,50,192,12,192,14,192, > > 2,192,4,0,150,0,65,0,4,0,5,0,47,192,8,192,18, > > 0,22,0,19,192,13,192,3,254,255,0,10,0,9,1,0,0, > > 40,0,0,0,14,0,12,0,0,9,108,111,99,97,108,104, > > 111,115,116,0,10,0,8,0,6,0,23,0,24,0,25,0,11, > > 0,2,1,0,0,35,0,0>>} (for all states) > > ** When State == hello > > ** Data == {state,server, > > {#Ref<0.0.0.796>,<0.292.0>}, > > gen_tcp,tcp,tcp_closed,"localhost",8089,#Port<0.3642>, > > {ssl_options,[],verify_none,#Fun,false, > > false,1, > > "/home/skruger/erlang/workspace/Surrogate/conf/fastinfra.crt", > > "/home/skruger/erlang/workspace/Surrogate/conf/fastinfra.key", > > undefined,undefined,[], > > [<<0,10>>,<<0,47>>,<<0,5>>,<<0,4>>,<<0,9>>], > > #Fun,true,[]}, > > {socket_options,binary,0,0,0,false}, > > ..... > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > From toddbittner@REDACTED Fri Jun 10 21:24:01 2011 From: toddbittner@REDACTED (Todd Bittner) Date: Fri, 10 Jun 2011 14:24:01 -0500 Subject: [erlang-questions] inets, httpc, streaming, and non-2xx status codes Message-ID: I'm trying to stream a response from a server via the inets client, using a call like this: httpc:request(,[{headers_as_is, true},{sync,false},{stream,self}])), 200 responses seem to follow the proper work flow, but, when I receive a non-2xx response I either get messages like this: (1) {http, {RequestId, {status_line(), headers(), body()}}, which seems to be expected as per this previous mailing list conversation - http://erlang.org/pipermail/erlang-bugs/2009-March/001222.html or I'll get a message that just drops right into a stream atom clause: (2) {http, {RequestId,stream,BinBodyPart}} -> which is unusual, because I thought only (1) should be returned for non-2xx responses. As a result, I have to first check to see if I receive a stream_start atom or a stream atom first to differentiate between 2xx and non-2xx responses. This is always followed up with a {http, {RequestId, {status_line(), headers(), <<>>}} I haven't been able to figure out what type of response causes to me get a type (1) message and what causes me to get a type (2). In addition, sometimes (1) doesn't function as expected either - I'll receive a body that doesn't match the content length, and even if I try and drop into a receive to capture the rest of the body, I never get any messages. Am I missing an option when calling the client? I'm running 14A, for what it's worth. Thanks, Todd -------------- next part -------------- An HTML attachment was scrubbed... URL: From jameschurchman@REDACTED Fri Jun 10 21:38:54 2011 From: jameschurchman@REDACTED (James Churchman) Date: Fri, 10 Jun 2011 20:38:54 +0100 Subject: [erlang-questions] Specs In-Reply-To: <4DEE8CCE.9070202@cs.ntua.gr> References: <4DEE8CCE.9070202@cs.ntua.gr> Message-ID: Hi Kostis, i must have had an error in my specs then maybe, as otherwise they i thought looked similar to yours, but were prefixed with the variable name and "::" Thanks very much for your help James On 7 June 2011 21:40, Kostis Sagonas wrote: > James Churchman wrote: > >> Hi guys >> >> When you parse a file with the epp module it has no problems with specs >> >> However when using erl_scan then passing on to erl_parse, in order to >> parse a string not a file, it fails on -specs in the code >> >> Is there a spec aware version of the erl_parse module ? >> > > I find this question a bit strange... the erl_parse module *is* spec aware. > This is actually how specs are parsed in the system. > > Does the following do what you want? > > 42> SpecInString = "-spec my_function(some_type()) -> 42 | 'gazonk'.", > 42> {ok, Toks, 1} = erl_scan:string(SpecInString), > 42> erl_parse:parse_form(Toks). > {ok,{attribute,1,spec, > {{my_function,1}, > [{type,1,'fun', > [{type,1,product,[{type,1,some_type,[]}]}, > > {type,1,union,[{integer,1,42},{atom,1,gazonk}]}]}]}}} > > Kostis > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jameschurchman@REDACTED Fri Jun 10 21:59:47 2011 From: jameschurchman@REDACTED (James Churchman) Date: Fri, 10 Jun 2011 20:59:47 +0100 Subject: [erlang-questions] Specs In-Reply-To: References: <4DEE8CCE.9070202@cs.ntua.gr> Message-ID: ok i FINALLY worked it out thanks, the spec i was passing in did not end in a full stop :-) On 10 June 2011 20:38, James Churchman wrote: > Hi Kostis, i must have had an error in my specs then maybe, as otherwise > they i thought looked similar to yours, but were prefixed with the variable > name and "::" > > Thanks very much for your help > > James > > On 7 June 2011 21:40, Kostis Sagonas wrote: > >> James Churchman wrote: >> >>> Hi guys >>> >>> When you parse a file with the epp module it has no problems with specs >>> >>> However when using erl_scan then passing on to erl_parse, in order to >>> parse a string not a file, it fails on -specs in the code >>> >>> Is there a spec aware version of the erl_parse module ? >>> >> >> I find this question a bit strange... the erl_parse module *is* spec >> aware. This is actually how specs are parsed in the system. >> >> Does the following do what you want? >> >> 42> SpecInString = "-spec my_function(some_type()) -> 42 | 'gazonk'.", >> 42> {ok, Toks, 1} = erl_scan:string(SpecInString), >> 42> erl_parse:parse_form(Toks). >> {ok,{attribute,1,spec, >> {{my_function,1}, >> [{type,1,'fun', >> [{type,1,product,[{type,1,some_type,[]}]}, >> >> {type,1,union,[{integer,1,42},{atom,1,gazonk}]}]}]}}} >> >> Kostis >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dale@REDACTED Sun Jun 12 02:25:29 2011 From: dale@REDACTED (Dale Harvey) Date: Sun, 12 Jun 2011 01:25:29 +0100 Subject: [erlang-questions] Documentation Format Changes (erldocs) Message-ID: Hey all I was wondering about the changes in the format documentation sources, in the last release I noticed a few function definitions changed from keymember(Key, N, TupleList) -> boolean() Test for membership of a list of tuples Key = term() N = 1..tuple_size(Tuple) TupleList = [Tuple]  Tuple = tuple() to Merge two key-sorted lists of tuples 1..tuple_size(Tuple) https://github.com/erlang/otp/blob/dev/lib/stdlib/doc/src/lists.xml#L299 is the source for these changes, somehow the recent erlang documention ( http://www.erlang.org/doc/man/lists.html#keymerge-3) shows the full parameter list information and I am quite confused at how that can be derived from the source xml. I was also just curious about the reason for the change and what was planned for going forward since http://erldocs.com/R14B03/stdlib/lists.html?i=0&search=lists%20keymerge#keymerge/3 is a lot less useful than http://erldocs.com/R14B02/stdlib/lists.html?i=0&search=lists%20keymerge#keymerge/3 Cheers, good work on the new release Dale -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom@REDACTED Sun Jun 12 08:08:32 2011 From: tom@REDACTED (Tom Samplonius) Date: Sat, 11 Jun 2011 23:08:32 -0700 (PDT) Subject: [erlang-questions] General help with unix sockets In-Reply-To: Message-ID: ... > my exim socket is working wonderfully but is it normal for it to > immediately disconnect: > > server app: > > > ./ss > Accepted connection! at ./ss line 51. > GOT: test at ./ss line 86. > Server session encountered read error 0: Normal disconnection. Depends on your client. Is the client closing the socket after each request? That is somewhat normal for Unix sockets, as they are cheap to open again when needed. And you mentioned you are using Exim as the client? Exim forks for each message, so if Exim opens the socket for each message, it will close it when done. Because Exim children just exit, when they are finished. Tom From dale@REDACTED Sun Jun 12 08:50:40 2011 From: dale@REDACTED (Dale Harvey) Date: Sun, 12 Jun 2011 07:50:40 +0100 Subject: [erlang-questions] is CEAN maintained? In-Reply-To: <30012253.358.1307520834207.JavaMail.geo-discussion-forums@yqnd17> References: <30012253.358.1307520834207.JavaMail.geo-discussion-forums@yqnd17> Message-ID: > we're in discussion with Yurii Rashkovskii (erlagner.org) and Martin > Logan (erlware.org) to provide community a common packaging environment > by the way. we'll try to merge our efforts to finally end up with this > packaging issue. +1 This is an effort I would love to see, all 3 package systems are really good but a good package manager is more than a sum of its parts, a single package manager to rule them all would rule Cheers Dale On 8 June 2011 09:13, Yurii Rashkovskii wrote: > To my best knowledge, Jacob is not planning much for neither epm nor sutro > at this moment, if I got this right. > > On Monday, June 6, 2011 9:44:49 AM UTC, Gleb Peregud wrote: > >> On Mon, Jun 6, 2011 at 10:22, Christophe Romain >> wrote: >> > we're in discussion with Yurii Rashkovskii (erlagner.org) and Martin >> > Logan (erlware.org) to provide community a common packaging environment >> > by the way. we'll try to merge our efforts to finally end up with this >> > packaging issue. >> >> Please include Jacob Vorreuter to the discussion. His EPM is a pretty >> handy thing too: >> >> https://github.com/JacobVorreuter/epm >> >> also it would be nice for rebar to support that common packaging >> environment and/or make it support specific package manager as sort of >> a plugin. >> >> Just my $0.02 >> _______________________________________________ >> erlang-questions mailing list >> erlang-q...@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Sun Jun 12 14:01:34 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 12 Jun 2011 13:01:34 +0100 Subject: [erlang-questions] is CEAN maintained? In-Reply-To: References: <30012253.358.1307520834207.JavaMail.geo-discussion-forums@yqnd17> Message-ID: > +1 > > This is an effort I would love to see, all 3 package systems are really good > but a good package manager is more than a sum of its parts, a single package > manager to rule them all would rule Something I really look forward to is being able to fetch dependencies (using e.g., rebar or whatever) as binary packages for my platform of choice. It takes all the hassle out of compiling dependencies and also means you can just install them once in your dev environment and you're good to go. That is really something to look forward to. From kostis@REDACTED Sun Jun 12 15:23:38 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sun, 12 Jun 2011 16:23:38 +0300 Subject: [erlang-questions] A PropEr announcement Message-ID: <4DF4BDDA.30809@cs.ntua.gr> We are happy to announce the first official public release of PropEr, a QuickCheck-inspired Property-Based Testing Tool for Erlang. The release comes with a proper site containing a User Guide, tips and tutorials for PropEr. It can be accessed at: http://proper.softlab.ntua.gr/ Among other things, it has a proper contact address on which we will be very happy to receive comments and feedback. Depending on interest, we may also set up a mailing list for PropEr users. Enjoy! Kostis Sagonas (on behalf of the PropEr developers) From ebegumisa@REDACTED Sun Jun 12 15:44:48 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sun, 12 Jun 2011 23:44:48 +1000 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <4DF4BDDA.30809@cs.ntua.gr> References: <4DF4BDDA.30809@cs.ntua.gr> Message-ID: Thank you, thank you, thank you! The tutorials are pure gold. It takes a little getting used to, but PropEr is a wonderful tool, even for someone new to automated testing. - Edmond - On Sun, 12 Jun 2011 23:23:38 +1000, Kostis Sagonas wrote: > We are happy to announce the first official public release of PropEr, a > QuickCheck-inspired Property-Based Testing Tool for Erlang. > > The release comes with a proper site containing a User Guide, tips and > tutorials for PropEr. It can be accessed at: > > http://proper.softlab.ntua.gr/ > > Among other things, it has a proper contact address on which we will be > very happy to receive comments and feedback. Depending on interest, we > may also set up a mailing list for PropEr users. > > Enjoy! > > Kostis Sagonas (on behalf of the PropEr developers) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From jameschurchman@REDACTED Sun Jun 12 15:45:48 2011 From: jameschurchman@REDACTED (James Churchman) Date: Sun, 12 Jun 2011 14:45:48 +0100 Subject: [erlang-questions] Documentation Format Changes (erldocs) In-Reply-To: References: Message-ID: <4060B6CB-F23C-458F-BFB9-8F56CA42F604@gmail.com> I have no idea personally, only a guess but ; All this type information in the xml files is duplicate now... the specs contain not only the function signature but also the types of the inputs. Specs are relatively recent addition, so the original creation of the xml documentation files probably predate the addition of specs to erlang. It might be that ErlDocs.com is incapable of parsing the specs out of the source and turning that into the documentation type information. The official erlang site appears to have it correct http://www.erlang.org/doc/man/lists.html#keymerge-3 So probably there is an effort to remove duplicate data from documentation xml files, reducing double updates along with the fact that the xml docs are just data, i don't believe they are machine checked for accuracy.. One other thing tho, why is the separate xml file preferred instead of the edoc inline notation, just to keep the two things separate and keep the code more readable? Anyhow thats my guess! James On 12 Jun 2011, at 01:25, Dale Harvey wrote: > Hey all > > I was wondering about the changes in the format documentation sources, in the last release I noticed a few function definitions changed from > > keymember(Key, N, TupleList) -> boolean() > Test for membership of a list of tuples > > Key = term() > N = 1..tuple_size(Tuple) > TupleList = [Tuple] >  Tuple = tuple() > > to > > > Merge two key-sorted lists of tuples > > 1..tuple_size(Tuple) > > > https://github.com/erlang/otp/blob/dev/lib/stdlib/doc/src/lists.xml#L299 is the source for these changes, somehow the recent erlang documention (http://www.erlang.org/doc/man/lists.html#keymerge-3) shows the full parameter list information and I am quite confused at how that can be derived from the source xml. > > I was also just curious about the reason for the change and what was planned for going forward > > since http://erldocs.com/R14B03/stdlib/lists.html?i=0&search=lists%20keymerge#keymerge/3 > is a lot less useful than http://erldocs.com/R14B02/stdlib/lists.html?i=0&search=lists%20keymerge#keymerge/3 > > Cheers, good work on the new release > Dale > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jameschurchman@REDACTED Sun Jun 12 16:06:29 2011 From: jameschurchman@REDACTED (James Churchman) Date: Sun, 12 Jun 2011 15:06:29 +0100 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> Message-ID: Amazing news for those who's projects fall below the radar of needing commercial support but really want to test their code to destruction, as we all should! Will lightly improve the standard of all erlang code out in the wild! James ps : One majorly nit-pciky thing... its more a personal hate than anything else.. I love the "Shirt & Tie with the erlang badge on it" logo, really fantastic.. less keen on the text as an image. On anything but the default zoom setting it looks blurry and you see lots of double anti-aliasing.. as the FontFace stuff is not yet widely supported best technique is to have a div with a background image of both a jpg/gif and an svg as well, any browsers that support svg will pick it up and all others get the jpg like ; #title_image_div { background-image: url('images/logo.jpg'); background-image: none,url('images/logo.svg'), url('images/logo.jpg'); } On 12 Jun 2011, at 14:44, Edmond Begumisa wrote: > Thank you, thank you, thank you! > > The tutorials are pure gold. > > It takes a little getting used to, but PropEr is a wonderful tool, even for someone new to automated testing. > > - Edmond - > > > On Sun, 12 Jun 2011 23:23:38 +1000, Kostis Sagonas wrote: > >> We are happy to announce the first official public release of PropEr, a QuickCheck-inspired Property-Based Testing Tool for Erlang. >> >> The release comes with a proper site containing a User Guide, tips and tutorials for PropEr. It can be accessed at: >> >> http://proper.softlab.ntua.gr/ >> >> Among other things, it has a proper contact address on which we will be very happy to receive comments and feedback. Depending on interest, we may also set up a mailing list for PropEr users. >> >> Enjoy! >> >> Kostis Sagonas (on behalf of the PropEr developers) >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From banibrata.dutta@REDACTED Sun Jun 12 16:29:37 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Sun, 12 Jun 2011 19:59:37 +0530 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? Message-ID: Hi, What would be a good way to correlate asynchronous events, spot patterns over a sliding window (s.a. of no. of events elapsed or time elapsed), with millions of events occurring simultaneously, using Erlang ? The set of possible events is known, and any unknown event is just flagged as 'unknown' (so all unknowns are similar). The set of possible event patterns can be enumerated, but is possibly quite a large set of patterns. -------------- next part -------------- An HTML attachment was scrubbed... URL: From banibrata.dutta@REDACTED Sun Jun 12 16:51:57 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Sun, 12 Jun 2011 20:21:57 +0530 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: References: Message-ID: Prematurely sent. On Sun, Jun 12, 2011 at 7:59 PM, Banibrata Dutta wrote: > > What would be a good way to correlate asynchronous events, spot patterns > over a sliding window (s.a. of no. of events elapsed or time elapsed), with > millions of events occurring simultaneously, using Erlang ? > > The set of possible events is known, and any unknown event is just flagged > as 'unknown' (so all unknowns are similar). The set of possible event > patterns can be enumerated, but is possibly quite a large set of patterns. > Was wondering as to what could be the approach taken to implement such a thing in pure Erlang. My initial thoughts were along the line of maintaining FSMs per event source, but with so many events and so many possible/valid patterns, the thing seems kind of unwieldy. Also, I'd like a non-programmer to be able to define new events and valid event patterns. I believe 'Complex Event Processing' is quite likely to be the standard approach for such things, as I've found from some posts, and solutions exist in Java world for same, but both as an academic exercise (for the fun of learning) and for a potentially simpler + better solution, would like to try doing this is Erlang. -- regards, Banibrata -------------- next part -------------- An HTML attachment was scrubbed... URL: From g9414002.pccu.edu.tw@REDACTED Sun Jun 12 17:38:09 2011 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Sun, 12 Jun 2011 23:38:09 +0800 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: References: Message-ID: Hi, Banibrata. Before my opinion, could you tell me where, from which books, did you know spot patterns? A request for reading materials suggestion. Back to this topic, On Sun, Jun 12, 2011 at 10:51 PM, Banibrata Dutta wrote: > Prematurely sent. > > > On Sun, Jun 12, 2011 at 7:59 PM, Banibrata Dutta < > banibrata.dutta@REDACTED> wrote: > >> >> What would be a good way to correlate asynchronous events, spot patterns >> over a sliding window (s.a. of no. of events elapsed or time elapsed), with >> millions of events occurring simultaneously, using Erlang ? >> >> The set of possible events is known, and any unknown event is just flagged >> as 'unknown' (so all unknowns are similar). The set of possible event >> patterns can be enumerated, but is possibly quite a large set of patterns. >> > > Was wondering as to what could be the approach taken to implement such a > thing in pure Erlang. My initial thoughts were along the line of maintaining > FSMs per event source, but with so many events and so many possible/valid > patterns, the thing seems kind of unwieldy. Also, I'd like a non-programmer > to be able to define new events and valid event patterns. > > I believe 'Complex Event Processing' is quite likely to be the standard > approach for such things, as I've found from some posts, and solutions exist > in Java world for same, but both as an academic exercise (for the fun of > learning) and for a potentially simpler + better solution, would like to try > doing this is Erlang. > > -- > regards, > Banibrata > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > The atomic message-sending command in Erlang is just to send things and done. "Send and forget it." Joe Armstrong and Erlang book writers said the word. A strip can be seen in many Erlang programs. Pid ! {self(), hello, world}, receive {Pid1, Pattern1} -> .... end, a 'send' followed by a 'receive'. It seems synchronous. There can be some asynchronous approach. For example, a listener is started and its PID is recorded as 'Pid1'. Every time it sends messages, the strip of sending can be Pid ! {Pid1, hello, world}, sending a message but no 'receive' instruction following. The PID of the listener is within the message, and it's treated as a callback endpoint by remote process. This is a general asynchronous model. On concepts of event-processing, in my opinion, it's similar to the Object-Oriented concepts. Some may want to define as many as possible, near all available, events/objects when planning and designing a system. However, most of these efforts tend to be in vain. Better approach can be to just list events/objects those in need and use their common patterns. /yhh -- Best Regards. --- Y-H. H. -------------- next part -------------- An HTML attachment was scrubbed... URL: From banibrata.dutta@REDACTED Sun Jun 12 17:38:20 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Sun, 12 Jun 2011 21:08:20 +0530 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> Message-ID: gr8 questions, and they certainly need clarification. cc'ing the group s.t. others could contribute too. On Sun, Jun 12, 2011 at 8:48 PM, Mihai Balea wrote: > > On Jun 12, 2011, at 10:51 AM, Banibrata Dutta wrote: > > Prematurely sent. > > On Sun, Jun 12, 2011 at 7:59 PM, Banibrata Dutta < > banibrata.dutta@REDACTED> wrote: > >> >> What would be a good way to correlate asynchronous events, spot patterns >> over a sliding window (s.a. of no. of events elapsed or time elapsed), with >> millions of events occurring simultaneously, using Erlang ? >> >> The set of possible events is known, and any unknown event is just flagged >> as 'unknown' (so all unknowns are similar). The set of possible event >> patterns can be enumerated, but is possibly quite a large set of patterns. >> > > Was wondering as to what could be the approach taken to implement such a > thing in pure Erlang. My initial thoughts were along the line of maintaining > FSMs per event source, but with so many events and so many possible/valid > patterns, the thing seems kind of unwieldy. Also, I'd like a non-programmer > to be able to define new events and valid event patterns. > > I believe 'Complex Event Processing' is quite likely to be the standard > approach for such things, as I've found from some posts, and solutions exist > in Java world for same, but both as an academic exercise (for the fun of > learning) and for a potentially simpler + better solution, would like to try > doing this is Erlang. > > > I think you need to define your problem better. > Sure, let me try. > What exactly do you mean by "millions of events occurring simultaneously"? > Okay, so I can say something like 500 events/second handled for correlation would be a more realistic number. > At exactly the same time? > Yes... some of the events might be from same source, but spaced by as little as 50ms, but mostly from different sources. There could be some heirarchical relationship between sources. Very typical case of network management scenario. E.g. a fault port on a switch, could probably cause hundreds of destination unreachable events, application response timeouts, heartbeat losses etc.. > Millions of events per second? Minute? Is that peak rate, average rate or > minimum rate? > Okay, I got over-enthusiastic :-) . Say 100 events/second typical, 500 events/second peak, no real minimum. What exactly is a pattern? > Node-A failed, Power in room-X where Node-A is kept failed, Nodes B,C,D which are served thru Node-A became unreachable, due to which Services L & M became unavailable, and due to which another dependent service N started giving inconsistent answers. So this is a pattern. However in this case, there's a possibility that Power-failure had nothing to do with Noda-A's failure, as backup power was available. Another pattern is, Power in room X failed, then Noda A failed, leading to failure of only Node D, because somehow Nodes B & C were dynamically configured to reroute. This is another pattern. What do you mean by "quite a large set of patterns"? Hundreds, thousands, > millions? > Several hundreds is a distinct possibility, and thousands are not impossible, but millions -- probably not. > How long is that sliding window? > >From few minutes (for certain type of events), to few days (for another type of events). > Can patterns encompass events coming from multiple sources or just one > source? > Yes, indeed. However in this case, there needs to a "relationship" between the event sources, that is pre-defined. E.g. some sense of "topology" exists. However it is likely that only 2% of the event sources are interrelated. > Are patterns concerned only with event ordering and occurrence or there are > timing issues involved as well? > Ordering, Timing, or any kind of causal relationship. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Sun Jun 12 18:18:52 2011 From: mihai@REDACTED (Mihai Balea) Date: Sun, 12 Jun 2011 12:18:52 -0400 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> Message-ID: <6289B011-5222-418A-A178-34DB304A44CE@hates.ms> On Jun 12, 2011, at 11:38 AM, Banibrata Dutta wrote: > gr8 questions, and they certainly need clarification. > cc'ing the group s.t. others could contribute too. > > On Sun, Jun 12, 2011 at 8:48 PM, Mihai Balea wrote: > > On Jun 12, 2011, at 10:51 AM, Banibrata Dutta wrote: > >> Prematurely sent. >> >> On Sun, Jun 12, 2011 at 7:59 PM, Banibrata Dutta wrote: >> >> What would be a good way to correlate asynchronous events, spot patterns over a sliding window (s.a. of no. of events elapsed or time elapsed), with millions of events occurring simultaneously, using Erlang ? >> >> The set of possible events is known, and any unknown event is just flagged as 'unknown' (so all unknowns are similar). The set of possible event patterns can be enumerated, but is possibly quite a large set of patterns. >> >> Was wondering as to what could be the approach taken to implement such a thing in pure Erlang. My initial thoughts were along the line of maintaining FSMs per event source, but with so many events and so many possible/valid patterns, the thing seems kind of unwieldy. Also, I'd like a non-programmer to be able to define new events and valid event patterns. >> >> I believe 'Complex Event Processing' is quite likely to be the standard approach for such things, as I've found from some posts, and solutions exist in Java world for same, but both as an academic exercise (for the fun of learning) and for a potentially simpler + better solution, would like to try doing this is Erlang. > > I think you need to define your problem better. > > Sure, let me try. > > What exactly do you mean by "millions of events occurring simultaneously"? > > Okay, so I can say something like 500 events/second handled for correlation would be a more realistic number. > > At exactly the same time? > > Yes... some of the events might be from same source, but spaced by as little as 50ms, but mostly from different sources. There could be some heirarchical relationship between sources. Very typical case of network management scenario. E.g. a fault port on a switch, could probably cause hundreds of destination unreachable events, application response timeouts, heartbeat losses etc.. > > Millions of events per second? Minute? Is that peak rate, average rate or minimum rate? > > Okay, I got over-enthusiastic :-) . Say 100 events/second typical, 500 events/second peak, no real minimum. > > What exactly is a pattern? > > Node-A failed, Power in room-X where Node-A is kept failed, Nodes B,C,D which are served thru Node-A became unreachable, due to which Services L & M became unavailable, and due to which another dependent service N started giving inconsistent answers. So this is a pattern. However in this case, there's a possibility that Power-failure had nothing to do with Noda-A's failure, as backup power was available. > > Another pattern is, Power in room X failed, then Noda A failed, leading to failure of only Node D, because somehow Nodes B & C were dynamically configured to reroute. This is another pattern. > > What do you mean by "quite a large set of patterns"? Hundreds, thousands, millions? > > Several hundreds is a distinct possibility, and thousands are not impossible, but millions -- probably not. > > How long is that sliding window? > > From few minutes (for certain type of events), to few days (for another type of events). > > Can patterns encompass events coming from multiple sources or just one source? > > Yes, indeed. However in this case, there needs to a "relationship" between the event sources, that is pre-defined. E.g. some sense of "topology" exists. However it is likely that only 2% of the event sources are interrelated. > > Are patterns concerned only with event ordering and occurrence or there are timing issues involved as well? > > Ordering, Timing, or any kind of causal relationship. Okay, that's a bit more descriptive :) First of all, don't dismiss your idea of using FSMs, however you might want to make them a bit more flexible. Maybe have a bunch of processes running pattern recognizers, say one process per pattern or class of patterns. Maybe you can filter events based on source so certain recognizers will only get certain events. If you have a low correlation between event sources maybe you can even design your system to distribute the processing of unrelated events to different nodes in a cluster (big scalability gain if you can do that). Another way of approaching this would be to duplicate the event stream to multiple nodes in your cluster and have each node only look for certain subsets of patterns. However, for a peak of 500 events/sec, you most likely can get away with running everything on a decently powerful machine (speaking from experience, we had a relatively similar, though simpler, system running just fine on a dual core server, handling up to 800 events/sec) If you intend to let non programmers define events and patterns you'll probably want to define some sort of DSL. Try as best as you can to make it declarative only, this way you can probably get away with files containing Erlang terms. Just a bunch of thoughts, hopefully this helps Mihai -------------- next part -------------- An HTML attachment was scrubbed... URL: From darach@REDACTED Sun Jun 12 18:46:56 2011 From: darach@REDACTED (Darach Ennis) Date: Sun, 12 Jun 2011 17:46:56 +0100 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> Message-ID: <4B5C5269-60F3-440D-85B5-6500B16F93B4@gmail.com> Hi guys, An excellent book introducing event processing is Opher Etzion's Event Processing in action: http://www.manning.com/etzion/ The Event Processing Technical Society also has a wealth of resources & use cases online: http://www.ep-ts.com/ Generally speaking you will want to separate the mechanics of windowing and correlation/combination from their utilisation so that you can compose synthetic events derived from external or 'ordinary' events easily and reuse windowed or other functions processed in the context of a window or other correlation/combination. A good example of windowing in Erlang projects is slide.erl/spiraltime.erl in riak core: https://github.com/basho/riak_core/blob/master/src/slide.erl https://github.com/basho/riak_core/blob/master/src/spiraltime.erl Event processing engines such as the open source Esper engine or commercial StreamBase (disclaimer: I work there) provide documentation online that could also be good references. Cheers, Darach. On 12 Jun 2011, at 16:38, Banibrata Dutta wrote: > gr8 questions, and they certainly need clarification. > cc'ing the group s.t. others could contribute too. > > On Sun, Jun 12, 2011 at 8:48 PM, Mihai Balea wrote: > > On Jun 12, 2011, at 10:51 AM, Banibrata Dutta wrote: > >> Prematurely sent. >> >> On Sun, Jun 12, 2011 at 7:59 PM, Banibrata Dutta wrote: >> >> What would be a good way to correlate asynchronous events, spot patterns over a sliding window (s.a. of no. of events elapsed or time elapsed), with millions of events occurring simultaneously, using Erlang ? >> >> The set of possible events is known, and any unknown event is just flagged as 'unknown' (so all unknowns are similar). The set of possible event patterns can be enumerated, but is possibly quite a large set of patterns. >> >> Was wondering as to what could be the approach taken to implement such a thing in pure Erlang. My initial thoughts were along the line of maintaining FSMs per event source, but with so many events and so many possible/valid patterns, the thing seems kind of unwieldy. Also, I'd like a non-programmer to be able to define new events and valid event patterns. >> >> I believe 'Complex Event Processing' is quite likely to be the standard approach for such things, as I've found from some posts, and solutions exist in Java world for same, but both as an academic exercise (for the fun of learning) and for a potentially simpler + better solution, would like to try doing this is Erlang. > > I think you need to define your problem better. > > Sure, let me try. > > What exactly do you mean by "millions of events occurring simultaneously"? > > Okay, so I can say something like 500 events/second handled for correlation would be a more realistic number. > > At exactly the same time? > > Yes... some of the events might be from same source, but spaced by as little as 50ms, but mostly from different sources. There could be some heirarchical relationship between sources. Very typical case of network management scenario. E.g. a fault port on a switch, could probably cause hundreds of destination unreachable events, application response timeouts, heartbeat losses etc.. > > Millions of events per second? Minute? Is that peak rate, average rate or minimum rate? > > Okay, I got over-enthusiastic :-) . Say 100 events/second typical, 500 events/second peak, no real minimum. > > What exactly is a pattern? > > Node-A failed, Power in room-X where Node-A is kept failed, Nodes B,C,D which are served thru Node-A became unreachable, due to which Services L & M became unavailable, and due to which another dependent service N started giving inconsistent answers. So this is a pattern. However in this case, there's a possibility that Power-failure had nothing to do with Noda-A's failure, as backup power was available. > > Another pattern is, Power in room X failed, then Noda A failed, leading to failure of only Node D, because somehow Nodes B & C were dynamically configured to reroute. This is another pattern. > > What do you mean by "quite a large set of patterns"? Hundreds, thousands, millions? > > Several hundreds is a distinct possibility, and thousands are not impossible, but millions -- probably not. > > How long is that sliding window? > > From few minutes (for certain type of events), to few days (for another type of events). > > Can patterns encompass events coming from multiple sources or just one source? > > Yes, indeed. However in this case, there needs to a "relationship" between the event sources, that is pre-defined. E.g. some sense of "topology" exists. However it is likely that only 2% of the event sources are interrelated. > > Are patterns concerned only with event ordering and occurrence or there are timing issues involved as well? > > Ordering, Timing, or any kind of causal relationship. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ebegumisa@REDACTED Sun Jun 12 19:54:59 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Mon, 13 Jun 2011 03:54:59 +1000 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> Message-ID: My interpretation of your question is that you want to do one-off message tracing. I think most of what you need is already in Erlang/OTP... 1. You want to trace particular massages passed in your system over a period Y, you are particularly interested in 'EXIT' messages, but not necessarily: You could use the trace BIFs/dbg tool for this. Start an appropriate tracer with dbg:tracer, specify which processes you want to trace with dbg:p with one of the m|s|r flags to log the events, watch your system for a while, turn the tracer off (keep in mind the negative impact such tracing might have on a live system.) Analyse the trace output. You could also use your own tracer handler fun to filter/process the messages. 2. You want to trace massages over a number of nodes: Though I haven't used it, I believe this is what the Inviso OTP tool is for. You could also have a look at the ttb OTP tool. 3. You want to trace "cascading" messages in your system: Have a look at erlang:seq_trace. I believe dbg and ttb both have support for sequential tracing (I don't think Inviso does.) 4. You want a convenient way of walking through traces: The OTP et_viewer is a really nice tool I use a lot. Though the OTP Event Tracer can output ordinary trace data produced by dbg, where it really gets fun is when you feed it trace data by calling et:trace_me in your code (I normally wrap this in a ?TRACE macro). I like to think of it as a replacement for using io:format to print debug messages. The "detail" argument is really nice for "zooming" in/out as you page through trace data. You might find that useful for analysing a large number of messages. In short, I'm not sure you need any special "architectural design patterns". Just design your system as you normally would, using processes to model what you observe in the world, then trace it when it misbehaves or when you want to observe it. The traces *are* your sliding windows... unless I've completely mis-understood your question! - Edmond - On Mon, 13 Jun 2011 01:38:20 +1000, Banibrata Dutta wrote: > gr8 questions, and they certainly need clarification. > cc'ing the group s.t. others could contribute too. > > On Sun, Jun 12, 2011 at 8:48 PM, Mihai Balea wrote: > >> >> On Jun 12, 2011, at 10:51 AM, Banibrata Dutta wrote: >> >> Prematurely sent. >> >> On Sun, Jun 12, 2011 at 7:59 PM, Banibrata Dutta < >> banibrata.dutta@REDACTED> wrote: >> >>> >>> What would be a good way to correlate asynchronous events, spot >>> patterns >>> over a sliding window (s.a. of no. of events elapsed or time elapsed), >>> with >>> millions of events occurring simultaneously, using Erlang ? >>> >>> The set of possible events is known, and any unknown event is just >>> flagged >>> as 'unknown' (so all unknowns are similar). The set of possible event >>> patterns can be enumerated, but is possibly quite a large set of >>> patterns. >>> >> >> Was wondering as to what could be the approach taken to implement such a >> thing in pure Erlang. My initial thoughts were along the line of >> maintaining >> FSMs per event source, but with so many events and so many >> possible/valid >> patterns, the thing seems kind of unwieldy. Also, I'd like a >> non-programmer >> to be able to define new events and valid event patterns. >> >> I believe 'Complex Event Processing' is quite likely to be the standard >> approach for such things, as I've found from some posts, and solutions >> exist >> in Java world for same, but both as an academic exercise (for the fun of >> learning) and for a potentially simpler + better solution, would like >> to try >> doing this is Erlang. >> >> >> I think you need to define your problem better. >> > > Sure, let me try. > > >> What exactly do you mean by "millions of events occurring >> simultaneously"? >> > > Okay, so I can say something like 500 events/second handled for > correlation > would be a more realistic number. > > >> At exactly the same time? >> > > Yes... some of the events might be from same source, but spaced by as > little > as 50ms, but mostly from different sources. There could be some > heirarchical > relationship between sources. Very typical case of network management > scenario. E.g. a fault port on a switch, could probably cause hundreds of > destination unreachable events, application response timeouts, heartbeat > losses etc.. > > >> Millions of events per second? Minute? Is that peak rate, average rate >> or >> minimum rate? >> > > Okay, I got over-enthusiastic :-) . Say 100 events/second typical, 500 > events/second peak, no real minimum. > > What exactly is a pattern? >> > > Node-A failed, Power in room-X where Node-A is kept failed, Nodes B,C,D > which are served thru Node-A became unreachable, due to which Services L > & M > became unavailable, and due to which another dependent service N started > giving inconsistent answers. So this is a pattern. However in this case, > there's a possibility that Power-failure had nothing to do with Noda-A's > failure, as backup power was available. > > Another pattern is, Power in room X failed, then Noda A failed, leading > to > failure of only Node D, because somehow Nodes B & C were dynamically > configured to reroute. This is another pattern. > > What do you mean by "quite a large set of patterns"? Hundreds, thousands, >> millions? >> > > Several hundreds is a distinct possibility, and thousands are not > impossible, but millions -- probably not. > > >> How long is that sliding window? >> > > From few minutes (for certain type of events), to few days (for another > type > of events). > > >> Can patterns encompass events coming from multiple sources or just one >> source? >> > > Yes, indeed. However in this case, there needs to a "relationship" > between > the event sources, that is pre-defined. E.g. some sense of "topology" > exists. However it is likely that only 2% of the event sources are > interrelated. > > >> Are patterns concerned only with event ordering and occurrence or there >> are >> timing issues involved as well? >> > > Ordering, Timing, or any kind of causal relationship. -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From jameschurchman@REDACTED Sun Jun 12 20:16:23 2011 From: jameschurchman@REDACTED (James Churchman) Date: Sun, 12 Jun 2011 19:16:23 +0100 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> Message-ID: <9A3DD84A-DB03-4284-B41C-A2CE9528E5C1@gmail.com> yes with out the font that you like in unsupporting browsers, so it looks totally different in them :-) using SVG you get pixel perfect rendering from a vector with an image fallback, which is more suitable for logo's etc... like the one in question :-) On 12 Jun 2011, at 19:00, Dale Harvey wrote: > @fontface is very well supported and falls back gracefully, which image replacement does not. > > On 12 Jun 2011 16:06, "James Churchman" wrote: > > Amazing news for those who's projects fall below the radar of needing commercial support but really want to test their code to destruction, as we all should! Will lightly improve the standard of all erlang code out in the wild! > > > > James > > > > > > > > ps : > > One majorly nit-pciky thing... its more a personal hate than anything else.. I love the "Shirt & Tie with the erlang badge on it" logo, really fantastic.. less keen on the text as an image. On anything but the default zoom setting it looks blurry and you see lots of double anti-aliasing.. as the FontFace stuff is not yet widely supported best technique is to have a div with a background image of both a jpg/gif and an svg as well, any browsers that support svg will pick it up and all others get the jpg like ; > > > > #title_image_div > > { > > background-image: url('images/logo.jpg'); > > background-image: none,url('images/logo.svg'), url('images/logo.jpg'); > > } > > > > On 12 Jun 2011, at 14:44, Edmond Begumisa wrote: > > > >> Thank you, thank you, thank you! > >> > >> The tutorials are pure gold. > >> > >> It takes a little getting used to, but PropEr is a wonderful tool, even for someone new to automated testing. > >> > >> - Edmond - > >> > >> > >> On Sun, 12 Jun 2011 23:23:38 +1000, Kostis Sagonas wrote: > >> > >>> We are happy to announce the first official public release of PropEr, a QuickCheck-inspired Property-Based Testing Tool for Erlang. > >>> > >>> The release comes with a proper site containing a User Guide, tips and tutorials for PropEr. It can be accessed at: > >>> > >>> http://proper.softlab.ntua.gr/ > >>> > >>> Among other things, it has a proper contact address on which we will be very happy to receive comments and feedback. Depending on interest, we may also set up a mailing list for PropEr users. > >>> > >>> Enjoy! > >>> > >>> Kostis Sagonas (on behalf of the PropEr developers) > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > >> -- > >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jameschurchman@REDACTED Sun Jun 12 20:28:45 2011 From: jameschurchman@REDACTED (James Churchman) Date: Sun, 12 Jun 2011 19:28:45 +0100 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <9A3DD84A-DB03-4284-B41C-A2CE9528E5C1@gmail.com> References: <4DF4BDDA.30809@cs.ntua.gr> <9A3DD84A-DB03-4284-B41C-A2CE9528E5C1@gmail.com> Message-ID: ( actually im wrong, it is widely supported, but requires a different font format for every browser and renders very poorly in IE at larger font sizes... http://blog.themeforest.net/tutorials/how-to-achieve-cross-browser-font-face-support/ ) On 12 Jun 2011, at 19:16, James Churchman wrote: > yes with out the font that you like in unsupporting browsers, so it looks totally different in them :-) using SVG you get pixel perfect rendering from a vector with an image fallback, which is more suitable for logo's etc... like the one in question :-) > > > > On 12 Jun 2011, at 19:00, Dale Harvey wrote: > >> @fontface is very well supported and falls back gracefully, which image replacement does not. >> >> On 12 Jun 2011 16:06, "James Churchman" wrote: >> > Amazing news for those who's projects fall below the radar of needing commercial support but really want to test their code to destruction, as we all should! Will lightly improve the standard of all erlang code out in the wild! >> > >> > James >> > >> > >> > >> > ps : >> > One majorly nit-pciky thing... its more a personal hate than anything else.. I love the "Shirt & Tie with the erlang badge on it" logo, really fantastic.. less keen on the text as an image. On anything but the default zoom setting it looks blurry and you see lots of double anti-aliasing.. as the FontFace stuff is not yet widely supported best technique is to have a div with a background image of both a jpg/gif and an svg as well, any browsers that support svg will pick it up and all others get the jpg like ; >> > >> > #title_image_div >> > { >> > background-image: url('images/logo.jpg'); >> > background-image: none,url('images/logo.svg'), url('images/logo.jpg'); >> > } >> > >> > On 12 Jun 2011, at 14:44, Edmond Begumisa wrote: >> > >> >> Thank you, thank you, thank you! >> >> >> >> The tutorials are pure gold. >> >> >> >> It takes a little getting used to, but PropEr is a wonderful tool, even for someone new to automated testing. >> >> >> >> - Edmond - >> >> >> >> >> >> On Sun, 12 Jun 2011 23:23:38 +1000, Kostis Sagonas wrote: >> >> >> >>> We are happy to announce the first official public release of PropEr, a QuickCheck-inspired Property-Based Testing Tool for Erlang. >> >>> >> >>> The release comes with a proper site containing a User Guide, tips and tutorials for PropEr. It can be accessed at: >> >>> >> >>> http://proper.softlab.ntua.gr/ >> >>> >> >>> Among other things, it has a proper contact address on which we will be very happy to receive comments and feedback. Depending on interest, we may also set up a mailing list for PropEr users. >> >>> >> >>> Enjoy! >> >>> >> >>> Kostis Sagonas (on behalf of the PropEr developers) >> >>> _______________________________________________ >> >>> erlang-questions mailing list >> >>> erlang-questions@REDACTED >> >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> >> -- >> >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cromain@REDACTED Sun Jun 12 20:53:01 2011 From: cromain@REDACTED (Christophe Romain) Date: Sun, 12 Jun 2011 20:53:01 +0200 Subject: [erlang-questions] is CEAN maintained? In-Reply-To: References: <30012253.358.1307520834207.JavaMail.geo-discussion-forums@yqnd17> Message-ID: That is something cean framework does for you, and a key feature of any serious packaging tool anyway On Jun 12, 2011 2:01 PM, "Tim Watson" wrote: >> +1 >> >> This is an effort I would love to see, all 3 package systems are really good >> but a good package manager is more than a sum of its parts, a single package >> manager to rule them all would rule > > Something I really look forward to is being able to fetch dependencies > (using e.g., rebar or whatever) as binary packages for my platform of > choice. It takes all the hassle out of compiling dependencies and also > means you can just install them once in your dev environment and > you're good to go. That is really something to look forward to. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ebegumisa@REDACTED Sun Jun 12 21:31:42 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Mon, 13 Jun 2011 05:31:42 +1000 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> Message-ID: Hmmm... on thinking about this more, I think I can offer a more useful response. What you really want is the **OTP Event Tracer** The general idea is you have an event collector, then you call et_collector:report_event to send events from your processes to the collector. Now your non-programmer users can use the et_viewer GUI as a sliding window through the events (complete with a spiffy detail zoom in/out). The et_viewer queries the et_collector for the events to display. Now, for selecting the "event patterns" you describe, et has a concept of filters which sounds ideal. Your users can choose filters from the et_viewer menu and apply these to the trace display (they won't be able to define their own though since it requires programming, you'll have to supply the filters -- see et_viewer:start filter_fun option.) BUT here's the problem -- your "event patterns" are not just filters, they need some way of accumulating the events to be useful. AFI can tell, et doesn't support this. So you can either patch et, or use an ugly hack similar to this: make your filter_fun have a side effect of calling a registered process that accumulates a maximum number of previously filtered events and decides weather these events fit the filter's "event pattern" (nasty, I know but...) - Edmond - See inline for a few more ideas... On Mon, 13 Jun 2011 03:54:59 +1000, Edmond Begumisa wrote: > My interpretation of your question is that you want to do one-off > message tracing. I think most of what you need is already in > Erlang/OTP... > > 1. You want to trace particular massages passed in your system over a > period Y, you are particularly interested in 'EXIT' messages, but not > necessarily: > > You could use the trace BIFs/dbg tool for this. Start an appropriate > tracer with dbg:tracer, specify which processes you want to trace with > dbg:p with one of the m|s|r flags to log the events, watch your system > for a while, turn the tracer off (keep in mind the negative impact such > tracing might have on a live system.) Analyse the trace output. You > could also use your own tracer handler fun to filter/process the > messages. > > 2. You want to trace massages over a number of nodes: > > Though I haven't used it, I believe this is what the Inviso OTP tool is > for. You could also have a look at the ttb OTP tool. > > 3. You want to trace "cascading" messages in your system: > > Have a look at erlang:seq_trace. I believe dbg and ttb both have support > for sequential tracing (I don't think Inviso does.) > > 4. You want a convenient way of walking through traces: > > The OTP et_viewer is a really nice tool I use a lot. Though the OTP > Event Tracer can output ordinary trace data produced by dbg, where it > really gets fun is when you feed it trace data by calling et:trace_me in > your code (I normally wrap this in a ?TRACE macro). I like to think of > it as a replacement for using io:format to print debug messages. The > "detail" argument is really nice for "zooming" in/out as you page > through trace data. You might find that useful for analysing a large > number of messages. > > In short, I'm not sure you need any special "architectural design > patterns". Just design your system as you normally would, using > processes to model what you observe in the world, then trace it when it > misbehaves or when you want to observe it. The traces *are* your sliding > windows... unless I've completely mis-understood your question! > > - Edmond - > > On Mon, 13 Jun 2011 01:38:20 +1000, Banibrata Dutta > wrote: > >> gr8 questions, and they certainly need clarification. >> cc'ing the group s.t. others could contribute too. >> >> On Sun, Jun 12, 2011 at 8:48 PM, Mihai Balea wrote: >> >>> >>> On Jun 12, 2011, at 10:51 AM, Banibrata Dutta wrote: >>> >>> Prematurely sent. >>> >>> On Sun, Jun 12, 2011 at 7:59 PM, Banibrata Dutta < >>> banibrata.dutta@REDACTED> wrote: >>> >>>> >>>> What would be a good way to correlate asynchronous events, spot >>>> patterns >>>> over a sliding window (s.a. of no. of events elapsed or time >>>> elapsed), with >>>> millions of events occurring simultaneously, using Erlang ? >>>> >>>> The set of possible events is known, and any unknown event is just >>>> flagged >>>> as 'unknown' (so all unknowns are similar). The set of possible event >>>> patterns can be enumerated, but is possibly quite a large set of >>>> patterns. >>>> >>> >>> Was wondering as to what could be the approach taken to implement such >>> a >>> thing in pure Erlang. My initial thoughts were along the line of >>> maintaining >>> FSMs per event source, but with so many events and so many >>> possible/valid >>> patterns, the thing seems kind of unwieldy. Also, I'd like a >>> non-programmer >>> to be able to define new events and valid event patterns. >>> >>> I believe 'Complex Event Processing' is quite likely to be the standard >>> approach for such things, as I've found from some posts, and solutions >>> exist >>> in Java world for same, but both as an academic exercise (for the fun >>> of >>> learning) and for a potentially simpler + better solution, would like >>> to try >>> doing this is Erlang. >>> >>> >>> I think you need to define your problem better. >>> >> >> Sure, let me try. >> >> >>> What exactly do you mean by "millions of events occurring >>> simultaneously"? >>> >> >> Okay, so I can say something like 500 events/second handled for >> correlation >> would be a more realistic number. >> >> >>> At exactly the same time? >>> >> >> Yes... some of the events might be from same source, but spaced by as >> little >> as 50ms, but mostly from different sources. There could be some >> heirarchical >> relationship between sources. Very typical case of network management >> scenario. E.g. a fault port on a switch, could probably cause hundreds >> of >> destination unreachable events, application response timeouts, heartbeat >> losses etc.. >> >> >>> Millions of events per second? Minute? Is that peak rate, average rate >>> or >>> minimum rate? >>> >> >> Okay, I got over-enthusiastic :-) . Say 100 events/second typical, 500 >> events/second peak, no real minimum. >> >> What exactly is a pattern? >>> >> >> Node-A failed, Power in room-X where Node-A is kept failed, Nodes B,C,D >> which are served thru Node-A became unreachable, due to which Services >> L & M >> became unavailable, and due to which another dependent service N started >> giving inconsistent answers. So this is a pattern. However in this case, >> there's a possibility that Power-failure had nothing to do with Noda-A's >> failure, as backup power was available. >> >> Another pattern is, Power in room X failed, then Noda A failed, leading >> to >> failure of only Node D, because somehow Nodes B & C were dynamically >> configured to reroute. This is another pattern. >> >> What do you mean by "quite a large set of patterns"? Hundreds, >> thousands, >>> millions? >>> >> >> Several hundreds is a distinct possibility, and thousands are not >> impossible, but millions -- probably not. >> >> >>> How long is that sliding window? >>> >> >> From few minutes (for certain type of events), to few days (for another >> type >> of events). >> >> >>> Can patterns encompass events coming from multiple sources or just one >>> source? >>> >> >> Yes, indeed. However in this case, there needs to a "relationship" >> between >> the event sources, that is pre-defined. E.g. some sense of "topology" >> exists. However it is likely that only 2% of the event sources are >> interrelated. >> >> >>> Are patterns concerned only with event ordering and occurrence or >>> there are >>> timing issues involved as well? >>> >> >> Ordering, Timing, or any kind of causal relationship. > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From bengt.kleberg@REDACTED Mon Jun 13 07:31:32 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 13 Jun 2011 07:31:32 +0200 Subject: [erlang-questions] ot: Swedish article featuring Erlang Solutions CTO Message-ID: <1307943092.4774.11.camel@seasc1137> Greetings, For the Swedish reading members of this list that does not normally read the Computer Sweden website. http://www.idg.se/2.1085/1.390217/manga-karnor-provar-hjarnor bengt From robert.virding@REDACTED Mon Jun 13 10:41:20 2011 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 13 Jun 2011 08:41:20 +0000 (GMT) Subject: [erlang-questions] ot: Swedish article featuring Erlang Solutions CTO In-Reply-To: <1307943092.4774.11.camel@seasc1137> Message-ID: <377433395.287951307954480738.JavaMail.root@zimbra> What is perhaps more interesting than the article itself are all the comments. "I know nothing about language X but I know that it is 10-15 times slower than language Y". Robert ----- "Bengt Kleberg" wrote: > Greetings, > > For the Swedish reading members of this list that does not normally > read > the Computer Sweden website. > > http://www.idg.se/2.1085/1.390217/manga-karnor-provar-hjarnor > > > bengt > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jesper.louis.andersen@REDACTED Mon Jun 13 11:46:17 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 13 Jun 2011 11:46:17 +0200 Subject: [erlang-questions] ot: Swedish article featuring Erlang Solutions CTO In-Reply-To: <377433395.287951307954480738.JavaMail.root@zimbra> References: <1307943092.4774.11.camel@seasc1137> <377433395.287951307954480738.JavaMail.root@zimbra> Message-ID: On Mon, Jun 13, 2011 at 10:41, Robert Virding wrote: > What is perhaps more interesting than the article itself are all the comments. "I know nothing about language X but I know that it is 10-15 times slower than language Y". He is considering the "Computer Language benchmarks game", specifically the "fasta" benchmark on a Quad Core: http://shootout.alioth.debian.org/u64q/benchmark.php?test=fasta&lang=all You can make the opposite conclusion by considering the thread-ring benchmark: http://shootout.alioth.debian.org/u64q/benchmark.php?test=threadring&lang=all Which shows Haskell is fastest, Erlang is about 3 times slower and GCC is 27 times slower than Haskell. The beauty is indeed in the eye of the beholder. The "fasta" benchmark is not very Erlang-optimized either. First, the benchmark has three parts which are independent. We are only using one out of the 4 CPUs at the moment. Second, the first part is about cycling on an array. It is straightforward to do in C, and we will have more work to do. Rearranging the code around the binaries used can probably yield a speedup. For starters, we could ask if BEAM optimizes binaries at all. But the run time is probably dwarfed by the next two parts, so going parallel should be the first step. Third, the last two parts are about lookups in tables. The C-version builds up an array in which it looks up values quickly in constant time. The array is built such that it minimizes floating point roundoff cases and uses a branch-prediction hint to make it faster. The Erlang code creates a list and does O(n) lookups in the list. Finally, this is floating point intensive, an area where Erlang doesn't really shine too much. In other words: Build up a tuple of 4096 elements like in the C-version and use it with element/2 to get constant time lookup and see what happens :) ------ The danger of concluding something from a synthetic benchmark is that you are going to conclude the wrong thing. Especially if you don't know the internals of the language. The Haskell version for instance does caching of the randomized lookup table as well. This ought to speed up the computation a bit for Haskell as well. I Dare say the algorithm used is different when considering the three languages. Also, note that the benchmark is *extremely* cache-friendly. All data easily fits into the CPU cache and you will never ever need to move anything from main memory if you do it somewhat correctly. Since you are not waiting on the memory bottleneck at all, of course it will be blindingly fast. In my experience, a language with the traits of Erlang (dynamically typed, interpreted) fares worse here than other languages. If the memory bottleneck comes into play, then not so much. ------ Finally, the whole notion that "Functional Programming is slow" is a fallacy. Yes, back in 1981 when memory was constrained and the only thing you had were interpreted Lisps, then yes, *perhaps*. What is more likely however is that people did not understand how to program functionally. Today, 30 years of research later, there has been so much gain that it is not clear at all it is slow on a single core. And for multiple cores... -- J. From lukas.larsson@REDACTED Mon Jun 13 12:03:28 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Mon, 13 Jun 2011 10:03:28 +0000 (GMT) Subject: [erlang-questions] Documentation Format Changes (erldocs) In-Reply-To: Message-ID: <327898192.289261307959408715.JavaMail.root@zimbra> Hi Dale! All of the documentation is right now being reworked to generate type information from the specs in the source instead of manually writing them. tells docbuilder (I think) to generate the type description from the specs instead of the XML file. If you dig around in the source code of docbuilder/edoc there should be functions somewhere there which you can use to generate type information for erldocs. Lukas ----- Original Message ----- From: "Dale Harvey" To: "erlang-questions" Sent: Sunday, 12 June, 2011 02:25:29 GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: [erlang-questions] Documentation Format Changes (erldocs) Hey all I was wondering about the changes in the format documentation sources, in the last release I noticed a few function definitions changed from keymember(Key, N, TupleList) -> boolean() Test for membership of a list of tuples Key = term() N = 1..tuple_size(Tuple) TupleList = [Tuple]   Tuple = tuple() to Merge two key-sorted lists of tuples 1..tuple_size(Tuple) https://github.com/erlang/otp/blob/dev/lib/stdlib/doc/src/lists.xml#L299 is the source for these changes, somehow the recent erlang documention ( http://www.erlang.org/doc/man/lists.html#keymerge-3 ) shows the full parameter list information and I am quite confused at how that can be derived from the source xml. I was also just curious about the reason for the change and what was planned for going forward since http://erldocs.com/R14B03/stdlib/lists.html?i=0&search=lists%20keymerge#keymerge/3 is a lot less useful than http://erldocs.com/R14B02/stdlib/lists.html?i=0&search=lists%20keymerge#keymerge/3 Cheers, good work on the new release Dale _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From joelr1@REDACTED Mon Jun 13 13:55:09 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 13 Jun 2011 15:55:09 +0400 Subject: [erlang-questions] rebar escriptize and shared libraries Message-ID: <716930ED-6676-4035-ACAC-BDF934A45F89@gmail.com> Given this rebar config: {lib_dirs,["deps"]}. {deps, [ {'getopt', ".*", {git, "git://github.com/jcomellas/getopt.git", "HEAD"}}, {'json', ".*", {git, "git://github.com/davisp/eep0018.git", "HEAD"}}, {'lhttpc', ".*", {git, "git://github.com/esl/lhttpc.git", "HEAD"}} ]}. {escript_incl_apps, [getopt, json, lhttpc]}. How do I tell rebar to include deps/json/prive/json.so? It doesn't seem to be doing it because I get an error =ERROR REPORT==== 13-Jun-2011::15:49:20 === The on_load function for module json returned {error, {load_failed, "Failed to load NIF library: 'dlopen(xxx/json/priv/json.so, 2): no suitable image found. Did find:\n\txxx/json/priv/json.so: stat() failed with errno=20'"}} Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From banibrata.dutta@REDACTED Mon Jun 13 15:12:51 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Mon, 13 Jun 2011 18:42:51 +0530 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: <6289B011-5222-418A-A178-34DB304A44CE@hates.ms> References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> <6289B011-5222-418A-A178-34DB304A44CE@hates.ms> Message-ID: On Sun, Jun 12, 2011 at 9:48 PM, Mihai Balea wrote: > > On Jun 12, 2011, at 11:38 AM, Banibrata Dutta wrote: > > gr8 questions, and they certainly need clarification. > cc'ing the group s.t. others could contribute too. > > On Sun, Jun 12, 2011 at 8:48 PM, Mihai Balea wrote: > >> >> On Jun 12, 2011, at 10:51 AM, Banibrata Dutta wrote: >> >> Prematurely sent. >> >> On Sun, Jun 12, 2011 at 7:59 PM, Banibrata Dutta < >> banibrata.dutta@REDACTED> wrote: >> >>> >>> What would be a good way to correlate asynchronous events, spot patterns >>> over a sliding window (s.a. of no. of events elapsed or time elapsed), with >>> millions of events occurring simultaneously, using Erlang ? >>> >>> The set of possible events is known, and any unknown event is just >>> flagged as 'unknown' (so all unknowns are similar). The set of possible >>> event patterns can be enumerated, but is possibly quite a large set of >>> patterns. >>> >> >> Was wondering as to what could be the approach taken to implement such a >> thing in pure Erlang. My initial thoughts were along the line of maintaining >> FSMs per event source, but with so many events and so many possible/valid >> patterns, the thing seems kind of unwieldy. Also, I'd like a non-programmer >> to be able to define new events and valid event patterns. >> >> I believe 'Complex Event Processing' is quite likely to be the standard >> approach for such things, as I've found from some posts, and solutions exist >> in Java world for same, but both as an academic exercise (for the fun of >> learning) and for a potentially simpler + better solution, would like to try >> doing this is Erlang. >> >> >> I think you need to define your problem better. >> > > Sure, let me try. > > >> What exactly do you mean by "millions of events occurring simultaneously"? >> >> > > Okay, so I can say something like 500 events/second handled for correlation > would be a more realistic number. > > >> At exactly the same time? >> > > Yes... some of the events might be from same source, but spaced by as > little as 50ms, but mostly from different sources. There could be some > heirarchical relationship between sources. Very typical case of network > management scenario. E.g. a fault port on a switch, could probably cause > hundreds of destination unreachable events, application response timeouts, > heartbeat losses etc.. > > >> Millions of events per second? Minute? Is that peak rate, average rate or >> minimum rate? >> > > Okay, I got over-enthusiastic :-) . Say 100 events/second typical, 500 > events/second peak, no real minimum. > > What exactly is a pattern? >> > > Node-A failed, Power in room-X where Node-A is kept failed, Nodes B,C,D > which are served thru Node-A became unreachable, due to which Services L & M > became unavailable, and due to which another dependent service N started > giving inconsistent answers. So this is a pattern. However in this case, > there's a possibility that Power-failure had nothing to do with Noda-A's > failure, as backup power was available. > > Another pattern is, Power in room X failed, then Noda A failed, leading to > failure of only Node D, because somehow Nodes B & C were dynamically > configured to reroute. This is another pattern. > > What do you mean by "quite a large set of patterns"? Hundreds, thousands, >> millions? >> > > Several hundreds is a distinct possibility, and thousands are not > impossible, but millions -- probably not. > > >> How long is that sliding window? >> > > From few minutes (for certain type of events), to few days (for another > type of events). > > >> Can patterns encompass events coming from multiple sources or just one >> source? >> > > Yes, indeed. However in this case, there needs to a "relationship" between > the event sources, that is pre-defined. E.g. some sense of "topology" > exists. However it is likely that only 2% of the event sources are > interrelated. > > >> Are patterns concerned only with event ordering and occurrence or there >> are timing issues involved as well? >> > > Ordering, Timing, or any kind of causal relationship. > > Thanks for answering. > Okay, that's a bit more descriptive :) > > First of all, don't dismiss your idea of using FSMs, > Well, I haven't. In fact, starting with a small set of possible patterns, seems to be the most intuitive approach. When I bring in the thought of permitting non-programmers to write "something" (e.g. natural language rules, or a simple DSL snippet), and then such a thing modifying the FSM, without me having to code anything further in Erlang, is when I am hitting a mental roadblock. I've coded fairly complex FSM's in C/C++ using the table-based approach. Does gen_fsm take a similar approach ? Will it be a good, natural fit ? Maybe a rather naive question for people extremely familiar with Erlang, but I'm still very much on the learning curve's positive vector. > however you might want to make them a bit more flexible. Maybe have a bunch > of processes running pattern recognizers, say one process per pattern or > class of patterns. > My natural tendency was to think more in terms of 1 process per source, e.g. 1st event from a source causes an Erlang process to be created, which then embodies the entire possible FSM. The main issue with pattern based processes (if I understood it correctly), is determining the right process to hand-off the event to, and once handed-off, backtracking would mean extra work. > Maybe you can filter events based on source so certain recognizers will > only get certain events. > Yes, that was the starting point, as I had imagined. > If you have a low correlation between event sources maybe you can even > design your system to distribute the processing of unrelated events to > different nodes in a cluster (big scalability gain if you can do that). > Excellent thought... indeed. Source based sharding. > Another way of approaching this would be to duplicate the event stream to > multiple nodes in your cluster and have each node only look for certain > subsets of patterns. > Actually, my bad, I should've mentioned, I wanted to do all this in Real-Time. So I don't really have a pattern-string, so to say. I don't, without that I can really take this approach. Or can I ? However, for a peak of 500 events/sec, you most likely can get away with > running everything on a decently powerful machine (speaking from experience, > we had a relatively similar, though simpler, system running just fine on a > dual core server, handling up to 800 events/sec) > > If you intend to let non programmers define events and patterns you'll > probably want to define some sort of DSL. Try as best as you can to make it > declarative only, this way you can probably get away with files containing > Erlang terms. > Thanks for the tip. A declarative DSL sounds good. > Just a bunch of thoughts, hopefully this helps > They certainly do ! Thanks. -- regards, Banibrata http://www.linkedin.com/in/bdutta http://twitter.com/edgeliving -------------- next part -------------- An HTML attachment was scrubbed... URL: From banibrata.dutta@REDACTED Mon Jun 13 15:24:30 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Mon, 13 Jun 2011 18:54:30 +0530 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: <4B5C5269-60F3-440D-85B5-6500B16F93B4@gmail.com> References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> <4B5C5269-60F3-440D-85B5-6500B16F93B4@gmail.com> Message-ID: On Sun, Jun 12, 2011 at 10:16 PM, Darach Ennis wrote: > > An excellent book introducing event processing is Opher Etzion's Event > Processing in action: > > http://www.manning.com/etzion/ > Read reviews and some sample pages on google-books, looks very interesting. > The Event Processing Technical Society also has a wealth of resources & use > cases online: > > http://www.ep-ts.com/ > Okay, will go through. One of my doubts still is, whether I really need a CEP framework (based on little I've read on CEP so far -- mostly on Esper site). Given your extensive background in this domain, and my requirements (happy to answer more questions if needed), do you believe CEP is the way to go ? Or, can I hope to achieve with a pure FSM based approach ? > Generally speaking you will want to separate the mechanics of windowing and > correlation/combination from their utilisation > I think I understand this part... > so that you can compose synthetic events derived from external or > 'ordinary' events easily and reuse windowed or other functions processed in > the > context of a window or other correlation/combination. > ...though not too sure about this. My external / raw events could be SNMP traps, lines from log-file or even be OA&M activity. I intend to convert them to a homogeneous form, which at-least identifies the event-type, some kind of hierarchical position (if possible), and other data. A good example of windowing in Erlang projects is slide.erl/spiraltime.erl > in riak core: > > https://github.com/basho/riak_core/blob/master/src/slide.erl > > https://github.com/basho/riak_core/blob/master/src/spiraltime.erl > > Event processing engines such as the open source Esper engine or commercial > StreamBase (disclaimer: I work there) provide documentation online that > could also be good references. > Thanks for all the tips. I am sure I'd be spending sometime going through some of that. I've gone thru the introduction to Esper -- definitely like what I see, although I am not quite sure (yet), if I really need all of that. On 12 Jun 2011, at 16:38, Banibrata Dutta wrote: > gr8 questions, and they certainly need clarification. > cc'ing the group s.t. others could contribute too. > > On Sun, Jun 12, 2011 at 8:48 PM, Mihai Balea wrote: > >> >> On Jun 12, 2011, at 10:51 AM, Banibrata Dutta wrote: >> >> Prematurely sent. >> >> On Sun, Jun 12, 2011 at 7:59 PM, Banibrata Dutta < >> banibrata.dutta@REDACTED> wrote: >> >>> >>> What would be a good way to correlate asynchronous events, spot patterns >>> over a sliding window (s.a. of no. of events elapsed or time elapsed), with >>> millions of events occurring simultaneously, using Erlang ? >>> >>> The set of possible events is known, and any unknown event is just >>> flagged as 'unknown' (so all unknowns are similar). The set of possible >>> event patterns can be enumerated, but is possibly quite a large set of >>> patterns. >>> >> >> Was wondering as to what could be the approach taken to implement such a >> thing in pure Erlang. My initial thoughts were along the line of maintaining >> FSMs per event source, but with so many events and so many possible/valid >> patterns, the thing seems kind of unwieldy. Also, I'd like a non-programmer >> to be able to define new events and valid event patterns. >> >> I believe 'Complex Event Processing' is quite likely to be the standard >> approach for such things, as I've found from some posts, and solutions exist >> in Java world for same, but both as an academic exercise (for the fun of >> learning) and for a potentially simpler + better solution, would like to try >> doing this is Erlang. >> >> >> I think you need to define your problem better. >> > > Sure, let me try. > > >> What exactly do you mean by "millions of events occurring simultaneously"? >> >> > > Okay, so I can say something like 500 events/second handled for correlation > would be a more realistic number. > > >> At exactly the same time? >> > > Yes... some of the events might be from same source, but spaced by as > little as 50ms, but mostly from different sources. There could be some > heirarchical relationship between sources. Very typical case of network > management scenario. E.g. a fault port on a switch, could probably cause > hundreds of destination unreachable events, application response timeouts, > heartbeat losses etc.. > > >> Millions of events per second? Minute? Is that peak rate, average rate or >> minimum rate? >> > > Okay, I got over-enthusiastic :-) . Say 100 events/second typical, 500 > events/second peak, no real minimum. > > What exactly is a pattern? >> > > Node-A failed, Power in room-X where Node-A is kept failed, Nodes B,C,D > which are served thru Node-A became unreachable, due to which Services L & M > became unavailable, and due to which another dependent service N started > giving inconsistent answers. So this is a pattern. However in this case, > there's a possibility that Power-failure had nothing to do with Noda-A's > failure, as backup power was available. > > Another pattern is, Power in room X failed, then Noda A failed, leading to > failure of only Node D, because somehow Nodes B & C were dynamically > configured to reroute. This is another pattern. > > What do you mean by "quite a large set of patterns"? Hundreds, thousands, >> millions? >> > > Several hundreds is a distinct possibility, and thousands are not > impossible, but millions -- probably not. > > >> How long is that sliding window? >> > > From few minutes (for certain type of events), to few days (for another > type of events). > > >> Can patterns encompass events coming from multiple sources or just one >> source? >> > > Yes, indeed. However in this case, there needs to a "relationship" between > the event sources, that is pre-defined. E.g. some sense of "topology" > exists. However it is likely that only 2% of the event sources are > interrelated. > > >> Are patterns concerned only with event ordering and occurrence or there >> are timing issues involved as well? >> > > Ordering, Timing, or any kind of causal relationship. > > -- regards, Banibrata http://www.linkedin.com/in/bdutta http://twitter.com/edgeliving -------------- next part -------------- An HTML attachment was scrubbed... URL: From dizzyd@REDACTED Mon Jun 13 15:56:59 2011 From: dizzyd@REDACTED (Dave Smith) Date: Mon, 13 Jun 2011 07:56:59 -0600 Subject: [erlang-questions] rebar escriptize and shared libraries In-Reply-To: <716930ED-6676-4035-ACAC-BDF934A45F89@gmail.com> References: <716930ED-6676-4035-ACAC-BDF934A45F89@gmail.com> Message-ID: Unfortunately, you can't embed a .so into an escript. Most dynamic loading APIs require the .so to be present as a file on disk, versus via memory. D. On Mon, Jun 13, 2011 at 5:55 AM, Joel Reymont wrote: > Given this rebar config: > > {lib_dirs,["deps"]}. > > {deps, [ > ?{'getopt', ".*", {git, "git://github.com/jcomellas/getopt.git", "HEAD"}}, > ?{'json', ".*", {git, "git://github.com/davisp/eep0018.git", "HEAD"}}, > ?{'lhttpc', ".*", {git, "git://github.com/esl/lhttpc.git", "HEAD"}} > ]}. > > {escript_incl_apps, [getopt, json, lhttpc]}. > > How do I tell rebar to include deps/json/prive/json.so? > > It doesn't seem to be doing it because I get an error > > =ERROR REPORT==== 13-Jun-2011::15:49:20 === > The on_load function for module json returned {error, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {load_failed, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"Failed to load NIF library: 'dlopen(xxx/json/priv/json.so, 2): no suitable image found. ?Did find:\n\txxx/json/priv/json.so: stat() failed with errno=20'"}} > > ? ? ? ?Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From banibrata.dutta@REDACTED Mon Jun 13 16:01:12 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Mon, 13 Jun 2011 19:31:12 +0530 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> Message-ID: On Sun, Jun 12, 2011 at 11:24 PM, Edmond Begumisa < ebegumisa@REDACTED> wrote: > My interpretation of your question is that you want to do one-off message > tracing. > Um, not too sure. What I want to do is to identify event-patterns in real time, and take some action. For example, I am processing event-A from source S1. At this time I have no context so I will probably do some minimal work and "remember" occurrence of event-A (i.e. update context). Then event-B might arrive (also from S1), and in this case . > > 1. You want to trace particular massages passed in your system over a > period Y, you are particularly interested in 'EXIT' messages, but not > necessarily: > > You could use the trace BIFs/dbg tool for this. Start an appropriate tracer > with dbg:tracer, specify which processes you want to trace with dbg:p with > one of the m|s|r flags to log the events, watch your system for a while, > turn the tracer off (keep in mind the negative impact such tracing might > have on a live system.) Analyse the trace output. You could also use your > own tracer handler fun to filter/process the messages. > My usage is not naturally process-tracing, but if I am able to draw a parallel or use the concept, happy to explore. So far, I am not able to imagine the mapping of what I'd like to do with this approach. If there is some example to illustrate, it'd help. > 2. You want to trace massages over a number of nodes: > > Though I haven't used it, I believe this is what the Inviso OTP tool is > for. You could also have a look at the ttb OTP tool. > > 3. You want to trace "cascading" messages in your system: > > Have a look at erlang:seq_trace. I believe dbg and ttb both have support > for sequential tracing (I don't think Inviso does.) > > 4. You want a convenient way of walking through traces: > > The OTP et_viewer is a really nice tool I use a lot. Though the OTP Event > Tracer can output ordinary trace data produced by dbg, where it really gets > fun is when you feed it trace data by calling et:trace_me in your code (I > normally wrap this in a ?TRACE macro). I like to think of it as a > replacement for using io:format to print debug messages. The "detail" > argument is really nice for "zooming" in/out as you page through trace data. > You might find that useful for analysing a large number of messages. > > In short, I'm not sure you need any special "architectural design > patterns". Just design your system as you normally would, using processes to > model what you observe in the world, then trace it when it misbehaves or > when you want to observe it. The traces *are* your sliding windows... unless > I've completely mis-understood your question! > > - Edmond - > > > On Mon, 13 Jun 2011 01:38:20 +1000, Banibrata Dutta < > banibrata.dutta@REDACTED> wrote: > > gr8 questions, and they certainly need clarification. >> cc'ing the group s.t. others could contribute too. >> >> On Sun, Jun 12, 2011 at 8:48 PM, Mihai Balea wrote: >> >> >>> On Jun 12, 2011, at 10:51 AM, Banibrata Dutta wrote: >>> >>> Prematurely sent. >>> >>> On Sun, Jun 12, 2011 at 7:59 PM, Banibrata Dutta < >>> banibrata.dutta@REDACTED> wrote: >>> >>> >>>> What would be a good way to correlate asynchronous events, spot patterns >>>> over a sliding window (s.a. of no. of events elapsed or time elapsed), >>>> with >>>> millions of events occurring simultaneously, using Erlang ? >>>> >>>> The set of possible events is known, and any unknown event is just >>>> flagged >>>> as 'unknown' (so all unknowns are similar). The set of possible event >>>> patterns can be enumerated, but is possibly quite a large set of >>>> patterns. >>>> >>>> >>> Was wondering as to what could be the approach taken to implement such a >>> thing in pure Erlang. My initial thoughts were along the line of >>> maintaining >>> FSMs per event source, but with so many events and so many possible/valid >>> patterns, the thing seems kind of unwieldy. Also, I'd like a >>> non-programmer >>> to be able to define new events and valid event patterns. >>> >>> I believe 'Complex Event Processing' is quite likely to be the standard >>> approach for such things, as I've found from some posts, and solutions >>> exist >>> in Java world for same, but both as an academic exercise (for the fun of >>> learning) and for a potentially simpler + better solution, would like to >>> try >>> doing this is Erlang. >>> >>> >>> I think you need to define your problem better. >>> >>> >> Sure, let me try. >> >> >> What exactly do you mean by "millions of events occurring >>> simultaneously"? >>> >>> >> Okay, so I can say something like 500 events/second handled for >> correlation >> would be a more realistic number. >> >> >> At exactly the same time? >>> >>> >> Yes... some of the events might be from same source, but spaced by as >> little >> as 50ms, but mostly from different sources. There could be some >> heirarchical >> relationship between sources. Very typical case of network management >> scenario. E.g. a fault port on a switch, could probably cause hundreds of >> destination unreachable events, application response timeouts, heartbeat >> losses etc.. >> >> >> Millions of events per second? Minute? Is that peak rate, average rate or >>> minimum rate? >>> >>> >> Okay, I got over-enthusiastic :-) . Say 100 events/second typical, 500 >> events/second peak, no real minimum. >> >> What exactly is a pattern? >> >>> >>> >> Node-A failed, Power in room-X where Node-A is kept failed, Nodes B,C,D >> which are served thru Node-A became unreachable, due to which Services L & >> M >> became unavailable, and due to which another dependent service N started >> giving inconsistent answers. So this is a pattern. However in this case, >> there's a possibility that Power-failure had nothing to do with Noda-A's >> failure, as backup power was available. >> >> Another pattern is, Power in room X failed, then Noda A failed, leading to >> failure of only Node D, because somehow Nodes B & C were dynamically >> configured to reroute. This is another pattern. >> >> What do you mean by "quite a large set of patterns"? Hundreds, thousands, >> >>> millions? >>> >>> >> Several hundreds is a distinct possibility, and thousands are not >> impossible, but millions -- probably not. >> >> >> How long is that sliding window? >>> >>> >> From few minutes (for certain type of events), to few days (for another >> type >> of events). >> >> >> Can patterns encompass events coming from multiple sources or just one >>> source? >>> >>> >> Yes, indeed. However in this case, there needs to a "relationship" between >> the event sources, that is pre-defined. E.g. some sense of "topology" >> exists. However it is likely that only 2% of the event sources are >> interrelated. >> >> >> Are patterns concerned only with event ordering and occurrence or there >>> are >>> timing issues involved as well? >>> >>> >> Ordering, Timing, or any kind of causal relationship. >> > > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > -- regards, Banibrata http://www.linkedin.com/in/bdutta http://twitter.com/edgeliving -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Mon Jun 13 18:04:37 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 13 Jun 2011 20:04:37 +0400 Subject: [erlang-questions] formatting timestamps Message-ID: <4FC73666-F79C-440D-9CFA-3967D5E07DBE@gmail.com> How do you format erlang:now() as YYYY-MM-DD HH:MM:SS.ffffff ? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From qrilka@REDACTED Mon Jun 13 18:08:06 2011 From: qrilka@REDACTED (Kirill Zaborsky) Date: Mon, 13 Jun 2011 20:08:06 +0400 Subject: [erlang-questions] formatting timestamps In-Reply-To: <4FC73666-F79C-440D-9CFA-3967D5E07DBE@gmail.com> References: <4FC73666-F79C-440D-9CFA-3967D5E07DBE@gmail.com> Message-ID: Maybe https://github.com/daleharvey/dh_date will do the trick? Kind regards, Kirill Zaborsky 2011/6/13 Joel Reymont > How do you format erlang:now() as YYYY-MM-DD HH:MM:SS.ffffff ? > > Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Mon Jun 13 18:30:11 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 13 Jun 2011 20:30:11 +0400 Subject: [erlang-questions] formatting timestamps In-Reply-To: References: <4FC73666-F79C-440D-9CFA-3967D5E07DBE@gmail.com> Message-ID: On Jun 13, 2011, at 8:08 PM, Kirill Zaborsky wrote: > Maybe https://github.com/daleharvey/dh_date will do the trick? I don't think so, e.g. %% This module formats erlang dates in the form %% {{Year, Month, Day}, {Hour, Minute, Second}} -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From bob@REDACTED Mon Jun 13 18:39:03 2011 From: bob@REDACTED (Bob Ippolito) Date: Mon, 13 Jun 2011 09:39:03 -0700 Subject: [erlang-questions] formatting timestamps In-Reply-To: References: <4FC73666-F79C-440D-9CFA-3967D5E07DBE@gmail.com> Message-ID: On Mon, Jun 13, 2011 at 9:30 AM, Joel Reymont wrote: > > On Jun 13, 2011, at 8:08 PM, Kirill Zaborsky wrote: > >> Maybe https://github.com/daleharvey/dh_date will do the trick? > > I don't think so, e.g. > > %% This module formats erlang dates in the form > %% {{Year, Month, Day}, {Hour, Minute, Second}} http://www.erlang.org/doc/man/calendar.html#now_to_universal_time-1 -bob From kennethstone@REDACTED Mon Jun 13 18:46:26 2011 From: kennethstone@REDACTED (Kenny Stone) Date: Mon, 13 Jun 2011 11:46:26 -0500 Subject: [erlang-questions] formatting timestamps In-Reply-To: References: <4FC73666-F79C-440D-9CFA-3967D5E07DBE@gmail.com> Message-ID: Date times are always frustrating in erlang. I think this will get you all the data you need... Now = now(). {{Year,Mon,Day},{Hour,Min,Sec}} = calendar:now_to_universal_time(Now). {_,_,MicroSec} = Now. Then you wrangle it into a string with io_lib: http://erlang.org/pipermail/erlang-questions/2010-October/054132.html ugly... maybe there's a better way On Mon, Jun 13, 2011 at 11:30 AM, Joel Reymont wrote: > > On Jun 13, 2011, at 8:08 PM, Kirill Zaborsky wrote: > > > Maybe https://github.com/daleharvey/dh_date will do the trick? > > I don't think so, e.g. > > %% This module formats erlang dates in the form > %% {{Year, Month, Day}, {Hour, Minute, Second}} > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Mon Jun 13 18:46:47 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 13 Jun 2011 20:46:47 +0400 Subject: [erlang-questions] formatting timestamps In-Reply-To: References: <4FC73666-F79C-440D-9CFA-3967D5E07DBE@gmail.com> Message-ID: <94A5AF9F-48C5-4537-9315-DFA19FDD01ED@gmail.com> On Jun 13, 2011, at 8:39 PM, Bob Ippolito wrote: > http://www.erlang.org/doc/man/calendar.html#now_to_universal_time-1 This does the trick, courtesy of Filipe Manana: timestamp(Now) -> {_, _, Micros} = Now, {{YY, MM, DD}, {Hour, Min, Sec}} = calendar:now_to_local_time(Now), io_lib:format("~4..0w-~2..0w-~2..0w ~2..0w:~2..0w:~2..0w.~p", [YY, MM, DD, Hour, Min, Sec, Micros]). -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From silent_vendetta@REDACTED Mon Jun 13 18:47:57 2011 From: silent_vendetta@REDACTED (Chris Hicks) Date: Mon, 13 Jun 2011 09:47:57 -0700 Subject: [erlang-questions] formatting timestamps In-Reply-To: References: <4FC73666-F79C-440D-9CFA-3967D5E07DBE@gmail.com>, , Message-ID: You're just going to have to do it yourself: {{Year, Month, Day}, {Hour, Minute, Second}} = erlang:universaltime(), %% Or erlang:localtime()Y = erlang:integer_to_list(Year),...Formatted_date = lists:concat([Y, "-", Mo, "-", D, " ", H, ":", Mi, ":", S, ".", Whatever_else_here]),... Not saying that is the most efficient way to do it but it would get you what you are looking for. Chris Hicks. > From: joelr1@REDACTED > Date: Mon, 13 Jun 2011 20:30:11 +0400 > To: qrilka@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] formatting timestamps > > > On Jun 13, 2011, at 8:08 PM, Kirill Zaborsky wrote: > > > Maybe https://github.com/daleharvey/dh_date will do the trick? > > I don't think so, e.g. > > %% This module formats erlang dates in the form > %% {{Year, Month, Day}, {Hour, Minute, Second}} > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From antoine.koener@REDACTED Mon Jun 13 18:49:57 2011 From: antoine.koener@REDACTED (Antoine Koener) Date: Mon, 13 Jun 2011 18:49:57 +0200 Subject: [erlang-questions] formatting timestamps In-Reply-To: References: <4FC73666-F79C-440D-9CFA-3967D5E07DBE@gmail.com> Message-ID: On Mon, Jun 13, 2011 at 6:47 PM, Chris Hicks wrote: > You're just going to have to do it yourself: > {{Year, Month, Day}, {Hour, Minute, Second}} = erlang:universaltime(), %% Or > erlang:localtime() > Y = erlang:integer_to_list(Year), > ... > Formatted_date = lists:concat([Y, "-", Mo, "-", D, " ", H, ":", Mi, ":", S, > ".", Whatever_else_here]), > ... > Not saying that is the most efficient way to do it but it would get you what > you are looking for. > Chris Hicks. {_, _, Usec} = Now = erlang:now(), {{Y, Mo, D}, {H, Mi, S}} = calendar:now_to_datetime(Now), Stamp = io_lib:format("~4.4.0w-~2.2.0w-~2.2.0wT~2.2.0w:~2.2.0w:~2.2.0w.~6.6.0w ", [ Y, Mo, D, H, Mi, S, Usec ]), ... Log = iolist_to_binary( [ Stamp, [ [ <<" ">>, X, <<" ">> ] || X <- [ Login ] ], io_lib:format(Format, lists:map( fun filter/1, Args)), Exe, <<"\n">> ]), Extracted from old code. > >> From: joelr1@REDACTED >> Date: Mon, 13 Jun 2011 20:30:11 +0400 >> To: qrilka@REDACTED >> CC: erlang-questions@REDACTED >> Subject: Re: [erlang-questions] formatting timestamps >> >> >> On Jun 13, 2011, at 8:08 PM, Kirill Zaborsky wrote: >> >> > Maybe https://github.com/daleharvey/dh_date will do the trick? >> >> I don't think so, e.g. >> >> %% This module formats erlang dates in the form >> %% {{Year, Month, Day}, {Hour, Minute, Second}} >> >> -------------------------------------------------------------------------- >> - for hire: mac osx device driver ninja, kernel extensions and usb drivers >> ---------------------+------------+--------------------------------------- >> http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont >> ---------------------+------------+--------------------------------------- >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From mihai@REDACTED Mon Jun 13 22:13:56 2011 From: mihai@REDACTED (Mihai Balea) Date: Mon, 13 Jun 2011 16:13:56 -0400 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> <6289B011-5222-418A-A178-34DB304A44CE@hates.ms> Message-ID: <4C1F8827-E9F6-4677-BA54-BAA7D1C34474@hates.ms> On Jun 13, 2011, at 9:12 AM, Banibrata Dutta wrote: > > Well, I haven't. In fact, starting with a small set of possible patterns, seems to be the most intuitive approach. When I bring in the thought of permitting non-programmers to write "something" (e.g. natural language rules, or a simple DSL snippet), and then such a thing modifying the FSM, without me having to code anything further in Erlang, is when I am hitting a mental roadblock. I've coded fairly complex FSM's in C/C++ using the table-based approach. Does gen_fsm take a similar approach ? Will it be a good, natural fit ? Maybe a rather naive question for people extremely familiar with Erlang, but I'm still very much on the learning curve's positive vector. Erlang gen_fsms are quite simple and standard, they define a bunch of states and use pattern matching to select actions for each state/event combination. Pretty much equivalent to a table-based fsm, only there's no distinct table to speak of. > > however you might want to make them a bit more flexible. Maybe have a bunch of processes running pattern recognizers, say one process per pattern or class of patterns. > > My natural tendency was to think more in terms of 1 process per source, e.g. 1st event from a source causes an Erlang process to be created, which then embodies the entire possible FSM. The main issue with pattern based processes (if I understood it correctly), is determining the right process to hand-off the event to, and once handed-off, backtracking would mean extra work. > > Maybe you can filter events based on source so certain recognizers will only get certain events. > > Yes, that was the starting point, as I had imagined. Event forwarding based on source is technically easy in Erlang, just use a process registry (proc, grpoc, even the built in process registrar) and use sources as keys. The problem with this approach is you might have at least some correlation between sources. Events originating from different sources but belonging to the same pattern (i.e. server going black due to switch failing in some other room, due to power outage and the like). > > If you have a low correlation between event sources maybe you can even design your system to distribute the processing of unrelated events to different nodes in a cluster (big scalability gain if you can do that). > > Excellent thought... indeed. Source based sharding. > > Another way of approaching this would be to duplicate the event stream to multiple nodes in your cluster and have each node only look for certain subsets of patterns. > > Actually, my bad, I should've mentioned, I wanted to do all this in Real-Time. So I don't really have a pattern-string, so to say. I don't, without that I can really take this approach. Or can I ? I think you can. If your patterns are independent, or at least can be grouped in independent categories, what would prevent you from assigning sets of independent patterns to different nodes? Not saying you need to do that, but it might be helpful if you find out that recognizing and processing thousands of patterns take up too much cpu. Mihai -------------- next part -------------- An HTML attachment was scrubbed... URL: From moxford@REDACTED Mon Jun 13 23:34:26 2011 From: moxford@REDACTED (Mike Oxford) Date: Mon, 13 Jun 2011 14:34:26 -0700 Subject: [erlang-questions] Gotta be an easier way... Message-ID: Updating nested records; keep all record (and sub-record!) information intact but change one value. A generic logon-type message, with nested records, this type of structure. v1_rec_system_message_user_logon v1_rec_header (header) value1 v1_rec_route [] (route) body Orig#v1_rec_system_message_user_logon{header=Orig#v1_rec_system_message_user_logon.header#v1_rec_header{route=Orig#v1_rec_system_message_user_logon.header#v1_rec_header.route#v1_rec_route{route_entries=[foo]}}}. Note how many times I have to fully qualify the record path to get the sub-records to take the correct values even though I'm only changing one value since every nested records seems to need to have a fully qualified path. Ugh. Shortening variable names, or writing accessor macros for everything are possibilities ... but is there a better way I'm missing? (Don't use nested records is a poor response.) Note that.... Orig#v1_rec_system_message_user_logon.header#v1_rec_header.route#v1_rec_route{route_entries=[foo]}. will give you #v1_rec_route{value1 = correct,route_entries = [foo]} but will not return the entire thing. Anyone have a better way...? Thanks! -mox From francis.joanis@REDACTED Tue Jun 14 01:01:48 2011 From: francis.joanis@REDACTED (Francis Joanis) Date: Mon, 13 Jun 2011 19:01:48 -0400 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <4DF4BDDA.30809@cs.ntua.gr> References: <4DF4BDDA.30809@cs.ntua.gr> Message-ID: Hi, Thanks a lot for the release :) I have a question about the GPL licensing of the tool itself: since it is GPL, doesn't it require to make the written tests themselves and to some extent the application under test GPL as well? I'm obviously not a lawyer, but I'd like to know whether it would be possible to test commercial code using PropEr given its GPL licensing? Thanks again, Francis On Sun, Jun 12, 2011 at 9:23 AM, Kostis Sagonas wrote: > We are happy to announce the first official public release of PropEr, a > QuickCheck-inspired Property-Based Testing Tool for Erlang. > > The release comes with a proper site containing a User Guide, tips and > tutorials for PropEr. It can be accessed at: > > http://proper.softlab.ntua.gr/ > > Among other things, it has a proper contact address on which we will be > very happy to receive comments and feedback. Depending on interest, we may > also set up a mailing list for PropEr users. > > Enjoy! > > Kostis Sagonas (on behalf of the PropEr developers) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vik@REDACTED Tue Jun 14 01:09:53 2011 From: vik@REDACTED (Vik Olliver) Date: Tue, 14 Jun 2011 11:09:53 +1200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> Message-ID: <4DF698C1.2060908@catalyst.net.nz> On 14/06/11 11:01, Francis Joanis wrote: > > I have a question about the GPL licensing of the tool itself: since it > is GPL, doesn't it require to make the written tests themselves and to > some extent the application under test GPL as well? Only if you distribute the tests. If you keep them to yourself, you don't owe anyone anything. The GPL respects privacy. Vik :v) From igorrs@REDACTED Tue Jun 14 07:55:52 2011 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Tue, 14 Jun 2011 02:55:52 -0300 Subject: [erlang-questions] "Killed" Mnesia transaction left locks behind Message-ID: An Erlang process in node N was killed while executing a transaction (our system does that automatically for some kinds of operations, when they take too long to finish). However, some locks were left behind in the 4 nodes that have the tables (this should be a bug, according to what Dan Gudmundsson once told me in this list). Anyway, I'd like to know if there's a way of releasing the locks without restarting the nodes. I tried mnesia_locker:release_tid(27211023009), but the locks still appear when I call mnesia:system_info(held_locks): - In nodes N1 and N2: [{{si_build_hq_frag31,7}, write, {tid,27211023009,<8057.12454.8>}}] - In nodes N3 and N4: [{{si_build_hq_frag191,{7,1}}, write, {tid,27211023009,<8057.12454.8>}}] - Also in node N3: [{{si_build_hq_frag191,{7,1}}, read, {tid,27211023009,<8057.12454.8>}}, I'm sure the process is no longer running in node N. Thank you. Igor. -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From joelr1@REDACTED Tue Jun 14 09:12:18 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 14 Jun 2011 11:12:18 +0400 Subject: [erlang-questions] ocaml vs erlang Message-ID: I used OCaml for a few months to build a distributed production system. The architecture included Mongrel2, ZeroMQ, Redis and Amazon AWS. Using OCaml in this scenario is like pushing a boulder uphill, I should have used Erlang! -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From torben.lehoff@REDACTED Tue Jun 14 09:44:24 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Tue, 14 Jun 2011 09:44:24 +0200 Subject: [erlang-questions] ocaml vs erlang In-Reply-To: References: Message-ID: Hi Joel, Instead of resorting to the "I could have told you so" reply I would like to hear if you could explain where the pain is - I think that will be an interesting read... as it normally is when you write up a story! Cheers, Torben On Tue, Jun 14, 2011 at 09:12, Joel Reymont wrote: > I used OCaml for a few months to build a distributed production system. > > The architecture included Mongrel2, ZeroMQ, Redis and Amazon AWS. > > Using OCaml in this scenario is like pushing a boulder uphill, I should > have used Erlang! > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From Mike.French@REDACTED Tue Jun 14 11:14:51 2011 From: Mike.French@REDACTED (French, Mike) Date: Tue, 14 Jun 2011 10:14:51 +0100 Subject: [erlang-questions] How to do this in Erlang -- any suggestion s ? Message-ID: <3F8EEA01CF53D74DB4A9EC314D82B4F3037A6783@wells154924> _____ From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Darach Ennis Sent: 12 June 2011 17:47 To: Banibrata Dutta Cc: erlang-questions Subject: Re: [erlang-questions] How to do this in Erlang -- any suggestions ? Hi guys, An excellent book introducing event processing is Opher Etzion's Event Processing in action: http://www.manning.com/etzion/ Caveat Lector A quick word of warning that Etzion (IBM) has patents on some of this material. I know you can't patent algorithms and software (yeah, right) but IBM probably has more lawyers than you. Mike Thales UK Ltd (Wells) DISCLAIMER: The information contained in this e-mail is confidential. It may also be legally privileged. It is intended only for the stated addressee(s) and access to it by any other person is unauthorised. If you are not an addressee, you must not disclose, copy, circulate or in any other way use or rely on the information contained in this e-mail. Such unauthorised use may be unlawful. We may monitor all e-mail communications through our networks. If you have received this e-mail in error, please inform us immediately on sender's telephone number above and delete it and all copies from your system. We accept no responsibility for changes to any e-mail which occur after it has been sent. Attachments to this e-mail may contain software viruses which could damage your system. We therefore recommend you virus-check all attachments before opening. Thales UK Ltd. Registered Office: 2 Dashwood Lang Road, The Bourne Business Park, Addlestone, Weybridge, Surrey KT15 2NX Registered in England No. 868273 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hyerle@REDACTED Tue Jun 14 11:16:59 2011 From: hyerle@REDACTED (robert) Date: Tue, 14 Jun 2011 02:16:59 -0700 (PDT) Subject: [erlang-questions] Integrating Erlang and OpenSAF In-Reply-To: References: Message-ID: <1308043019955-3596050.post@n4.nabble.com> Note: this is a repost as my original attempt never made it. Sorry for any confusion. -- robert I'm pretty much the Java editor for SAF and have some visibility into alternative language mappings for SAF. Right now, the only official SAF mappings are C and Java (with a higher level Java interface standardized via the JCP). There is some work going on in OpenSAF with a scripting language (Perl? Python?) ('cause it's easy and was sort of a side-effect of test frameworks, as I understand). I don't know if Erlang interfaces exist in certain companies, even as Ericsson is a major participant in SAF and OpenSAF. Maybe someone from Ericsson will speak up? I've been thinking about using Erlang/Mnesia for implementing parts of SAF: the IMM service. Still in the thinking stage. It would also be interesting to consider the correspondence between OTP and the SAF Availability Management Framework. Certainly it is possible to integrate Erlang with SAF .. it's just software ;-) Erlang can certainly handle the "escapes" to native transports. Moreover, SAF is very much built on a location transparent, client/servers w/callbacks model which fits well with Erlang/OTP. For the concrete case of OpenSAF (an open source implementation of the SAF standards), the Java mapping was done with JNI wrappers. Not sure how this approach would play with Erlang. It is seductive to think that API's can wrapped and integrated into a single process while preserving semantics, behavior, and safety. I'd personally opt for a well defined message protocol, but the SAF standardization is based on an API. Frank (Usenet) wrote: > > Hi, > > I've been trying to get up to speed with the SAF (Service Availability > Framework - HA middleware) implementation OpenSAF > (http://www.opensaf.org/) > and I was wondering if there were ways to integrate Erlang with it? > > They already seem to provide APIs for Java and C. Since Ericsson seems to > be > backing both OpenSAF and Erlang ( > http://www.ericsson.com/article/open_source_1714451213_c), an Erlang > implementation might make sense as well :) > > > P.S. I've been trying to send this email through another email account and > it didn't seem to make it to the list. If it does make it in the end then > I'm sorry for the multiple posts... > > > Thanks, > Frank > > -- View this message in context: http://erlang.2086793.n4.nabble.com/Integrating-Erlang-and-OpenSAF-tp3509589p3596050.html Sent from the Erlang Questions mailing list archive at Nabble.com. From joelr1@REDACTED Tue Jun 14 12:05:32 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 14 Jun 2011 14:05:32 +0400 Subject: [erlang-questions] ocaml vs erlang In-Reply-To: References: Message-ID: <6F995EAB-A74F-47ED-961B-38EF6F34D646@gmail.com> On Jun 14, 2011, at 11:44 AM, Torben Hoffmann wrote: > Instead of resorting to the "I could have told you so" reply I would like to hear if you could explain where the pain is I could have told myself "I could have told you so" but it was interesting to try OCaml. I'll write up a blog post, haven't done that in a while. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From torben.lehoff@REDACTED Tue Jun 14 12:35:54 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Tue, 14 Jun 2011 12:35:54 +0200 Subject: [erlang-questions] ocaml vs erlang In-Reply-To: <6F995EAB-A74F-47ED-961B-38EF6F34D646@gmail.com> References: <6F995EAB-A74F-47ED-961B-38EF6F34D646@gmail.com> Message-ID: I - for one - is looking forward to that! On Tue, Jun 14, 2011 at 12:05, Joel Reymont wrote: > > On Jun 14, 2011, at 11:44 AM, Torben Hoffmann wrote: > > > Instead of resorting to the "I could have told you so" reply I would like > to hear if you could explain where the pain is > > I could have told myself "I could have told you so" but it was interesting > to try OCaml. > > I'll write up a blog post, haven't done that in a while. > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From richardprideauxevans@REDACTED Tue Jun 14 13:13:36 2011 From: richardprideauxevans@REDACTED (Richard Evans) Date: Tue, 14 Jun 2011 12:13:36 +0100 Subject: [erlang-questions] Debugging c code when interoperating with erlang Message-ID: I have a c program which interoperates with erlang, following the guidelines described in Joe Armstrong's book and in the erlang docs: http://www.erlang.org/doc/tutorial/c_port.html#id56436 Everything works nicely, but I sometimes have problems debugging the c code. Specifically: if I attach the debugger to the c process, with breakpoints *already* added, then I can debug the c code fine. But if I attach the debugger to the c process, and *then* add extra breakpoints, this causes the erlang calling code to break. It gives the error: ** exception error: bad argument in function server:call_port/1 The call_port function is the standard one: call_port(Msg) -> server ! {call, self(), Msg}, receive {server, Result} -> Result end. Do you guys know why this would happen? (I am using Xcode 4.0, Mac OSX 10.6, Erlang R14B01). A further question: if my c code crashes, the erlang node hangs; I have to run again to reproduce the problem. Is it possible to get the erlang node to print out a dump or anything? thanks, Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From francis.joanis@REDACTED Tue Jun 14 13:19:03 2011 From: francis.joanis@REDACTED (Francis Joanis) Date: Tue, 14 Jun 2011 07:19:03 -0400 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <4DF698C1.2060908@catalyst.net.nz> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> Message-ID: On Mon, Jun 13, 2011 at 7:09 PM, Vik Olliver wrote: > > Only if you distribute the tests. If you keep them to yourself, you > don't owe anyone anything. The GPL respects privacy. > Thanks, that makes sense. Francis From ulf.wiger@REDACTED Tue Jun 14 14:02:40 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 14 Jun 2011 14:02:40 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <4DF698C1.2060908@catalyst.net.nz> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> Message-ID: <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> On 14 Jun 2011, at 01:09, Vik Olliver wrote: > On 14/06/11 11:01, Francis Joanis wrote: >> >> I have a question about the GPL licensing of the tool itself: since it >> is GPL, doesn't it require to make the written tests themselves and to >> some extent the application under test GPL as well? > > Only if you distribute the tests. If you keep them to yourself, you > don't owe anyone anything. The GPL respects privacy. This could become an obstacle to adoption similar to what's been seen with Quviq QuickCheck: people who have Open Source projects need to publish (=distribute) test suites as well. Just as it's not very helpful to distribute test suites that require an expensive tool, people might be reluctant to publish PropEr-based tests if it requires them to convert to GPL. Kostis did say that they would make an exception for OSS projects. I could not find any such exception when quickly scanning the repos. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From zabrane3@REDACTED Tue Jun 14 14:15:44 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 14 Jun 2011 14:15:44 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> Message-ID: <97E89C26-DEE9-45B5-9D06-5C3FBBB29373@gmail.com> +1 to try to convince Kostis to release PropEr with a better license! Please vote! Regards, Zabrane Le 14 juin 2011 ? 14:02, Ulf Wiger a ?crit : > On 14 Jun 2011, at 01:09, Vik Olliver wrote: > >> On 14/06/11 11:01, Francis Joanis wrote: >>> >>> I have a question about the GPL licensing of the tool itself: since it >>> is GPL, doesn't it require to make the written tests themselves and to >>> some extent the application under test GPL as well? >> >> Only if you distribute the tests. If you keep them to yourself, you >> don't owe anyone anything. The GPL respects privacy. > > > This could become an obstacle to adoption similar to what's been seen with Quviq QuickCheck: people who have Open Source projects need to publish (=distribute) test suites as well. Just as it's not very helpful to distribute test suites that require an expensive tool, people might be reluctant to publish PropEr-based tests if it requires them to convert to GPL. > > Kostis did say that they would make an exception for OSS projects. I could not find any such exception when quickly scanning the repos. > > BR, > Ulf W > From carlsson.richard@REDACTED Tue Jun 14 14:25:52 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 14 Jun 2011 14:25:52 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> Message-ID: <4DF75350.7020208@gmail.com> On 06/14/2011 02:02 PM, Ulf Wiger wrote: > On 14 Jun 2011, at 01:09, Vik Olliver wrote: >> On 14/06/11 11:01, Francis Joanis wrote: >>> I have a question about the GPL licensing of the tool itself: >>> since it is GPL, doesn't it require to make the written tests >>> themselves and to some extent the application under test GPL as >>> well? >> >> Only if you distribute the tests. If you keep them to yourself, >> you don't owe anyone anything. The GPL respects privacy. > > This could become an obstacle to adoption similar to what's been seen > with Quviq QuickCheck: people who have Open Source projects need to > publish (=distribute) test suites as well. Just as it's not very > helpful to distribute test suites that require an expensive tool, > people might be reluctant to publish PropEr-based tests if it > requires them to convert to GPL. > > Kostis did say that they would make an exception for OSS projects. I > could not find any such exception when quickly scanning the repos. > > BR, Ulf W As I have remarked before on this list, I believe the normal GPL (all versions) cannot be used for Erlang modules, because the dynamic loading and linking of classes/modules at runtime is not considered to be any different from static linking - at least according to the FSF. The standard Erlang library modules and the runtime system BIFs are published under the Erlang Public License, which is an even more restrictive variant of MPL. The MPL is not compatible with GPL (http://en.wikipedia.org/wiki/Mozilla_Public_License#Compatibility_with_GPL), due to some of its restrictions, so it should follow that the EPL is also not compatible. If you want to enforce copyleft in your code, but not force it on other libraries that link with your code, you can use LGPL instead of GPL. (Preferably, use v3, which is compatible with Apache License 2.) http://www.gnu.org/licenses/lgpl-java.html /Richard (not a lawyer) From ericbmerritt@REDACTED Tue Jun 14 14:33:47 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 14 Jun 2011 07:33:47 -0500 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <4DF75350.7020208@gmail.com> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> Message-ID: I talked with Kostis about this at the erlang factory. He said he is going to provide the exception to the gpl for open source projects. I think he just has not had time to do the actual work. On Jun 14, 2011 7:26 AM, "Richard Carlsson" wrote: > On 06/14/2011 02:02 PM, Ulf Wiger wrote: >> On 14 Jun 2011, at 01:09, Vik Olliver wrote: >>> On 14/06/11 11:01, Francis Joanis wrote: >>>> I have a question about the GPL licensing of the tool itself: >>>> since it is GPL, doesn't it require to make the written tests >>>> themselves and to some extent the application under test GPL as >>>> well? >>> >>> Only if you distribute the tests. If you keep them to yourself, >>> you don't owe anyone anything. The GPL respects privacy. >> >> This could become an obstacle to adoption similar to what's been seen >> with Quviq QuickCheck: people who have Open Source projects need to >> publish (=distribute) test suites as well. Just as it's not very >> helpful to distribute test suites that require an expensive tool, >> people might be reluctant to publish PropEr-based tests if it >> requires them to convert to GPL. >> >> Kostis did say that they would make an exception for OSS projects. I >> could not find any such exception when quickly scanning the repos. >> >> BR, Ulf W > > As I have remarked before on this list, I believe the normal GPL (all > versions) cannot be used for Erlang modules, because the dynamic loading > and linking of classes/modules at runtime is not considered to be any > different from static linking - at least according to the FSF. > > The standard Erlang library modules and the runtime system BIFs are > published under the Erlang Public License, which is an even more > restrictive variant of MPL. The MPL is not compatible with GPL > ( http://en.wikipedia.org/wiki/Mozilla_Public_License#Compatibility_with_GPL), > due to some of its restrictions, so it should follow that the EPL is > also not compatible. > > If you want to enforce copyleft in your code, but not force it on other > libraries that link with your code, you can use LGPL instead of GPL. > (Preferably, use v3, which is compatible with Apache License 2.) > > http://www.gnu.org/licenses/lgpl-java.html > > /Richard (not a lawyer) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jameschurchman@REDACTED Tue Jun 14 16:13:33 2011 From: jameschurchman@REDACTED (James Churchman) Date: Tue, 14 Jun 2011 15:13:33 +0100 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> Message-ID: <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> Could the lgpl not be chosen instead? Would this not allow the tests to be distributed along as they don't ship a custom version of propEr as somebody else states? Also is it really the case that the tests could not be distributed, or more a case that it is totally fine to distribute the tests along as the person receiving the tests downloads proper themselves? the code that is not tests would still run fine. Also what was the original reasoning behind choosing the gpl in this case? Regards James also +1 to try to convince Kostis to release PropEr with a better license! On 14 Jun 2011, at 13:33, Eric Merritt wrote: > I talked with Kostis about this at the erlang factory. He said he is going to provide the exception to the gpl for open source projects. I think he just has not had time to do the actual work. > > On Jun 14, 2011 7:26 AM, "Richard Carlsson" wrote: > > On 06/14/2011 02:02 PM, Ulf Wiger wrote: > >> On 14 Jun 2011, at 01:09, Vik Olliver wrote: > >>> On 14/06/11 11:01, Francis Joanis wrote: > >>>> I have a question about the GPL licensing of the tool itself: > >>>> since it is GPL, doesn't it require to make the written tests > >>>> themselves and to some extent the application under test GPL as > >>>> well? > >>> > >>> Only if you distribute the tests. If you keep them to yourself, > >>> you don't owe anyone anything. The GPL respects privacy. > >> > >> This could become an obstacle to adoption similar to what's been seen > >> with Quviq QuickCheck: people who have Open Source projects need to > >> publish (=distribute) test suites as well. Just as it's not very > >> helpful to distribute test suites that require an expensive tool, > >> people might be reluctant to publish PropEr-based tests if it > >> requires them to convert to GPL. > >> > >> Kostis did say that they would make an exception for OSS projects. I > >> could not find any such exception when quickly scanning the repos. > >> > >> BR, Ulf W > > > > As I have remarked before on this list, I believe the normal GPL (all > > versions) cannot be used for Erlang modules, because the dynamic loading > > and linking of classes/modules at runtime is not considered to be any > > different from static linking - at least according to the FSF. > > > > The standard Erlang library modules and the runtime system BIFs are > > published under the Erlang Public License, which is an even more > > restrictive variant of MPL. The MPL is not compatible with GPL > > (http://en.wikipedia.org/wiki/Mozilla_Public_License#Compatibility_with_GPL), > > due to some of its restrictions, so it should follow that the EPL is > > also not compatible. > > > > If you want to enforce copyleft in your code, but not force it on other > > libraries that link with your code, you can use LGPL instead of GPL. > > (Preferably, use v3, which is compatible with Apache License 2.) > > > > http://www.gnu.org/licenses/lgpl-java.html > > > > /Richard (not a lawyer) > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From francis.joanis@REDACTED Tue Jun 14 17:01:35 2011 From: francis.joanis@REDACTED (Francis Joanis) Date: Tue, 14 Jun 2011 11:01:35 -0400 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> Message-ID: +1 as well for me. Even if it is a far fetched use case, I can imagine a customer wanting to request a copy of the "test suite" (i.e. including PropEr based tests) of a commercial product. Thanks, Francis From jared@REDACTED Tue Jun 14 18:03:03 2011 From: jared@REDACTED (Jared Morrow) Date: Tue, 14 Jun 2011 10:03:03 -0600 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> Message-ID: At a very minimum, the LGPL was invented for cases exactly like these. So +1 to at least moving it to LGPL if something like Apache, BSD, or MIT isn't wanted. On Tue, Jun 14, 2011 at 9:01 AM, Francis Joanis wrote: > +1 as well for me. > > Even if it is a far fetched use case, I can imagine a customer wanting > to request a copy of the "test suite" (i.e. including PropEr based > tests) of a commercial product. > > Thanks, > Francis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ebegumisa@REDACTED Tue Jun 14 18:05:38 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Wed, 15 Jun 2011 02:05:38 +1000 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> Message-ID: Ah, ok, this changes things. I think I've got another idea... but first some clarifications... When you say "real-time", what do you mean? Soft real-time? If so, how soft? How late is too late to respond to events? Or do you actually mean you want to respond synchronously (blocking) rather than asynchronously (non-blocking)? And when you say "take some action" after identifying a pattern of events, is this automatic programmatic action or human action? I first thought you meant human action after-the-fact but now I'm inclined to believe that you mean the former. - Edmond - On Tue, 14 Jun 2011 00:01:12 +1000, Banibrata Dutta wrote: > On Sun, Jun 12, 2011 at 11:24 PM, Edmond Begumisa < > ebegumisa@REDACTED> wrote: > >> My interpretation of your question is that you want to do one-off >> message >> tracing. >> > > Um, not too sure. What I want to do is to identify event-patterns in real > time, and take some action. For example, I am processing event-A from > source > S1. At this time I have no context so I will probably do some minimal > work > and "remember" occurrence of event-A (i.e. update context). Then event-B > might arrive (also from S1), and in this case . > > >> >> 1. You want to trace particular massages passed in your system over a >> period Y, you are particularly interested in 'EXIT' messages, but not >> necessarily: >> >> You could use the trace BIFs/dbg tool for this. Start an appropriate >> tracer >> with dbg:tracer, specify which processes you want to trace with dbg:p >> with >> one of the m|s|r flags to log the events, watch your system for a while, >> turn the tracer off (keep in mind the negative impact such tracing might >> have on a live system.) Analyse the trace output. You could also use >> your >> own tracer handler fun to filter/process the messages. >> > > My usage is not naturally process-tracing, but if I am able to draw a > parallel or use the concept, happy to explore. So far, I am not able to > imagine the mapping of what I'd like to do with this approach. If there > is > some example to illustrate, it'd help. > > >> 2. You want to trace massages over a number of nodes: >> >> Though I haven't used it, I believe this is what the Inviso OTP tool is >> for. You could also have a look at the ttb OTP tool. >> >> 3. You want to trace "cascading" messages in your system: >> >> Have a look at erlang:seq_trace. I believe dbg and ttb both have support >> for sequential tracing (I don't think Inviso does.) >> >> 4. You want a convenient way of walking through traces: >> >> The OTP et_viewer is a really nice tool I use a lot. Though the OTP >> Event >> Tracer can output ordinary trace data produced by dbg, where it really >> gets >> fun is when you feed it trace data by calling et:trace_me in your code >> (I >> normally wrap this in a ?TRACE macro). I like to think of it as a >> replacement for using io:format to print debug messages. The "detail" >> argument is really nice for "zooming" in/out as you page through trace >> data. >> You might find that useful for analysing a large number of messages. >> >> In short, I'm not sure you need any special "architectural design >> patterns". Just design your system as you normally would, using >> processes to >> model what you observe in the world, then trace it when it misbehaves or >> when you want to observe it. The traces *are* your sliding windows... >> unless >> I've completely mis-understood your question! >> >> - Edmond - >> >> >> On Mon, 13 Jun 2011 01:38:20 +1000, Banibrata Dutta < >> banibrata.dutta@REDACTED> wrote: >> >> gr8 questions, and they certainly need clarification. >>> cc'ing the group s.t. others could contribute too. >>> >>> On Sun, Jun 12, 2011 at 8:48 PM, Mihai Balea wrote: >>> >>> >>>> On Jun 12, 2011, at 10:51 AM, Banibrata Dutta wrote: >>>> >>>> Prematurely sent. >>>> >>>> On Sun, Jun 12, 2011 at 7:59 PM, Banibrata Dutta < >>>> banibrata.dutta@REDACTED> wrote: >>>> >>>> >>>>> What would be a good way to correlate asynchronous events, spot >>>>> patterns >>>>> over a sliding window (s.a. of no. of events elapsed or time >>>>> elapsed), >>>>> with >>>>> millions of events occurring simultaneously, using Erlang ? >>>>> >>>>> The set of possible events is known, and any unknown event is just >>>>> flagged >>>>> as 'unknown' (so all unknowns are similar). The set of possible event >>>>> patterns can be enumerated, but is possibly quite a large set of >>>>> patterns. >>>>> >>>>> >>>> Was wondering as to what could be the approach taken to implement >>>> such a >>>> thing in pure Erlang. My initial thoughts were along the line of >>>> maintaining >>>> FSMs per event source, but with so many events and so many >>>> possible/valid >>>> patterns, the thing seems kind of unwieldy. Also, I'd like a >>>> non-programmer >>>> to be able to define new events and valid event patterns. >>>> >>>> I believe 'Complex Event Processing' is quite likely to be the >>>> standard >>>> approach for such things, as I've found from some posts, and solutions >>>> exist >>>> in Java world for same, but both as an academic exercise (for the fun >>>> of >>>> learning) and for a potentially simpler + better solution, would like >>>> to >>>> try >>>> doing this is Erlang. >>>> >>>> >>>> I think you need to define your problem better. >>>> >>>> >>> Sure, let me try. >>> >>> >>> What exactly do you mean by "millions of events occurring >>>> simultaneously"? >>>> >>>> >>> Okay, so I can say something like 500 events/second handled for >>> correlation >>> would be a more realistic number. >>> >>> >>> At exactly the same time? >>>> >>>> >>> Yes... some of the events might be from same source, but spaced by as >>> little >>> as 50ms, but mostly from different sources. There could be some >>> heirarchical >>> relationship between sources. Very typical case of network management >>> scenario. E.g. a fault port on a switch, could probably cause hundreds >>> of >>> destination unreachable events, application response timeouts, >>> heartbeat >>> losses etc.. >>> >>> >>> Millions of events per second? Minute? Is that peak rate, average >>> rate or >>>> minimum rate? >>>> >>>> >>> Okay, I got over-enthusiastic :-) . Say 100 events/second typical, 500 >>> events/second peak, no real minimum. >>> >>> What exactly is a pattern? >>> >>>> >>>> >>> Node-A failed, Power in room-X where Node-A is kept failed, Nodes B,C,D >>> which are served thru Node-A became unreachable, due to which Services >>> L & >>> M >>> became unavailable, and due to which another dependent service N >>> started >>> giving inconsistent answers. So this is a pattern. However in this >>> case, >>> there's a possibility that Power-failure had nothing to do with >>> Noda-A's >>> failure, as backup power was available. >>> >>> Another pattern is, Power in room X failed, then Noda A failed, >>> leading to >>> failure of only Node D, because somehow Nodes B & C were dynamically >>> configured to reroute. This is another pattern. >>> >>> What do you mean by "quite a large set of patterns"? Hundreds, >>> thousands, >>> >>>> millions? >>>> >>>> >>> Several hundreds is a distinct possibility, and thousands are not >>> impossible, but millions -- probably not. >>> >>> >>> How long is that sliding window? >>>> >>>> >>> From few minutes (for certain type of events), to few days (for another >>> type >>> of events). >>> >>> >>> Can patterns encompass events coming from multiple sources or just one >>>> source? >>>> >>>> >>> Yes, indeed. However in this case, there needs to a "relationship" >>> between >>> the event sources, that is pre-defined. E.g. some sense of "topology" >>> exists. However it is likely that only 2% of the event sources are >>> interrelated. >>> >>> >>> Are patterns concerned only with event ordering and occurrence or >>> there >>>> are >>>> timing issues involved as well? >>>> >>>> >>> Ordering, Timing, or any kind of causal relationship. >>> >> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> > > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Tue Jun 14 18:39:27 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Wed, 15 Jun 2011 02:39:27 +1000 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: <4C1F8827-E9F6-4677-BA54-BAA7D1C34474@hates.ms> References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> <6289B011-5222-418A-A178-34DB304A44CE@hates.ms> <4C1F8827-E9F6-4677-BA54-BAA7D1C34474@hates.ms> Message-ID: The more I think about it, the more I think Banibrata is really onto something here! A generic tool of this nature could be rather useful. If such a tool allowed me to say things like... "If 30 'EXIT' messages appear in supervision subtree Y over a period of 1sec and 60% of these contain {tcp_error, _} then call sasl:set_alarm({'network_down', Y})" in which case an SMS would be sent to my phone. I can think of a million uses for such a tool :) - Edmond - On Tue, 14 Jun 2011 06:13:56 +1000, Mihai Balea wrote: > > On Jun 13, 2011, at 9:12 AM, Banibrata Dutta wrote: > >> >> Well, I haven't. In fact, starting with a small set of possible >> patterns, seems to be the most intuitive approach. When I bring in the >> thought of permitting non-programmers to write "something" (e.g. >> natural language rules, or a simple DSL snippet), and then such a thing >> modifying the FSM, without me having to code anything further in >> Erlang, is when I am hitting a mental roadblock. I've coded fairly >> complex FSM's in C/C++ using the table-based approach. Does gen_fsm >> take a similar approach ? Will it be a good, natural fit ? Maybe a >> rather naive question for people extremely familiar with Erlang, but >> I'm still very much on the learning curve's positive vector. > > Erlang gen_fsms are quite simple and standard, they define a bunch of > states and use pattern matching to select actions for each state/event > combination. Pretty much equivalent to a table-based fsm, only there's > no distinct table to speak of. > > >> >> however you might want to make them a bit more flexible. Maybe have a >> bunch of processes running pattern recognizers, say one process per >> pattern or class of patterns. >> >> My natural tendency was to think more in terms of 1 process per source, >> e.g. 1st event from a source causes an Erlang process to be created, >> which then embodies the entire possible FSM. The main issue with >> pattern based processes (if I understood it correctly), is determining >> the right process to hand-off the event to, and once handed-off, >> backtracking would mean extra work. >> >> Maybe you can filter events based on source so certain recognizers will >> only get certain events. >> >> Yes, that was the starting point, as I had imagined. > > Event forwarding based on source is technically easy in Erlang, just use > a process registry (proc, grpoc, even the built in process registrar) > and use sources as keys. > The problem with this approach is you might have at least some > correlation between sources. Events originating from different sources > but belonging to the same pattern (i.e. server going black due to switch > failing in some other room, due to power outage and the like). > >> >> If you have a low correlation between event sources maybe you can even >> design your system to distribute the processing of unrelated events to >> different nodes in a cluster (big scalability gain if you can do that). >> >> Excellent thought... indeed. Source based sharding. >> >> Another way of approaching this would be to duplicate the event stream >> to multiple nodes in your cluster and have each node only look for >> certain subsets of patterns. >> >> Actually, my bad, I should've mentioned, I wanted to do all this in >> Real-Time. So I don't really have a pattern-string, so to say. I don't, >> without that I can really take this approach. Or can I ? > > I think you can. If your patterns are independent, or at least can be > grouped in independent categories, what would prevent you from assigning > sets of independent patterns to different nodes? Not saying you need to > do that, but it might be helpful if you find out that recognizing and > processing thousands of patterns take up too much cpu. > > Mihai > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From banibrata.dutta@REDACTED Tue Jun 14 19:06:58 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Tue, 14 Jun 2011 22:36:58 +0530 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> <6289B011-5222-418A-A178-34DB304A44CE@hates.ms> <4C1F8827-E9F6-4677-BA54-BAA7D1C34474@hates.ms> Message-ID: On Tue, Jun 14, 2011 at 10:09 PM, Edmond Begumisa < ebegumisa@REDACTED> wrote: > The more I think about it, the more I think Banibrata is really onto > something here! > :-) , well, the more I read about CEP (Complex Event Processing), the more I feel, people have been onto this thing for quite a while now. The use-cases all seem very real. > A generic tool of this nature could be rather useful. If such a tool > allowed me to say things like... > > "If 30 'EXIT' messages appear in supervision subtree Y over a period of > 1sec and 60% of these contain {tcp_error, _} then call > sasl:set_alarm({'network_down', Y})" in which case an SMS would be sent to > my phone. > Precisely. It is extremely close to what I was thinking. > > I can think of a million uses for such a tool :) > Indeed. This is the reason, the more I think of FSM (in gen_fsm terms, for instance), the more I get the feeling of getting into something that is "static", something that is embedded deep within an application. Mihai here, mentioned DSL to represent the patterns & actions, and EPL (Event Processing Language) in the CEP terminology, appears to be just that. Thanks to Darach for recommending a really interesting book as well. And to answer your questions from the previous mail -- by real-time I mean soft real-time, or more like nearly synchronous processing, and not an offline/batch processing for post-facto analysis. Also all this processing, and looking into to time window to correlate events, needs to be done by the computer. No human looking here... well, other than to fix things if they are broken. On Tue, 14 Jun 2011 06:13:56 +1000, Mihai Balea wrote: > On Jun 13, 2011, at 9:12 AM, Banibrata Dutta wrote: > > >> Well, I haven't. In fact, starting with a small set of possible patterns, >> seems to be the most intuitive approach. When I bring in the thought of >> permitting non-programmers to write "something" (e.g. natural language >> rules, or a simple DSL snippet), and then such a thing modifying the FSM, >> without me having to code anything further in Erlang, is when I am hitting a >> mental roadblock. I've coded fairly complex FSM's in C/C++ using the >> table-based approach. Does gen_fsm take a similar approach ? Will it be a >> good, natural fit ? Maybe a rather naive question for people extremely >> familiar with Erlang, but I'm still very much on the learning curve's >> positive vector. >> > > Erlang gen_fsms are quite simple and standard, they define a bunch of > states and use pattern matching to select actions for each state/event > combination. Pretty much equivalent to a table-based fsm, only there's no > distinct table to speak of. > > > >> however you might want to make them a bit more flexible. Maybe have a >> bunch of processes running pattern recognizers, say one process per pattern >> or class of patterns. >> >> My natural tendency was to think more in terms of 1 process per source, >> e.g. 1st event from a source causes an Erlang process to be created, which >> then embodies the entire possible FSM. The main issue with pattern based >> processes (if I understood it correctly), is determining the right process >> to hand-off the event to, and once handed-off, backtracking would mean extra >> work. >> >> Maybe you can filter events based on source so certain recognizers will >> only get certain events. >> >> Yes, that was the starting point, as I had imagined. >> > > Event forwarding based on source is technically easy in Erlang, just use a > process registry (proc, grpoc, even the built in process registrar) and use > sources as keys. > The problem with this approach is you might have at least some correlation > between sources. Events originating from different sources but belonging to > the same pattern (i.e. server going black due to switch failing in some > other room, due to power outage and the like). > > >> If you have a low correlation between event sources maybe you can even >> design your system to distribute the processing of unrelated events to >> different nodes in a cluster (big scalability gain if you can do that). >> >> Excellent thought... indeed. Source based sharding. >> >> Another way of approaching this would be to duplicate the event stream to >> multiple nodes in your cluster and have each node only look for certain >> subsets of patterns. >> >> Actually, my bad, I should've mentioned, I wanted to do all this in >> Real-Time. So I don't really have a pattern-string, so to say. I don't, >> without that I can really take this approach. Or can I ? >> > > I think you can. If your patterns are independent, or at least can be > grouped in independent categories, what would prevent you from assigning > sets of independent patterns to different nodes? Not saying you need to do > that, but it might be helpful if you find out that recognizing and > processing thousands of patterns take up too much cpu. > > Mihai > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dang@REDACTED Tue Jun 14 20:20:20 2011 From: dang@REDACTED (Daniel Goertzen) Date: Tue, 14 Jun 2011 13:20:20 -0500 Subject: [erlang-questions] ocaml vs erlang In-Reply-To: References: Message-ID: I'll second that. It was your documented struggles with Haskell years ago that convinced me to learn Erlang in the first place. (thank you BTW :) Dan. On Tue, Jun 14, 2011 at 2:44 AM, Torben Hoffmann wrote: > Hi Joel, > > Instead of resorting to the "I could have told you so" reply I would like > to hear if you could explain where the pain is - I think that will be an > interesting read... as it normally is when you write up a story! > > Cheers, > Torben > > > On Tue, Jun 14, 2011 at 09:12, Joel Reymont wrote: > >> I used OCaml for a few months to build a distributed production system. >> >> The architecture included Mongrel2, ZeroMQ, Redis and Amazon AWS. >> >> Using OCaml in this scenario is like pushing a boulder uphill, I should >> have used Erlang! >> >> -------------------------------------------------------------------------- >> - for hire: mac osx device driver ninja, kernel extensions and usb drivers >> ---------------------+------------+--------------------------------------- >> http://wagerlabs.com | @wagerlabs | >> http://www.linkedin.com/in/joelreymont >> ---------------------+------------+--------------------------------------- >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > http://www.linkedin.com/in/torbenhoffmann > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Daniel Goertzen ----------------- dang@REDACTED (work) daniel.goertzen@REDACTED (home) ----------------- 1 204 272 6149 (home/office) 1 204 470 8360 (mobile) ----------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From igouy2@REDACTED Tue Jun 14 20:39:34 2011 From: igouy2@REDACTED (Isaac Gouy) Date: Tue, 14 Jun 2011 11:39:34 -0700 (PDT) Subject: [erlang-questions] ot: Swedish article featuring Erlang Solutions CTO Message-ID: <511624.55298.qm@web65604.mail.ac4.yahoo.com> On Mon, Jun 13, 2011 at 10:41, Robert Virding wrote: > What is perhaps more interesting than the article itself are > all the comments. "I know nothing about language X but I know > that it is 10-15 times slower than language Y". I did see a comment like that, but at first glance I didn't read it correctly. I think it actually said something more like - Haskell is X times slower and Erlang is Y times slower than C. On Mon, Jun 13, 2011 at 2:46, Jesper Louis Andersen > He is considering the "Computer Language benchmarks game", > specifically the "fasta" benchmark on a Quad Core Fasta? What a strange choice! Oh! Could it be that none of the tests before Fasta show a C program as fastest :-) > You can make the opposite conclusion by considering > the thread-ring benchmark Another sequential program ;-) (At least spectral-norm keeps all the cores working.) -snip- > The danger of concluding something from a synthetic benchmark > is that you are going to conclude the wrong thing. I doubt the person who made that comment on the Computer Sweden website actually was concluding something from looking at the benchmarks game web pages. It seems more likely that they went through the web pages looking for some measurement they could use to demonstrate what they already believed. The salutary lesson is that we all seem to have a weakness for ignoring contradictions to our beliefs and seeking confirmation for our beliefs. From carlsson.richard@REDACTED Tue Jun 14 21:28:26 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 14 Jun 2011 21:28:26 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> Message-ID: <4DF7B65A.90009@gmail.com> On 2011-06-14 19:50, Daniel Luna wrote: > The GNU GPL has always allowed for linking with "system libraries" and > in version 3 that now explicitly includes the standard libraries of > common programming libraries. > > So I don't know whether older versions of GPL are incompatible with > Erlang or not, but it seems clear cut that version 3 is. GPL v2 talks rather specifically about only "the major components (compiler, kernel, and so on) of the operating system on which the executable runs", which does not seem to cover things like runtime libraries of a language implementation apart from the language the OS is written in. You're right that v3 seems to be much more "modern" in this regard, so the libraries and runtime system of the Erlang/OTP standard distribution are probably not a problem under GPL v3. Still, any other software (i.e., your tests) specifically designed to depend on PropEr as currently published under the GPL would itself have to be offered under the terms of the GPL. > Generally I would not recommend LGPL, but would rather recommend that > Kostis dual licenses the software. Most (if not all) users would be > fine with GPL. The few that really, really want to give the tests to > their clients without also giving away the source code, could then > purchase a closed source version of the software. There are many open source projects that wish to include their test code as part of their distribution, both because it's easier not to strip them out, and because their users may want to run the tests. If they want to distribute their main code offering under a non-copyleft license like MIT or Apache - it's their choice - they could only do that by splitting the code into one non-copyleft distribution without the tests and one GPL-compatible distribution with the tests. That's asking quite a lot of your poor users. Test frameworks are simply a bit different from other library code, and I think the licensing should reflect this. But that's all down to Kostis and Manolis to decide, of course. /Richard From overminddl1@REDACTED Tue Jun 14 21:48:14 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Tue, 14 Jun 2011 13:48:14 -0600 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> Message-ID: Ditto, GPL is completely incompatible with the (non-fsf) open licenses of things I use. LGPL would work, but with the GPL I would never be able to use PropEr. On Jun 14, 2011 8:13 AM, "James Churchman" wrote: > Could the lgpl not be chosen instead? Would this not allow the tests to be distributed along as they don't ship a custom version of propEr as somebody else states? > Also is it really the case that the tests could not be distributed, or more a case that it is totally fine to distribute the tests along as the person receiving the tests downloads proper themselves? the code that is not tests would still run fine. > > Also what was the original reasoning behind choosing the gpl in this case? > > Regards > > James > > > also +1 to try to convince Kostis to release PropEr with a better license! > > On 14 Jun 2011, at 13:33, Eric Merritt wrote: > >> I talked with Kostis about this at the erlang factory. He said he is going to provide the exception to the gpl for open source projects. I think he just has not had time to do the actual work. >> >> On Jun 14, 2011 7:26 AM, "Richard Carlsson" wrote: >> > On 06/14/2011 02:02 PM, Ulf Wiger wrote: >> >> On 14 Jun 2011, at 01:09, Vik Olliver wrote: >> >>> On 14/06/11 11:01, Francis Joanis wrote: >> >>>> I have a question about the GPL licensing of the tool itself: >> >>>> since it is GPL, doesn't it require to make the written tests >> >>>> themselves and to some extent the application under test GPL as >> >>>> well? >> >>> >> >>> Only if you distribute the tests. If you keep them to yourself, >> >>> you don't owe anyone anything. The GPL respects privacy. >> >> >> >> This could become an obstacle to adoption similar to what's been seen >> >> with Quviq QuickCheck: people who have Open Source projects need to >> >> publish (=distribute) test suites as well. Just as it's not very >> >> helpful to distribute test suites that require an expensive tool, >> >> people might be reluctant to publish PropEr-based tests if it >> >> requires them to convert to GPL. >> >> >> >> Kostis did say that they would make an exception for OSS projects. I >> >> could not find any such exception when quickly scanning the repos. >> >> >> >> BR, Ulf W >> > >> > As I have remarked before on this list, I believe the normal GPL (all >> > versions) cannot be used for Erlang modules, because the dynamic loading >> > and linking of classes/modules at runtime is not considered to be any >> > different from static linking - at least according to the FSF. >> > >> > The standard Erlang library modules and the runtime system BIFs are >> > published under the Erlang Public License, which is an even more >> > restrictive variant of MPL. The MPL is not compatible with GPL >> > ( http://en.wikipedia.org/wiki/Mozilla_Public_License#Compatibility_with_GPL), >> > due to some of its restrictions, so it should follow that the EPL is >> > also not compatible. >> > >> > If you want to enforce copyleft in your code, but not force it on other >> > libraries that link with your code, you can use LGPL instead of GPL. >> > (Preferably, use v3, which is compatible with Apache License 2.) >> > >> > http://www.gnu.org/licenses/lgpl-java.html >> > >> > /Richard (not a lawyer) >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Tue Jun 14 22:27:36 2011 From: mihai@REDACTED (Mihai Balea) Date: Tue, 14 Jun 2011 16:27:36 -0400 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> <6289B011-5222-418A-A178-34DB304A44CE@hates.ms> <4C1F8827-E9F6-4677-BA54-BAA7D1C34474@hates.ms> Message-ID: <7D9BCBD2-9512-499D-A575-A71D853695F5@hates.ms> On Jun 14, 2011, at 12:39 PM, Edmond Begumisa wrote: > The more I think about it, the more I think Banibrata is really onto something here! > > A generic tool of this nature could be rather useful. If such a tool allowed me to say things like... > > "If 30 'EXIT' messages appear in supervision subtree Y over a period of 1sec and 60% of these contain {tcp_error, _} then call sasl:set_alarm({'network_down', Y})" in which case an SMS would be sent to my phone. > > I can think of a million uses for such a tool :) I think you just gave me an idea for an open source project :) Mihai From jozsef.berces@REDACTED Wed Jun 15 00:22:24 2011 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces?=) Date: Wed, 15 Jun 2011 00:22:24 +0200 Subject: [erlang-questions] discarding signals Message-ID: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> In case async signal communication, is it possible to discard the signals queuing if the process detects that it will not be able process them? I remember that the process info returns the signal queue length, so it is possible to set a threshold. But I did not find anything for discarding. Thanks, Jozsef -------------- next part -------------- An HTML attachment was scrubbed... URL: From masklinn@REDACTED Wed Jun 15 00:44:14 2011 From: masklinn@REDACTED (Masklinn) Date: Wed, 15 Jun 2011 00:44:14 +0200 Subject: [erlang-questions] discarding signals In-Reply-To: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> Message-ID: On 2011-06-15, at 00:22 , J?zsef B?rces wrote: > In case async signal communication, is it possible to discard the signals queuing if the process detects that it will not be able process them? > > I remember that the process info returns the signal queue length, so it is possible to set a threshold. But I did not find anything for discarding. Any message read from a process's mailbox (via a receive statement) is removed from the mailbox. If you don't act on it, it will be discarded. So the way to discard a message is "read it, and ignore it". From ebegumisa@REDACTED Wed Jun 15 02:02:01 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Wed, 15 Jun 2011 10:02:01 +1000 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> <6289B011-5222-418A-A178-34DB304A44CE@hates.ms> <4C1F8827-E9F6-4677-BA54-BAA7D1C34474@hates.ms> Message-ID: Been thinking about this some more... First you need to detect/gather the events, then you need a way for your actors to subscribe to the events, then you a way for the actors to filter and process the gathered events. Some possibilities: 1. DETECTING/ACCUMULATING EVENTS I think you have two choices here: send massages or trace messages. Sending messages to whatever process is collecting them is easy so I won't bother with an example. But it means you have to hard-code the events in advance, which might be restrictive for your purposes. Personally, I'd think you'd have better luck with tracing (as long as you limit tracing to a few processes - like your supervisors). Using the previous "network_down" example, I think all you'd have to do is trace massages sent to supervisor Y, then filter those for 'EXIT' messages (NOTE: I haven't tested this AT ALL, this is purely thinking aloud)... === detector.erl === start_collecting(Node, MaxMicroSecs, MaxMsgCnt) -> {ok, Tracer} = dbg:tracer(Node, process, {fun handle_trace/2, {MaxMicroSecs, MaxMsgCnt, null, []}}), % Accumulate messages you're interested in {ok, _} = dbg:p(sup_y, [r]), % Trace messages received by supervisor Y Tracer. handle_trace({trace, _, 'recieve', {'EXIT',_,_}} = Trace, {MaxMicroSecs, MaxMsgCnt, null, []}) -> handle_trace(Trace, {MaxMicroSecs, MaxMsgCnt, erlang:now(), []}); handle_trace({trace, _, 'recieve', {'EXIT',_,_} = Msg}, {MaxMicroSecs, MaxMsgCnt, LastTime0, Acc0}) -> HeadTime = erlang:now(), Acc1 = [{HeadTime, Msg} | Acc0], case length(Acc1) =:= MaxMsgCnt of true -> collector:report_events(Acc1), {MaxMicroSecs, MaxMsgCnt, null, []}; false -> case timer:now_diff(HeadTime, LastTime0) > MaxMicroSecs of true -> Acc2 = lists:dropwhile( fun ({X,_}) -> timer:now_diff(HeadTime, X) > MaxMicroSecs end, lists:reverse(Acc1)), [LastTime1, _] = Acc2, {MaxMicroSecs, MaxMsgCnt, LastTime1, lists:reverse(Acc2)}; false -> {MaxMicroSecs, MaxMsgCnt, LastTime0, Acc1} end end; handle_trace(_, Data) -> Data. === Now, so if you called... > detector:start_collecting(Node, 1000000, 30). ... collector:report_events/1 should be called with a list of 30 {Time, ExitMessage} tuples detected over a period of 1 second (I hope.) I suppose you could achieve the same by using sasl and a custom handler instead then respond to supervisor reports, but the advantage of the above method is that it could be easily modified to handle messages other than 'EXIT' messages. 2. SUBSCRIBERS: You'll require an easy way for many actors to respond to the same events and to add/remove themselves as possible handlers. I see two shortcuts for this: gen_event or et_collector. For gen_event, just follow the documentation -- collector.erl would be a gen_event behaviour and collector:report_events/1 would call gen_event:notify(Ref, L) where L is the {Time, ExitMessage} list already grouped/accumulated by detector.erl. Then your actors call gen_event:add_handler, etc, etc. However, I see advantages to trying out et_collector instead. It will take some hacking though, but I think it would be worth it. It has it's own dictionary service and built in a way of filtering events. Not to mention a nice GUI (et_viewer) that you can use later to view events post-morten. (a quick look at it's source reveals that you'll probably end up trying to rewrite a lot of the same code.) From the (sparse) documentation... "...The Collector is a generic full-fledged framework that allows processes to 'subscribe' to the Events that it collects. One Collector can serve several Viewers... The architecture does also allow you to implement your own Viewer program as long as it complies to the protocol between the Collector/Viewer protocol..." So your actors would be non-visual viewers created from a module written by you based code inspired by et_viewer.erl. Then, the collector:report_events/1 could look like... report_events(L) -> et_collector:report_event(99,sup_y,alarm,exit_seq,L). Then all subscribers would be notified do their thing. Best of all, later, you can use the standard et_viewer to see... sup_y alarm | exit_seq | |------>------| | | Clicking on exit_seq would display {Time, ExitMsg} list L. 3. PROCESSING: With gen_event, it would be a matter of applying whatever statistics you want to apply to the {Time, ExitMsg} list L passed to each handler, then performing your automatic actions based on the stats. Erlang being a soft-real-time system, I think you'd usually get to respond in time even though messages would arrive asynchronously. With et, it would be the same except you'd perform the stats and automated with the subscribers. What's left is finding a way of doing the stat processing and auto actions based on a set of instructions coming from non-programmers. I honestly have ZERO suggestions there :( - Edmond - On Wed, 15 Jun 2011 03:06:58 +1000, Banibrata Dutta wrote: > On Tue, Jun 14, 2011 at 10:09 PM, Edmond Begumisa < > ebegumisa@REDACTED> wrote: > >> The more I think about it, the more I think Banibrata is really onto >> something here! >> > > :-) , well, the more I read about CEP (Complex Event Processing), the > more I > feel, people have been onto this thing for quite a while now. The > use-cases > all seem very real. > > >> A generic tool of this nature could be rather useful. If such a tool >> allowed me to say things like... >> >> "If 30 'EXIT' messages appear in supervision subtree Y over a period of >> 1sec and 60% of these contain {tcp_error, _} then call >> sasl:set_alarm({'network_down', Y})" in which case an SMS would be sent >> to >> my phone. >> > > Precisely. It is extremely close to what I was thinking. > >> >> I can think of a million uses for such a tool :) >> > > Indeed. This is the reason, the more I think of FSM (in gen_fsm terms, > for > instance), the more I get the feeling of getting into something that is > "static", something that is embedded deep within an application. Mihai > here, > mentioned DSL to represent the patterns & actions, and EPL (Event > Processing > Language) in the CEP terminology, appears to be just that. Thanks to > Darach > for recommending a really interesting book as well. > > And to answer your questions from the previous mail -- by real-time I > mean > soft real-time, or more like nearly synchronous processing, and not an > offline/batch processing for post-facto analysis. Also all this > processing, > and looking into to time window to correlate events, needs to be done by > the > computer. No human looking here... well, other than to fix things if they > are broken. > > On Tue, 14 Jun 2011 06:13:56 +1000, Mihai Balea wrote: > > >> On Jun 13, 2011, at 9:12 AM, Banibrata Dutta wrote: >> >> >>> Well, I haven't. In fact, starting with a small set of possible >>> patterns, >>> seems to be the most intuitive approach. When I bring in the thought of >>> permitting non-programmers to write "something" (e.g. natural language >>> rules, or a simple DSL snippet), and then such a thing modifying the >>> FSM, >>> without me having to code anything further in Erlang, is when I am >>> hitting a >>> mental roadblock. I've coded fairly complex FSM's in C/C++ using the >>> table-based approach. Does gen_fsm take a similar approach ? Will it >>> be a >>> good, natural fit ? Maybe a rather naive question for people extremely >>> familiar with Erlang, but I'm still very much on the learning curve's >>> positive vector. >>> >> >> Erlang gen_fsms are quite simple and standard, they define a bunch of >> states and use pattern matching to select actions for each state/event >> combination. Pretty much equivalent to a table-based fsm, only there's >> no >> distinct table to speak of. >> >> >> >>> however you might want to make them a bit more flexible. Maybe have a >>> bunch of processes running pattern recognizers, say one process per >>> pattern >>> or class of patterns. >>> >>> My natural tendency was to think more in terms of 1 process per source, >>> e.g. 1st event from a source causes an Erlang process to be created, >>> which >>> then embodies the entire possible FSM. The main issue with pattern >>> based >>> processes (if I understood it correctly), is determining the right >>> process >>> to hand-off the event to, and once handed-off, backtracking would mean >>> extra >>> work. >>> >>> Maybe you can filter events based on source so certain recognizers will >>> only get certain events. >>> >>> Yes, that was the starting point, as I had imagined. >>> >> >> Event forwarding based on source is technically easy in Erlang, just >> use a >> process registry (proc, grpoc, even the built in process registrar) and >> use >> sources as keys. >> The problem with this approach is you might have at least some >> correlation >> between sources. Events originating from different sources but >> belonging to >> the same pattern (i.e. server going black due to switch failing in some >> other room, due to power outage and the like). >> >> >>> If you have a low correlation between event sources maybe you can even >>> design your system to distribute the processing of unrelated events to >>> different nodes in a cluster (big scalability gain if you can do that). >>> >>> Excellent thought... indeed. Source based sharding. >>> >>> Another way of approaching this would be to duplicate the event stream >>> to >>> multiple nodes in your cluster and have each node only look for certain >>> subsets of patterns. >>> >>> Actually, my bad, I should've mentioned, I wanted to do all this in >>> Real-Time. So I don't really have a pattern-string, so to say. I don't, >>> without that I can really take this approach. Or can I ? >>> >> >> I think you can. If your patterns are independent, or at least can be >> grouped in independent categories, what would prevent you from assigning >> sets of independent patterns to different nodes? Not saying you need to >> do >> that, but it might be helpful if you find out that recognizing and >> processing thousands of patterns take up too much cpu. >> >> Mihai >> >> -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Wed Jun 15 02:21:56 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Wed, 15 Jun 2011 10:21:56 +1000 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: <7D9BCBD2-9512-499D-A575-A71D853695F5@hates.ms> References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> <6289B011-5222-418A-A178-34DB304A44CE@hates.ms> <4C1F8827-E9F6-4677-BA54-BAA7D1C34474@hates.ms> <7D9BCBD2-9512-499D-A575-A71D853695F5@hates.ms> Message-ID: Count me in! - Edmond - On Wed, 15 Jun 2011 06:27:36 +1000, Mihai Balea wrote: > > On Jun 14, 2011, at 12:39 PM, Edmond Begumisa wrote: > >> The more I think about it, the more I think Banibrata is really onto >> something here! >> >> A generic tool of this nature could be rather useful. If such a tool >> allowed me to say things like... >> >> "If 30 'EXIT' messages appear in supervision subtree Y over a period of >> 1sec and 60% of these contain {tcp_error, _} then call >> sasl:set_alarm({'network_down', Y})" in which case an SMS would be sent >> to my phone. >> >> I can think of a million uses for such a tool :) > > I think you just gave me an idea for an open source project :) > > Mihai -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From jozsef.berces@REDACTED Wed Jun 15 04:02:47 2011 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces?=) Date: Wed, 15 Jun 2011 04:02:47 +0200 Subject: [erlang-questions] discarding signals In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> Message-ID: <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> Thanks. Actually, the real question is the performance of going through the inbox and discarding 1 by 1. If we are talking about an overloaded system that, for any reason, cannot tell the sender to reduce the traffic, then probably the only way to avoid a system crash is to discard the messages and notify the operators (raise an alarm, write event log, etc.) about this. So the discard shall be very quick as we are already overloaded. Do we have something for that? -----Original Message----- From: Masklinn [mailto:masklinn@REDACTED] Sent: Wednesday, June 15, 2011 5:44 To: J?zsef B?rces Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] discarding signals On 2011-06-15, at 00:22 , J?zsef B?rces wrote: > In case async signal communication, is it possible to discard the signals queuing if the process detects that it will not be able process them? > > I remember that the process info returns the signal queue length, so it is possible to set a threshold. But I did not find anything for discarding. Any message read from a process's mailbox (via a receive statement) is removed from the mailbox. If you don't act on it, it will be discarded. So the way to discard a message is "read it, and ignore it". From fred.hebert@REDACTED Wed Jun 15 06:02:55 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Wed, 15 Jun 2011 00:02:55 -0400 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> Message-ID: I'm wondering about this myself. If I only provide the tests (including the -include_lib line) but not the PropEr code itself, is it breaking the license? Technically, none of the PropEr code is running in there, only its API. -- Fred H?bert http://www.erlang-solutions.com On 2011-06-14, at 10:13 AM, James Churchman wrote: > Also is it really the case that the tests could not be distributed, or more a case that it is totally fine to distribute the tests along as the person receiving the tests downloads proper themselves? the code that is not tests would still run fine. > > Regards > > James > > > also +1 to try to convince Kostis to release PropEr with a better license! From max.lapshin@REDACTED Wed Jun 15 06:59:51 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 15 Jun 2011 08:59:51 +0400 Subject: [erlang-questions] discarding signals In-Reply-To: <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> Message-ID: On Wed, Jun 15, 2011 at 6:02 AM, J?zsef B?rces wrote: > Thanks. > > Actually, the real question is the performance of going through the inbox and discarding 1 by 1. > > If we are talking about an overloaded system that, for any reason, cannot tell the sender to reduce the traffic, then probably the only way to avoid a system crash is to discard the messages and notify the operators (raise an alarm, write event log, etc.) about this. So the discard shall be very quick as we are already overloaded. Do we have something for that? > flush() -> receive Message -> flush() after 0 -> ok end. Is really fast. If you _think_ that it is not fast, than you are on wrong way, because you are going to build high loaded system, based on thinks. You must have benchmarks, that appears to look like real load. And if you will find on these benchmarks that it is not enough, than you will need to do something more. From matteo.redaelli@REDACTED Wed Jun 15 09:27:03 2011 From: matteo.redaelli@REDACTED (matteo.redaelli@REDACTED) Date: Wed, 15 Jun 2011 09:27:03 +0200 (CEST) Subject: [erlang-questions] Oauth 2.0 support Message-ID: <8980542.5113221308122823278.JavaMail.defaultUser@defaultHost> Hi all What about Oauth 2.0 support? Is httpc a good place for it? Please read this thread https://github.com/tim/erlang- oauth/issues/4#comment_1218849 Regards Matteo From torben.lehoff@REDACTED Wed Jun 15 09:52:33 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Wed, 15 Jun 2011 09:52:33 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> Message-ID: I am afraid that the GPLv3 is contaminating in this case - I have spent some time reading the license and how to interpret it and I also asked around, so I do not think you can ship PropEr test code with your product without becoming GPLv3. At least I personally would not run that risk. Note that if you are using PropEr internally and you are not shipping the test source or beams you are free to use PropEr to test your code. But if you ship then your code becomes GPLv3 (as I read the thing). However, as Eric mentioned earlier, the PropEr team is looking into adding a FOSS License Exception and now that the big v1.0 is behind them they hopefully get the time need to investigate this and see if it is possible to do. The PropEr team has been very approachable about this issue so I think we should just wait for them to find the time it takes to get this done right. With a FOSS License Exception there will be no issues and it will open the doors for PropEr to prosper for Open Source Erlang projects. If you want to work with PropEr and receive some guidance from a semi-veteran (me) you can join us on the Erlware team where we are working on property based tests for the erlware-commons and gradually adding them where it makes sense. Cheers, Torben 2011/6/15 Fr?d?ric Trottier-H?bert > I'm wondering about this myself. If I only provide the tests (including the > -include_lib line) but not the PropEr code > itself, is it breaking the license? Technically, none of the PropEr code is > running in there, only its API. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-06-14, at 10:13 AM, James Churchman wrote: > > > Also is it really the case that the tests could not be distributed, or > more a case that it is totally fine to distribute the tests along as the > person receiving the tests downloads proper themselves? the code that is not > tests would still run fine. > > > > Regards > > > > James > > > > > > also +1 to try to convince Kostis to release PropEr with a better > license! > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed Jun 15 11:27:21 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 15 Jun 2011 10:27:21 +0100 Subject: [erlang-questions] How to do this in Erlang -- any suggestions ? In-Reply-To: References: <87E79BF3-E510-4122-BBFB-B982767FFC8F@hates.ms> <6289B011-5222-418A-A178-34DB304A44CE@hates.ms> <4C1F8827-E9F6-4677-BA54-BAA7D1C34474@hates.ms> <7D9BCBD2-9512-499D-A575-A71D853695F5@hates.ms> Message-ID: +1. :) On 15 June 2011 01:21, Edmond Begumisa wrote: > Count me in! > > - Edmond - > > On Wed, 15 Jun 2011 06:27:36 +1000, Mihai Balea wrote: > >> >> On Jun 14, 2011, at 12:39 PM, Edmond Begumisa wrote: >> >>> The more I think about it, the more I think Banibrata is really onto >>> something here! >>> >>> A generic tool of this nature could be rather useful. If such a tool >>> allowed me to say things like... >>> >>> "If 30 'EXIT' messages appear in supervision subtree Y over a period of >>> 1sec and 60% of these contain {tcp_error, _} then call >>> sasl:set_alarm({'network_down', Y})" in which case an SMS would be sent to >>> my phone. >>> >>> I can think of a million uses for such a tool :) >> >> I think you just gave me an idea for an open source project :) >> >> Mihai > > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From erlangsiri@REDACTED Wed Jun 15 11:36:02 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Wed, 15 Jun 2011 11:36:02 +0200 Subject: [erlang-questions] Running a release as a Windows service with erlsrv.exe and start_erl.exe In-Reply-To: References: Message-ID: Hi Francis, Ben! This problem exists on all windows platforms. Correction will be included in R14B04. Regards /siri 2011/6/10 Francis Joanis > Hi, > > Do you know if this issue only affects Windows Server 2008 or does it also > happen on Windows Server 2003? > > Thanks, > Francis > > > On Thu, Jun 9, 2011 at 6:55 AM, Ben Godfrey wrote: > >> Thanks Siri, >> >> Is there anything I can do to help? >> >> Ben >> >> On 7 June 2011 09:11, Siri Hansen wrote: >> > Hi Ben! >> > Unfortunately, the release handling for windows, including erlsrv.erl >> and >> > start_erl.c, is broken. This was discovered not long ago, after >> refreshing >> > the release_handler test suite where these tests were skipped on >> windows. We >> > are currently working on this issue, and plan to release a correction as >> > soon as possible. >> > Regards >> > siri@REDACTED >> > >> > 2011/6/6 Ben Godfrey >> >> >> >> Hello all, >> >> >> >> I've been trying to deploy an Erlang application as a Windows service >> >> and not having much luck. >> >> >> >> I built a release package using rebar. I'm registering the service as >> >> follows: >> >> >> >> c:\erl5.8.3\erts-5.8.3\bin\erlsrv.exe add Rolf -c "Collects system >> >> data for monitoring." -w c:\rolf -m >> >> c:\erl5.8.3\erts-5.8.3\bin\start_erl.exe -debugtype reuse -args >> >> "-setcookie xyz123 ++ -reldir c:\rolf\releases" >> >> >> >> This fails with the error "Failed to query BinDir of release." in the >> >> .debug file. Looking at the code for start_erl.exe, this is related to >> >> registry keys. I can't see any sign of any keys which match the >> >> pattern described in the source, but curiously the preceding call to >> >> open a registry key succeeds [2]. >> >> >> >> Any one have any idea about any of this stuff? >> >> >> >> Server is Windows Server 2008. >> >> >> >> Ben >> >> >> >> [1] >> https://github.com/erlang/otp/blob/dev/erts/etc/win32/start_erl.c#L364 >> >> [2] >> https://github.com/erlang/otp/blob/dev/erts/etc/win32/start_erl.c#L342 >> >> >> >> -- >> >> Ben Godfrey, Wandering Hacker >> >> http://aftnn.org | http://twitter.com/afternoon | >> >> http://www.linkedin.com/in/bengodfrey >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> >> >> >> -- >> Ben Godfrey, Wandering Hacker >> http://aftnn.org | http://twitter.com/afternoon | >> http://www.linkedin.com/in/bengodfrey >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Wed Jun 15 13:07:14 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 15 Jun 2011 14:07:14 +0300 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> Message-ID: <4DF89262.5060500@cs.ntua.gr> Torben Hoffmann wrote: > I am afraid that the GPLv3 is contaminating in this case ... > > However, as Eric mentioned earlier, the PropEr team is looking into > adding a FOSS License Exception and now that the big v1.0 is behind them > they hopefully get the time need to investigate this and see if it is > possible to do. > > The PropEr team has been very approachable about this issue so I think > we should just wait for them to find the time it takes to get this done > right. Thanks for this comment! We have discussed the issue extensively yesterday and today we wrote a mail to the Free Software Foundation to also ask their opinion and their suggestion on how to properly deal with the issues involved. Quoting from the mail we sent to FSF: Our primary concern is that PropEr be as useful as possible to FOSS programmers. Therefore, the most important requirement from PropEr's license is that it allows FOSS projects (regardless of license) to: * use PropEr without restrictions, and especially without needing to switch to another license * release their testing code under whatever license they choose * distribute PropEr freely, to be used as part of the build cycle of the project, or for other uses (e.g. if the project in question is an Erlang IDE, it could distribute PropEr as a plugin), again without needing to switch licenses The above benefits need *not* extend to developers of proprietary software. If PropEr's license could restrict them in some way, that would be an added bonus, but this is not a strict requirement. Another requirement is that PropEr's license is a (preferably strong) copyleft: we would like to guarantee that any extensions to PropEr are also free software. // ... We also told them about the issue that the GPL is supposedly // unusable for Erlang programs due to the nature of Erlang. In short, we are looking for a solution/license that will allow at least all open-source projects to use PropEr without any restrictions or by being contaminated by GPL. I'll be surprised if we do not find a solution, but please bear with us until we resolve this the proper way. In the meantime, you can use the software and the proper e-mail address for questions and comments. Kostis From francesco@REDACTED Wed Jun 15 14:21:44 2011 From: francesco@REDACTED (Francesco Cesarini) Date: Wed, 15 Jun 2011 13:21:44 +0100 Subject: [erlang-questions] CUFP Talk Submission Deadline Today (15/6) for Tokyo Message-ID: <4DF8A3D8.8000007@erlang-solutions.com> Hi All, a reminder that today (June 15^th ) is the deadline to submit your talks for the Commercial Users of Functional Programming in Tokyo September 22nd - 24th. CUFP is designed to serve this community. The annual CUFP workshop is a place where people can see how others are using functional programming to solve real world problems; where practitioners meet and collaborate; where language designers and users can share ideas about the future of their favorite language; and where one can learn practical techniques and approaches for putting functional programming to work. We would love to see Erlang represented in force. Even better, case studies of Erlang being used alongside other functional (and non) functional languages, using and highlighting the use of the right tool for the right job. For more information on CUFP and the submission, visit the CUFP website: http://cufp.org/2011-call-presentations Hope to see you there, Francesco -- Erlang Solutions Ltd. http://www.erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Wed Jun 15 14:35:41 2011 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 15 Jun 2011 12:35:41 +0000 (GMT) Subject: [erlang-questions] wxWidgets In-Reply-To: <63334269.302261308140350677.JavaMail.root@zimbra> Message-ID: <1050086879.302371308141341437.JavaMail.root@zimbra> The wx application binding of wxWidgets seems to be cause crashes on many systems, including MacOSX. After pondering on it I have a number of questions: Is it the wxWidget implementation itself which is buggy? OR Is the use of them directly in the BEAM which causes the crashes? Either through some inherent match problem, or version mismatch, or ...? If it is the second, would implementing a completely separate wx process solve the problem? I.e. would it be possible to implement such a process which would make linking with wxWidgets "safe"? If so, wouldn't it be better to do just that? Run wxWidgets in a separate OS process and use a (non-linked in driver) port to access it? Yes, it would be slower but it would work and be more robust. If this has already been discussed and my questions answered please point me in the right direction. Robert From iostres@REDACTED Wed Jun 15 15:38:43 2011 From: iostres@REDACTED (Ivan Ostres) Date: Wed, 15 Jun 2011 15:38:43 +0200 Subject: [erlang-questions] Fwd: Re: How to do this in Erlang -- any suggestions ? Message-ID: Wth...it's not like I have anything better to do. +1 ;-). I. On 15/6/11 11:27 AM, Tim Watson wrote: > +1. :) > > On 15 June 2011 01:21, Edmond Begumisa wrote: >> Count me in! >> >> - Edmond - >> >> On Wed, 15 Jun 2011 06:27:36 +1000, Mihai Balea wrote: >> >>> On Jun 14, 2011, at 12:39 PM, Edmond Begumisa wrote: >>> >>>> The more I think about it, the more I think Banibrata is really onto >>>> something here! >>>> >>>> A generic tool of this nature could be rather useful. If such a tool >>>> allowed me to say things like... >>>> >>>> "If 30 'EXIT' messages appear in supervision subtree Y over a period of >>>> 1sec and 60% of these contain {tcp_error, _} then call >>>> sasl:set_alarm({'network_down', Y})" in which case an SMS would be sent to >>>> my phone. >>>> >>>> I can think of a million uses for such a tool :) >>> I think you just gave me an idea for an open source project :) >>> >>> Mihai >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From dmitrii@REDACTED Wed Jun 15 16:31:09 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Wed, 15 Jun 2011 17:31:09 +0300 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <4DF89262.5060500@cs.ntua.gr> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> Message-ID: <8E8325DF-88D0-4AB2-8D51-87B43A91EB20@dmitriid.com> Then MPL or EPL should work for PropEr, won't they? To quote from EPL: 3.2. Availability of Source Code. Any Modification which You contribute must be made available in Source Code form under the terms of this License This covers the "requirement is that PropEr's license is a (preferably strong) copyleft: we would like to guarantee that any extensions to PropEr are also free software". I believe that extensions can be treated as modifications (I can be wrong here). As for distribution and usage: 2.1. The Initial Developer Grant. The Initial Developer hereby grants You a world-wide, royalty-free, non-exclusive license, subject to third party intellectual property claims: (a) to use, reproduce, modify, display, perform, sublicense and distribute the Original Code (or portions thereof) with or without Modifications, or as part of a Larger Work; and (b) under patents now or hereafter owned or controlled by Initial Developer, to make, have made, use and sell (``Utilize'') the Original Code (or portions thereof), but solely to the extent that any such patent is reasonably necessary to enable You to Utilize the Original Code (or portions thereof) and not to any greater extent that may be necessary to Utilize further Modifications or combinations. 3.7. Larger Works. You may create a Larger Work by combining Covered Code with other code not governed by the terms of this License and distribute the Larger Work as a single product. In such a case, You must make sure the requirements of this License are fulfilled for the Covered Code. > Torben Hoffmann wrote: >> I am afraid that the GPLv3 is contaminating in this case ... >> However, as Eric mentioned earlier, the PropEr team is looking into adding a FOSS License Exception and now that the big v1.0 is behind them they hopefully get the time need to investigate this and see if it is possible to do. >> The PropEr team has been very approachable about this issue so I think we should just wait for them to find the time it takes to get this done right. > > Thanks for this comment! > > We have discussed the issue extensively yesterday and today we wrote a mail to the Free Software Foundation to also ask their opinion and their suggestion on how to properly deal with the issues involved. > > Quoting from the mail we sent to FSF: > > Our primary concern is that PropEr be as useful as possible to FOSS > programmers. Therefore, the most important requirement from PropEr's > license is that it allows FOSS projects (regardless of license) to: > > * use PropEr without restrictions, and especially without needing to > switch to another license > * release their testing code under whatever license they choose > * distribute PropEr freely, to be used as part of the build cycle of > the project, or for other uses (e.g. if the project in question is > an Erlang IDE, it could distribute PropEr as a plugin), again > without needing to switch licenses > > The above benefits need *not* extend to developers of proprietary > software. If PropEr's license could restrict them in some way, that > would be an added bonus, but this is not a strict requirement. > > Another requirement is that PropEr's license is a (preferably strong) > copyleft: we would like to guarantee that any extensions to PropEr are > also free software. > > // ... We also told them about the issue that the GPL is supposedly > // unusable for Erlang programs due to the nature of Erlang. > > In short, we are looking for a solution/license that will allow at least all open-source projects to use PropEr without any restrictions or by being contaminated by GPL. > > I'll be surprised if we do not find a solution, but please bear with us until we resolve this the proper way. > > In the meantime, you can use the software and the proper e-mail address for questions and comments. > > Kostis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jesper.louis.andersen@REDACTED Wed Jun 15 17:34:16 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 15 Jun 2011 17:34:16 +0200 Subject: [erlang-questions] discarding signals In-Reply-To: <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> Message-ID: On Wed, Jun 15, 2011 at 04:02, J?zsef B?rces wrote: > If we are talking about an overloaded system that, for any reason, cannot tell the sender to reduce the traffic, then probably the only way to avoid a system crash is to discard the messages and notify the operators (raise an alarm, write event log, etc.) about this. So the discard shall be very quick as we are already overloaded. Do we have something for that? I dare say that you want to concentrate on avoidance rather than recovery. In other words, you want to write in some flow-control into the application such that the problem never occurs in the first place. The problem with recovering by emptying the queue is that it is as if we lost those messages to the senders, so they won't react and probably push us into another recovery phase soon thereafter. If we have a flow-control system built in, the senders will know, mainly due to timeouts, and can hence react to the situation. Effectively, you are implementing queue bounds by the flow-control scheme so you can limit how many messages are in the queue and keep the queue loaded at a sustainable rate. Another reason I'd prefer avoidance is that recovery is often way more expensive in the long run. You will perhaps try to resend the messages or redo a lot of heavy computation. Avoiding that in the first place will also improve the throughput. -- J. From robert.virding@REDACTED Wed Jun 15 17:39:40 2011 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 15 Jun 2011 15:39:40 +0000 (GMT) Subject: [erlang-questions] ot: Swedish article featuring Erlang Solutions CTO In-Reply-To: <511624.55298.qm@web65604.mail.ac4.yahoo.com> Message-ID: <63763193.303771308152380160.JavaMail.root@zimbra> ----- "Isaac Gouy" wrote: > On Mon, Jun 13, 2011 at 10:41, Robert Virding wrote: > > > What is perhaps more interesting than the article itself are > > all the comments. "I know nothing about language X but I know > > that it is 10-15 times slower than language Y". > > I did see a comment like that, but at first glance I didn't read it > correctly. > > I think it actually said something more like - Haskell is X times > slower and Erlang is Y times slower than C. That was the type of comment I meant, I was just generalising it a bit. It was in another part of the comment where he mentioned that he actually knew very little about either Haskell or Erlang. > On Mon, Jun 13, 2011 at 2:46, Jesper Louis Andersen > > > He is considering the "Computer Language benchmarks game", > > specifically the "fasta" benchmark on a Quad Core > > Fasta? What a strange choice! > > Oh! Could it be that none of the tests before Fasta show a C program > as fastest :-) > > > You can make the opposite conclusion by considering > > the thread-ring benchmark > > Another sequential program ;-) > > (At least spectral-norm keeps all the cores working.) From b.nicholson@REDACTED Wed Jun 15 17:58:08 2011 From: b.nicholson@REDACTED (Barry Nicholson) Date: Wed, 15 Jun 2011 10:58:08 -0500 Subject: [erlang-questions] wxWidgets In-Reply-To: <1050086879.302371308141341437.JavaMail.root@zimbra> References: <1050086879.302371308141341437.JavaMail.root@zimbra> Message-ID: <4DF8D690.7070407@niceng.com> I can only give you my experience with wxWidgets. I developed a small application with wxWidgets (SLOC - 198k). It was written in a much older version of wxWidgets but has been upgraded all the way to wxWidgets 2.8.10. The application was developed under VC 2005 on Windows. It's in daily use by approximately 10,000 users with no reported faults on every OS from Windows 95 (don't ask) all the way to Windows 7, and Windows 2000 Server to Windows 2008 Server. At no time have I had a crash that wasn't found to be my fault. But, I learned very early on that wxWidgets is like virtually all C/C++ GUI APIs. Handling multi-threading has to be handled very carefully. The application has 15 main threads interacting heavily on a core set of data. Changes in that data can cause MVC (model/view/controller) updates that reflect updates into the wxWidgets GUI world. It is very important that thread 'x' doesn't call any wxWidgets functions but instead send messages or signals of some kind to the 'single' GUI thread that handles all GUI updates. Forgetting that even once can cause a crash an hour later due to GUI data corruption. Experience was a great teacher. My gut feeling is that your idea of putting the GUI in a separate process would help. It would enforce a more single threaded view of the world in the GUI. But there's a lot of data that would have to be transferred back and forth to the wxWidgets process. As an example, my application has a wxGrid object (similar to a spreadsheet) that can have several thousand rows and columns with each cell being up to 1000 bytes. That ends up being megabytes of data that have to be transferred back and forth potentially multiple times a second. It can be done, just how to do it efficiently and fast. I'm not sure that answers all your questions, Robert but it's a start. Barry Nicholson On 6/15/11 7:35 AM, Robert Virding wrote: > The wx application binding of wxWidgets seems to be cause crashes on many systems, including MacOSX. After pondering on it I have a number of questions: > > Is it the wxWidget implementation itself which is buggy? > OR > Is the use of them directly in the BEAM which causes the crashes? Either through some inherent match problem, or version mismatch, or ...? > > If it is the second, would implementing a completely separate wx process solve the problem? I.e. would it be possible to implement such a process which would make linking with wxWidgets "safe"? > > If so, wouldn't it be better to do just that? Run wxWidgets in a separate OS process and use a (non-linked in driver) port to access it? Yes, it would be slower but it would work and be more robust. > > If this has already been discussed and my questions answered please point me in the right direction. > > Robert > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mihai@REDACTED Wed Jun 15 18:02:26 2011 From: mihai@REDACTED (Mihai Balea) Date: Wed, 15 Jun 2011 12:02:26 -0400 Subject: [erlang-questions] Kill process if message mailbox reaches a certain size (was discarding signals) In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> Message-ID: Hi all, Was reading the "discarding signals" thread and thought that maybe a better approach to handling this condition would be to kill the process if the mailbox exceeds a certain size. It seems to me that it would be a cleaner approach than flushing the mailbox and attempting to recover and it also fits better with Erlang's "let it crash" philosophy. Basically, the way I see it, we'd have this implemented at system level. One could set a flag, similar to, say, fullsweep_after - either per process or system wide. Default could be infinity, which reverts to the current behavior. Any thoughts on this? Any technical or philosophical arguments, pro or contra? For the record, I do believe that one should try as hard as possible to avoid getting into this situation to begin with, but occasionally it happens and it is one of Erlang's more spectacular failure modes. Cheers, Mihai >> If we are talking about an overloaded system that, for any reason, cannot tell the sender to reduce the traffic, then probably the only way to avoid a system crash is to discard the messages and notify the operators (raise an alarm, write event log, etc.) about this. So the discard shall be very quick as we are already overloaded. Do we have something for that? From max.lapshin@REDACTED Wed Jun 15 18:07:27 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 15 Jun 2011 20:07:27 +0400 Subject: [erlang-questions] Kill process if message mailbox reaches a certain size (was discarding signals) In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> Message-ID: I think we will stop in the same place, as with discussion: How not to kill erlang system due to OOM. From ulf.wiger@REDACTED Wed Jun 15 18:24:49 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 15 Jun 2011 18:24:49 +0200 Subject: [erlang-questions] Kill process if message mailbox reaches a certain size (was discarding signals) In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> Message-ID: <593ED5BC-610E-4809-A36F-5DF8C626BF38@erlang-solutions.com> I once made an experimental version of plain_fsm which could enforce a message queue limit. http://erlang.org/pipermail/erlang-questions/2008-June/035571.html I don't think I ever checked it in, but I remember that it wasn't terribly hard to implement. Plain_fsm itself nowadays lives at: http://github.com/esl/plain_fsm BR, Ulf On 15 Jun 2011, at 18:02, Mihai Balea wrote: > Hi all, > > Was reading the "discarding signals" thread and thought that maybe a better approach to handling this condition would be to kill the process if the mailbox exceeds a certain size. > It seems to me that it would be a cleaner approach than flushing the mailbox and attempting to recover and it also fits better with Erlang's "let it crash" philosophy. > > Basically, the way I see it, we'd have this implemented at system level. One could set a flag, similar to, say, fullsweep_after - either per process or system wide. Default could be infinity, which reverts to the current behavior. > > Any thoughts on this? Any technical or philosophical arguments, pro or contra? > > For the record, I do believe that one should try as hard as possible to avoid getting into this situation to begin with, but occasionally it happens and it is one of Erlang's more spectacular failure modes. > > Cheers, > Mihai > > >>> If we are talking about an overloaded system that, for any reason, cannot tell the sender to reduce the traffic, then probably the only way to avoid a system crash is to discard the messages and notify the operators (raise an alarm, write event log, etc.) about this. So the discard shall be very quick as we are already overloaded. Do we have something for that? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Wed Jun 15 18:32:23 2011 From: mihai@REDACTED (Mihai Balea) Date: Wed, 15 Jun 2011 12:32:23 -0400 Subject: [erlang-questions] Kill process if message mailbox reaches a certain size (was discarding signals) In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> Message-ID: On Jun 15, 2011, at 12:07 PM, Max Lapshin wrote: > I think we will stop in the same place, as with discussion: How not to > kill erlang system due to OOM. Are you referring to this thread? http://erlang.2086793.n4.nabble.com/Why-Beam-smp-crashes-when-memory-is-over-tt2118397.html#none Mihai From jesper.louis.andersen@REDACTED Wed Jun 15 19:08:43 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 15 Jun 2011 19:08:43 +0200 Subject: [erlang-questions] Gotta be an easier way... In-Reply-To: References: Message-ID: On Mon, Jun 13, 2011 at 23:34, Mike Oxford wrote: > Orig#v1_rec_system_message_user_logon{header=Orig#v1_rec_system_message_user_logon.header#v1_rec_header{route=Orig#v1_rec_system_message_user_logon.header#v1_rec_header.route#v1_rec_route{route_entries=[foo]}}}. Ugh. You could wrap this up in a helper-function and then call the helper when you want to do the update. The inliner should make short work of any trouble there with call overhead. > Shortening variable names, or writing accessor macros for everything > are possibilities ... but is there a better way I'm missing? ?(Don't > use nested records is a poor response.) You may claim that "use something but nested records" is a poor response. But if you have that requirement there is not much we can do. Almost every problem with updates in functional programming can be solved by reordering the structure of your data. In your case you could either place the variable you are updating close to the top-level or outright flatten the whole structure completely. You could also look into alternate data structures and use those. Nobody says your structure should reflect an external format, you can just render it as such. The main problem is that you need to qualify the records so access can be resolved at compile time. Avoiding that means either packing the access away in functions (my first suggestion) or reordering the data (my second suggestion). Note that matching can help quite a lot though: set_route(#system_message { header = Header } = Orig, Val) -> Orig#system_message { header = set_route(Header, Val) }; set_route(#rec_header { route = Route } = Orig, Val) -> Orig#rec_header { route = set_route(Route, Val) }; set_route(#rec_route { } = Route, Val) -> Route#rec_route { route_entries = Val }. ... But you are probably still better off by altering the structure of the code. Represent the data in another way and then finally build up your record structure in one go where everything is correctly set. This avoids the continual "set this and that in the structure" which is effective in a language with an ephemeral heap, but is somewhat wasteful in a language where the heap is persistent -- like Erlang. -- J. From max.lapshin@REDACTED Wed Jun 15 19:25:23 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 15 Jun 2011 21:25:23 +0400 Subject: [erlang-questions] Kill process if message mailbox reaches a certain size (was discarding signals) In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> Message-ID: On Wed, Jun 15, 2011 at 8:32 PM, Mihai Balea wrote: > Are you referring to this thread? > > http://erlang.2086793.n4.nabble.com/Why-Beam-smp-crashes-when-memory-is-over-tt2118397.html#none exactly From dangud@REDACTED Wed Jun 15 21:35:17 2011 From: dangud@REDACTED (Dan Gudmundsson) Date: Wed, 15 Jun 2011 21:35:17 +0200 Subject: [erlang-questions] wxWidgets In-Reply-To: <1050086879.302371308141341437.JavaMail.root@zimbra> References: <63334269.302261308140350677.JavaMail.root@zimbra> <1050086879.302371308141341437.JavaMail.root@zimbra> Message-ID: On Wed, Jun 15, 2011 at 2:35 PM, Robert Virding < robert.virding@REDACTED> wrote: > The wx application binding of wxWidgets seems to be cause crashes on many > systems, including MacOSX. After pondering on it I have a number of > questions: > > Is it the wxWidget implementation itself which is buggy? > OR > Is the use of them directly in the BEAM which causes the crashes? Either > through some inherent match problem, or version mismatch, or ...? > > If it is the second, would implementing a completely separate wx process > solve the problem? I.e. would it be possible to implement such a process > which would make linking with wxWidgets "safe"? > > If so, wouldn't it be better to do just that? Run wxWidgets in a separate > OS process and use a (non-linked in driver) port to access it? Yes, it would > be slower but it would work and be more robust. > > If this has already been discussed and my questions answered please point > me in the right direction. > > In most cases it is a bug in the erlang program, then in my driver code, and a few times it's in wxWidgets. So mostly it's about testing your application extensively on every platform. They all behave slightly different, if you program by the book it works everywhere, but that is hard since both wxWidgets (and especially wx) lacks good documentation, so it is always a little trial and error. In my experience the main problem is deleting objects, add the same object twice to a window somewhere and delete the window, wxWidgets will cleanup and destroy the object twice and erlang will crash. Or you delete the object yourself but wx has a reference to it somewhere which also deletes it. My opinion is that if you have a mission critical application you don't run the gui on that node, distributed programming is easy in erlang so use a separate node to connect your gui to the mission critical system. Then if the gui crashes it still is a bug whether you have an erlang prompt with a non descriptive 'EXIT' message from the port or not. Which is why I chose to write it as a driver in the first place. I think extending gs and writing a backend for wx is a good idea, where can hopefully safe guard the programmer from some errors. If you want to have it as a port program as an option, it is doable but a lot of work, though we accept patches :-) /Dan PS: I still haven't had time to look at your reltool crash, sorry about that. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Wed Jun 15 22:27:44 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 15 Jun 2011 13:27:44 -0700 Subject: [erlang-questions] Oauth 2.0 support In-Reply-To: <8980542.5113221308122823278.JavaMail.defaultUser@defaultHost> References: <8980542.5113221308122823278.JavaMail.defaultUser@defaultHost> Message-ID: I think oauth 2.0 is entirely an application level feature. It requires application-specific storage, and application-specific implementation in what you delegate. That being said, building oauth 2.0 clients is super easy, and building 2.0 servers is not hard if you already have a sane user authentication and authorization system, because it all just relies on https. I've never found oauth 2.0 "helper libraries" to be much help, personally. Jw On Jun 15, 2011 12:27 AM, "matteo.redaelli@REDACTED" < matteo.redaelli@REDACTED> wrote: > Hi all > > What about Oauth 2.0 support? Is httpc a good place for it? > > Please read this thread https://github.com/tim/erlang- > oauth/issues/4#comment_1218849 > > Regards > Matteo > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Wed Jun 15 22:36:34 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 15 Jun 2011 13:36:34 -0700 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <4DF89262.5060500@cs.ntua.gr> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> Message-ID: Isn't that almost exactly what the lesser/limited/library gpl is for? However, the "all patent rights may become contested" parts of gplv3 and mpl make those licenses unlikely to be used by real companies, even if they generally support sharing source. Else one of my competitors can take something I release, use it in a system infringing my product patents, and life becomes really complex really quickly. I would recommend upgrading to lgpl v2.1, or even BSD/MIT, if you really care about people being free to use PropEr for real. As it is, gpl3 based software is not "free" (as in speech) at all for most people. Anyway, it sounds as if the "L" addition might solve the stated problem, assuming no desired user/collaborator has any patents they care about. Jw On Jun 15, 2011 4:12 AM, "Kostis Sagonas" wrote: > Torben Hoffmann wrote: >> I am afraid that the GPLv3 is contaminating in this case ... >> >> However, as Eric mentioned earlier, the PropEr team is looking into >> adding a FOSS License Exception and now that the big v1.0 is behind them >> they hopefully get the time need to investigate this and see if it is >> possible to do. >> >> The PropEr team has been very approachable about this issue so I think >> we should just wait for them to find the time it takes to get this done >> right. > > Thanks for this comment! > > We have discussed the issue extensively yesterday and today we wrote a > mail to the Free Software Foundation to also ask their opinion and their > suggestion on how to properly deal with the issues involved. > > Quoting from the mail we sent to FSF: > > Our primary concern is that PropEr be as useful as possible to FOSS > programmers. Therefore, the most important requirement from PropEr's > license is that it allows FOSS projects (regardless of license) to: > > * use PropEr without restrictions, and especially without needing to > switch to another license > * release their testing code under whatever license they choose > * distribute PropEr freely, to be used as part of the build cycle of > the project, or for other uses (e.g. if the project in question is > an Erlang IDE, it could distribute PropEr as a plugin), again > without needing to switch licenses > > The above benefits need *not* extend to developers of proprietary > software. If PropEr's license could restrict them in some way, that > would be an added bonus, but this is not a strict requirement. > > Another requirement is that PropEr's license is a (preferably strong) > copyleft: we would like to guarantee that any extensions to PropEr are > also free software. > > // ... We also told them about the issue that the GPL is supposedly > // unusable for Erlang programs due to the nature of Erlang. > > In short, we are looking for a solution/license that will allow at least > all open-source projects to use PropEr without any restrictions or by > being contaminated by GPL. > > I'll be surprised if we do not find a solution, but please bear with us > until we resolve this the proper way. > > In the meantime, you can use the software and the proper e-mail address > for questions and comments. > > Kostis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From fred.hebert@REDACTED Wed Jun 15 22:39:18 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Wed, 15 Jun 2011 16:39:18 -0400 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> Message-ID: <490AD11D-50EE-478A-B173-8FBD59D92205@erlang-solutions.com> But BSD and MIT do allow commercial users to change and extend the software, write extensions in ways that they can be closed. This is something the PropEr developers want to avoid given the previous messages. So clearly, BSD, MIT and DWTFYW licenses are not acceptable. -- Fred H?bert http://www.erlang-solutions.com On 2011-06-15, at 16:36 PM, Jon Watte wrote: > Isn't that almost exactly what the lesser/limited/library gpl is for? > > However, the "all patent rights may become contested" parts of gplv3 and mpl make those licenses unlikely to be used by real companies, even if they generally support sharing source. Else one of my competitors can take something I release, use it in a system infringing my product patents, and life > becomes really complex really quickly. I would recommend upgrading to lgpl v2.1, or even BSD/MIT, if you really care about people being free to use PropEr for real. As it is, gpl3 based software is not "free" (as in speech) at all for most people. > > Anyway, it sounds as if the "L" addition might solve the stated problem, assuming no desired user/collaborator has any patents they care about. > > Jw > On Jun 15, 2011 4:12 AM, "Kostis Sagonas" wrote: > > Torben Hoffmann wrote: > >> I am afraid that the GPLv3 is contaminating in this case ... > >> > >> However, as Eric mentioned earlier, the PropEr team is looking into > >> adding a FOSS License Exception and now that the big v1.0 is behind them > >> they hopefully get the time need to investigate this and see if it is > >> possible to do. > >> > >> The PropEr team has been very approachable about this issue so I think > >> we should just wait for them to find the time it takes to get this done > >> right. > > > > Thanks for this comment! > > > > We have discussed the issue extensively yesterday and today we wrote a > > mail to the Free Software Foundation to also ask their opinion and their > > suggestion on how to properly deal with the issues involved. > > > > Quoting from the mail we sent to FSF: > > > > Our primary concern is that PropEr be as useful as possible to FOSS > > programmers. Therefore, the most important requirement from PropEr's > > license is that it allows FOSS projects (regardless of license) to: > > > > * use PropEr without restrictions, and especially without needing to > > switch to another license > > * release their testing code under whatever license they choose > > * distribute PropEr freely, to be used as part of the build cycle of > > the project, or for other uses (e.g. if the project in question is > > an Erlang IDE, it could distribute PropEr as a plugin), again > > without needing to switch licenses > > > > The above benefits need *not* extend to developers of proprietary > > software. If PropEr's license could restrict them in some way, that > > would be an added bonus, but this is not a strict requirement. > > > > Another requirement is that PropEr's license is a (preferably strong) > > copyleft: we would like to guarantee that any extensions to PropEr are > > also free software. > > > > // ... We also told them about the issue that the GPL is supposedly > > // unusable for Erlang programs due to the nature of Erlang. > > > > In short, we are looking for a solution/license that will allow at least > > all open-source projects to use PropEr without any restrictions or by > > being contaminated by GPL. > > > > I'll be surprised if we do not find a solution, but please bear with us > > until we resolve this the proper way. > > > > In the meantime, you can use the software and the proper e-mail address > > for questions and comments. > > > > Kostis > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Wed Jun 15 22:45:15 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 15 Jun 2011 13:45:15 -0700 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <490AD11D-50EE-478A-B173-8FBD59D92205@erlang-solutions.com> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <490AD11D-50EE-478A-B173-8FBD59D92205@erlang-solutions.com> Message-ID: Yes. To clarify: My message had two parts: 1) LGPL will solve the stated problem according to the current definition of "acceptable." 2) An argument for changing the definition of "acceptable." If I want to contribute to PropEr, I can't, because of license. For me, that means the license is not free at all. Jw On Jun 15, 2011 1:39 PM, "Fr?d?ric Trottier-H?bert" < fred.hebert@REDACTED> wrote: > But BSD and MIT do allow commercial users to change and extend the software, write extensions in ways that they can be closed. This is something the PropEr developers want to avoid given the previous messages. So clearly, BSD, MIT and DWTFYW licenses are not acceptable. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-06-15, at 16:36 PM, Jon Watte wrote: > >> Isn't that almost exactly what the lesser/limited/library gpl is for? >> >> However, the "all patent rights may become contested" parts of gplv3 and mpl make those licenses unlikely to be used by real companies, even if they generally support sharing source. Else one of my competitors can take something I release, use it in a system infringing my product patents, and life >> becomes really complex really quickly. I would recommend upgrading to lgpl v2.1, or even BSD/MIT, if you really care about people being free to use PropEr for real. As it is, gpl3 based software is not "free" (as in speech) at all for most people. >> >> Anyway, it sounds as if the "L" addition might solve the stated problem, assuming no desired user/collaborator has any patents they care about. >> >> Jw >> On Jun 15, 2011 4:12 AM, "Kostis Sagonas" wrote: >> > Torben Hoffmann wrote: >> >> I am afraid that the GPLv3 is contaminating in this case ... >> >> >> >> However, as Eric mentioned earlier, the PropEr team is looking into >> >> adding a FOSS License Exception and now that the big v1.0 is behind them >> >> they hopefully get the time need to investigate this and see if it is >> >> possible to do. >> >> >> >> The PropEr team has been very approachable about this issue so I think >> >> we should just wait for them to find the time it takes to get this done >> >> right. >> > >> > Thanks for this comment! >> > >> > We have discussed the issue extensively yesterday and today we wrote a >> > mail to the Free Software Foundation to also ask their opinion and their >> > suggestion on how to properly deal with the issues involved. >> > >> > Quoting from the mail we sent to FSF: >> > >> > Our primary concern is that PropEr be as useful as possible to FOSS >> > programmers. Therefore, the most important requirement from PropEr's >> > license is that it allows FOSS projects (regardless of license) to: >> > >> > * use PropEr without restrictions, and especially without needing to >> > switch to another license >> > * release their testing code under whatever license they choose >> > * distribute PropEr freely, to be used as part of the build cycle of >> > the project, or for other uses (e.g. if the project in question is >> > an Erlang IDE, it could distribute PropEr as a plugin), again >> > without needing to switch licenses >> > >> > The above benefits need *not* extend to developers of proprietary >> > software. If PropEr's license could restrict them in some way, that >> > would be an added bonus, but this is not a strict requirement. >> > >> > Another requirement is that PropEr's license is a (preferably strong) >> > copyleft: we would like to guarantee that any extensions to PropEr are >> > also free software. >> > >> > // ... We also told them about the issue that the GPL is supposedly >> > // unusable for Erlang programs due to the nature of Erlang. >> > >> > In short, we are looking for a solution/license that will allow at least >> > all open-source projects to use PropEr without any restrictions or by >> > being contaminated by GPL. >> > >> > I'll be surprised if we do not find a solution, but please bear with us >> > until we resolve this the proper way. >> > >> > In the meantime, you can use the software and the proper e-mail address >> > for questions and comments. >> > >> > Kostis >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel@REDACTED Wed Jun 15 22:57:33 2011 From: daniel@REDACTED (Daniel Luna) Date: Wed, 15 Jun 2011 22:57:33 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <4DF89262.5060500@cs.ntua.gr> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> Message-ID: On 15 June 2011 13:07, Kostis Sagonas wrote: > ?// ... We also told them about the issue that the GPL is supposedly > ?// ? ? unusable for Erlang programs due to the nature of Erlang. (Hoping that I'm now properly subscribed so that this message actually reaches the list) The GNU GPL has an exception for "system libraries" and in version 3 that now explicitly includes the standard libraries of common programming libraries. So Erlang should be fine with version 3. The Quick Guide on gnu.org has a short section about that exception here: http://www.gnu.org/licenses/quick-guide-gplv3.html#less-source-to-distribute-new-system-libraries-exception /Daniel From overminddl1@REDACTED Thu Jun 16 00:43:48 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Wed, 15 Jun 2011 15:43:48 -0700 (PDT) Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> Message-ID: <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> As for a non-license related question, how does PropEr compare to http://krestenkrab.github.com/triq/ in features, speed, reducibility, etc...? On Jun 15, 2:57?pm, Daniel Luna wrote: > /* snip */ From dan@REDACTED Thu Jun 16 02:47:53 2011 From: dan@REDACTED (Daniel Dormont) Date: Wed, 15 Jun 2011 20:47:53 -0400 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> Message-ID: I'm not a legal expert, but I did study some of this licensing business some time ago, so just a couple of things to bear in mind: - the question of what constitutes a derived work is an issue under copyright law and may vary by jurisdiction; it is not up to the author of the license to define that. See for example http://en.wikipedia.org/wiki/GNU_General_Public_License#Point_of_view:_dynamically_and_static-linking_violates_GPL - I don't imagine most proprietary software programs ship with unit tests included. Then again, I could be wrong :) dan On Wed, Jun 15, 2011 at 4:57 PM, Daniel Luna wrote: > On 15 June 2011 13:07, Kostis Sagonas wrote: > > // ... We also told them about the issue that the GPL is supposedly > > // unusable for Erlang programs due to the nature of Erlang. > > (Hoping that I'm now properly subscribed so that this message actually > reaches the list) > > The GNU GPL has an exception for "system libraries" and in version 3 > that now explicitly includes the standard libraries of common > programming libraries. So Erlang should be fine with version 3. > > The Quick Guide on gnu.org has a short section about that exception > here: > http://www.gnu.org/licenses/quick-guide-gplv3.html#less-source-to-distribute-new-system-libraries-exception > > /Daniel > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jameschurchman@REDACTED Thu Jun 16 03:18:59 2011 From: jameschurchman@REDACTED (James Churchman) Date: Thu, 16 Jun 2011 02:18:59 +0100 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: in that its apache 2.0 :-) > As for a non-license related question, how does PropEr compare to > http://krestenkrab.github.com/triq/ in features, speed, reducibility, > etc...? Is this a fact or a stated FSF opinion? seems counter intuitive that something that writes to an api ( even if it's dynamically linked ) can be considered an alteration of derived work, as what is to stop someone producing an identical api with the same, or indeed a different set of behaviour, does the existence of this "new" api now change things. Better still what if it targets an intermediate api. In that case if all calls in your code go to notgplmodule:do() which is a dual licensed opensource project, which calls gplv3 code, what happens then?? James > I'm not a legal expert, but I did study some of this licensing business some time ago, so just a couple of things to bear in mind: > > - the question of what constitutes a derived work is an issue under copyright law and may vary by jurisdiction; it is not up to the author of the license to define that. See for example http://en.wikipedia.org/wiki/GNU_General_Public_License#Point_of_view:_dynamically_and_static-linking_violates_GPL > > - I don't imagine most proprietary software programs ship with unit tests included. Then again, I could be wrong :) > > dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From jozsef.berces@REDACTED Thu Jun 16 05:11:03 2011 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces?=) Date: Thu, 16 Jun 2011 05:11:03 +0200 Subject: [erlang-questions] Kill process if message mailbox reaches a certain size (was discarding signals) In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> Message-ID: <3717CEF51B134A4F8073627839A9DF4120F405A9F8@ESESSCMS0353.eemea.ericsson.se> Thanks for all the thoughts and suggestions. If I got it right, there were two main branches: 1. Avoid the congestion situation 2. Detect and kill/restart the problematic process(es) The problem with these approaches that the Erlang applications are not just playing with themselves but receive input from other nodes. Those nodes can be very numerous and uncontrollable. As an example, just let's take the mobile network where the traffic is generated by millions of subscribers using mobile devices from many vendors. In this case we (1) cannot control the volume of the traffic and (2) cannot make sure that all the devices follow the protocol. So there can be situations when we cannot avoid congestion simply because the source of the traffic is beyond our reach. Killing and restarting is not the right way either: - A restart causes total outage for a while that is very unwelcome by the users (e.g. network operators) of our boxes - Erlang is advertised to be robust but killing and restarting is not a sign of robustness. So the user can easily call us liar: "You say your node is robust but it is restarting frequently!" So I still believe that very quick discard of the signals is a key for real robustness. Obviously, it shall be used *only* in the right circumstances, but in those cases that would be the only way to keep the node alive and minimize the traffic loss. Then the question is still open: Discarding 1-by-1 is the best what we can do or there is something more efficient to get rid of the excess traffic? -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Max Lapshin Sent: Thursday, June 16, 2011 0:25 To: Mihai Balea Cc: erlang-questions Questions Subject: Re: [erlang-questions] Kill process if message mailbox reaches a certain size (was discarding signals) On Wed, Jun 15, 2011 at 8:32 PM, Mihai Balea wrote: > Are you referring to this thread? > > http://erlang.2086793.n4.nabble.com/Why-Beam-smp-crashes-when-memory-i > s-over-tt2118397.html#none exactly _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From bob@REDACTED Thu Jun 16 05:25:42 2011 From: bob@REDACTED (Bob Ippolito) Date: Wed, 15 Jun 2011 20:25:42 -0700 Subject: [erlang-questions] Kill process if message mailbox reaches a certain size (was discarding signals) In-Reply-To: <3717CEF51B134A4F8073627839A9DF4120F405A9F8@ESESSCMS0353.eemea.ericsson.se> References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A9F8@ESESSCMS0353.eemea.ericsson.se> Message-ID: On Wed, Jun 15, 2011 at 8:11 PM, J?zsef B?rces wrote: > Thanks for all the thoughts and suggestions. If I got it right, there were two main branches: > > 1. Avoid the congestion situation > 2. Detect and kill/restart the problematic process(es) > > The problem with these approaches that the Erlang applications are not just playing with themselves but receive input from other nodes. Those nodes can be very numerous and uncontrollable. > > As an example, just let's take the mobile network where the traffic is generated by millions of subscribers using mobile devices from many vendors. In this case we (1) cannot control the volume of the traffic and (2) cannot make sure that all the devices follow the protocol. > So there can be situations when we cannot avoid congestion simply because the source of the traffic is beyond our reach. The Erlang distribution protocol is only suitable for connecting a relatively small number of trusted nodes on a LAN. If you were to expertly implement such an application you would have some Erlang nodes speaking to these mobile devices, but with another protocol (probably over TCP), and then you would have as much control as you need over the other details. For example, you can avoid congestion by rate limiting or refusing to accept new connections. When the Erlang nodes speak to each other (with or without Erlang distribution), you also control that protocol and can avoid congestion there as well. -bob From banibrata.dutta@REDACTED Thu Jun 16 06:06:58 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Thu, 16 Jun 2011 09:36:58 +0530 Subject: [erlang-questions] Kill process if message mailbox reaches a certain size (was discarding signals) In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A9F8@ESESSCMS0353.eemea.ericsson.se> Message-ID: On Thu, Jun 16, 2011 at 8:55 AM, Bob Ippolito wrote: > On Wed, Jun 15, 2011 at 8:11 PM, J?zsef B?rces > wrote: > > Thanks for all the thoughts and suggestions. If I got it right, there > were two main branches: > > > > 1. Avoid the congestion situation > > 2. Detect and kill/restart the problematic process(es) > > > > The problem with these approaches that the Erlang applications are not > just playing with themselves but receive input from other nodes. Those nodes > can be very numerous and uncontrollable. > > > > As an example, just let's take the mobile network where the traffic is > generated by millions of subscribers using mobile devices from many vendors. > In this case we (1) cannot control the volume of the traffic and (2) cannot > make sure that all the devices follow the protocol. > > So there can be situations when we cannot avoid congestion simply because > the source of the traffic is beyond our reach. > > The Erlang distribution protocol is only suitable for connecting a > relatively small number of trusted nodes on a LAN. > I think I get the jist of it, but could someone quantify, as to how "small" is "relatively small number" here ? Fifty, few hundreds, couple of thousands ? What is the largest 'Erlang cloud' (i.e. hosts running Erlang processes communicating accross nodes in a cluster), that has been seen ? If you were to expertly implement such an application you would have > some Erlang nodes speaking to these mobile devices, but with another > protocol (probably over TCP), and then you would have as much control > as you need over the other details. For example, you can avoid > congestion by rate limiting or refusing to accept new connections. > When the Erlang nodes speak to each other (with or without Erlang > distribution), you also control that protocol and can avoid congestion > there as well. > In telecom world such a situation is pretty common, however think of the situation that even to discard a message (due to congestion) that starts a new transaction if I need to determine things like priority, transaction-id or application level session-id etc., I'd have to have the ability to decode that much message in the rate-limiter process, which I think we are saying, will be written in C/C++ and communicating over IP or another protocol. Duplicating the decode logic, I guess would be unavoidable, in most such cases. Or is there a better behavioral pattern someone has figured out ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Thu Jun 16 06:15:04 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jun 2011 08:15:04 +0400 Subject: [erlang-questions] Kill process if message mailbox reaches a certain size (was discarding signals) In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A9F8@ESESSCMS0353.eemea.ericsson.se> Message-ID: On Jun 16, 2011, at 8:06 AM, Banibrata Dutta wrote: > I think I get the jist of it, but could someone quantify, as to how "small" is "relatively small number" here ? Fifty, few hundreds, couple of thousands ? > What is the largest 'Erlang cloud' (i.e. hosts running Erlang processes communicating accross nodes in a cluster), that has been seen ? I had a cluster of 150 nodes on Amazon EC2, one node per small instance. That was too much with mesh connections between nodes, e.g. processes exchanging messages everywhere. Heavy message traffic between nodes caused frequent network splits (partitions). I had to resort to limiting traffic to make it manageable. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From jwatte@REDACTED Thu Jun 16 07:09:04 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 15 Jun 2011 22:09:04 -0700 Subject: [erlang-questions] Kill process if message mailbox reaches a certain size (was discarding signals) In-Reply-To: <3717CEF51B134A4F8073627839A9DF4120F405A9F8@ESESSCMS0353.eemea.ericsson.se> References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A9F8@ESESSCMS0353.eemea.ericsson.se> Message-ID: I sense a few misconceptions here. First, I assume that the proposal for "kill and restart" is to kill and restart the particular process that has an inbox that is "too full." This does not mean that the client crashes -- it means that those messages are lost. This is no different to how Erlang systems are robust in the face of rare crashing bugs (as opposed to, say, C++, where generally the entire system goes down because of a rare stray pointer bug, for example). Basically, crashing and re-starting an Erlang worker process is just one way of clearing out the message queue, and also making sure that any possible state corruption goes away because the process re-starts "afresh." The Erlang/OTP supervision tree is designed to work in this mode. Second, when you have an amount of load that comes in, and you cannot control it, then what is generally done is to simply model the load, model the application, and provision enough server hardware that you can keep up with the load. In an emergency (an unexpected surge that doubles load compared to anything seen before), you'll additionally want capabilities to reject some part of the incoming requests. For HTTP, this is where status 503 (Server Busy) comes in, for example. I'm assuming all your clients use some common protocol, like TCP or HTTPS or whatever, and that you do appropriate protection against un-trusted data at that layer. When it comes to cluster sizes, we're running a 11 node cluster with >100,000 users and it's running mostly idle on a gigabit switch. I would consider this a "smallish cluster." We're planning on increasing our data rates a lot in the future, though -- at some point, we'll need to provision to 10 GBps. We scale using a crossbar and consistent hasning. I've heard of clusters that do a million users per node, and use broadcast to all other nodes in a cluster of 50 nodes. That also scales on available networking hardware, as long as most users are not generators of large or frequent packets. I would advise against single-core nodes or cloud-based nodes that don't have local networking, because these get much less work done per node (and per network packet) than larger systems. Buying a single server from Dell today, you get 12 cores and 24 hardware threads even on the low end. Next year, that number will be 40, 80 or even 160 (for the higher end). So, in your case, I would suggest making sure that you know what the protocol is that clients use to connect to the server, and then making sure that you have some way of reporting temporary capacity overload to the clients, and then making sure that you have good metrics on the utilization of the server cluster (CPU, memory, network bandwidth, etc) so that you can put in more hardware when needed. If CPU goes 100% for a long time (which would be a precondition for a queue to fill up), start rejecting requests and log an alert for the operator to buy more hardware. I also recommend modeling the traffic across the backplane of the Erlang nodes. How much data do you send per user "event" to other users, and how many other users? Broadcast or point-to-point? Sum it all up, double it, and see if you can still swing that on your current network backplane. If not, buy a bigger network, or start working on ways to compress/reduce the data stream :-) Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Wed, Jun 15, 2011 at 8:11 PM, J?zsef B?rces wrote: > Thanks for all the thoughts and suggestions. If I got it right, there were > two main branches: > > 1. Avoid the congestion situation > 2. Detect and kill/restart the problematic process(es) > > The problem with these approaches that the Erlang applications are not just > playing with themselves but receive input from other nodes. Those nodes can > be very numerous and uncontrollable. > > As an example, just let's take the mobile network where the traffic is > generated by millions of subscribers using mobile devices from many vendors. > In this case we (1) cannot control the volume of the traffic and (2) cannot > make sure that all the devices follow the protocol. > So there can be situations when we cannot avoid congestion simply because > the source of the traffic is beyond our reach. > > Killing and restarting is not the right way either: > - A restart causes total outage for a while that is very unwelcome by the > users (e.g. network operators) of our boxes > - Erlang is advertised to be robust but killing and restarting is not a > sign of robustness. > So the user can easily call us liar: "You say your node is robust but it > is restarting frequently!" > > So I still believe that very quick discard of the signals is a key for real > robustness. Obviously, it shall be used *only* in the right circumstances, > but in those cases that would be the only way to keep the node alive and > minimize the traffic loss. > > Then the question is still open: Discarding 1-by-1 is the best what we can > do or there is something more efficient to get rid of the excess traffic? > > > -----Original Message----- > From: erlang-questions-bounces@REDACTED [mailto: > erlang-questions-bounces@REDACTED] On Behalf Of Max Lapshin > Sent: Thursday, June 16, 2011 0:25 > To: Mihai Balea > Cc: erlang-questions Questions > Subject: Re: [erlang-questions] Kill process if message mailbox reaches a > certain size (was discarding signals) > > On Wed, Jun 15, 2011 at 8:32 PM, Mihai Balea wrote: > > > Are you referring to this thread? > > > > http://erlang.2086793.n4.nabble.com/Why-Beam-smp-crashes-when-memory-i > > s-over-tt2118397.html#none > > exactly > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From octopusfluff@REDACTED Thu Jun 16 07:38:11 2011 From: octopusfluff@REDACTED (Amy Lear) Date: Wed, 15 Jun 2011 22:38:11 -0700 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: On Wed, Jun 15, 2011 at 6:18 PM, James Churchman wrote: > > - the question of what constitutes a derived work is an issue under > > copyright law and may vary by jurisdiction; it is not up to the author of > > the license to define that. See for > > example http://en.wikipedia.org/wiki/GNU_General_Public_License#Point_of_view:_dynamically_and_static-linking_violates_GPL > Is this a fact or a stated FSF opinion? seems counter intuitive that > something that writes to an api ( even if it's dynamically linked ) can be > considered an alteration of derived work, as what is to stop someone > producing an identical api with the same, or indeed a different set of > behaviour, does the existence of this "new" api now change things. > Better still what if it targets an intermediate api. In that case if all > calls in your code go to notgplmodule:do() which is a dual licensed > opensource project, which calls gplv3 code, what happens then?? > James Magical Things Happen. The tendency for linking to GPL works being required to be GPL works is a complicated thing that gets contested in different ways in different places. Making an intermediate API to present a non-GPL interface to proprietary code is a pretty well established technique. See: some video card drivers in Linux, with having a GPL compatible bit that just creates an interface for the non-GPL compatible bits to later use. Making an identical API to an existing GPL based API also complicates things. >From here, the only correct answer is "Consult a lawyer in your jurisdiction." Unfortunate, but that's how it goes. From max.lapshin@REDACTED Thu Jun 16 08:13:18 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 16 Jun 2011 10:13:18 +0400 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: Damn, what a problem in GPL for a testing program? Guys just want to protect their investment, but want to make a free advertisement by offering their excelent product to community. If it would be MIT, they can see their work in commercial product without any profit for them. There is absolutely no problem in using GPL software to test your products: just don't include it into distribution. Tests are not part of PropEr, but if any of you is afraid of GPL virus, so don't include tests into your main repository: keep it separate. For example, I don't include test files for erlyvideo tests into repository, because I don't want accidential discussions with their potential copyright holders. Btw, about PropEr. It is so widely told, that it is a magic tool, that can remove all bugs in software, that I'm very interested to try it. I will try to find time for it and tell if it helped me with erlyvideo or not. From vladdu55@REDACTED Thu Jun 16 08:30:28 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 16 Jun 2011 08:30:28 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: Hi, This kind of issues are worse than flame wars! :-) I have yet another point of view that increases the confusion: PropEr is mostly compatible with Triq and QuickCheck (there's a free older version). Each tool might have their extensions, but the basic functionality is the same and it should be possible to run basic tests with any one of them (if not at the moment, then in the future). Now if I distribute property-based tests with my open-sourced application, the users could in principle use any one of the tools to execute them. In this case, would the tests still be affected by the fact that PropEr is GPL? If I have tests that depend on a certain behaviour that only PropEr has, and those tests are ifdef'd, does that change the answer to the previous question? best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Jun 16 08:43:14 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 16 Jun 2011 10:43:14 +0400 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: Of course, tests aren't covered with the same license. GCC is licensed under GPL. You are writing C code, which has ifdefs for GCC and for MSVC. How can you imagine, that someone will order you to release your program under GPL? PropEr is just a tool. If you distribute this tool, you must share sources. You may sell it, but with opened sources. But you don't have any restrictions about files, that this tool works with. From ulf.wiger@REDACTED Thu Jun 16 08:43:23 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 16 Jun 2011 08:43:23 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: On 16 Jun 2011, at 08:30, Vlad Dumitrescu wrote: > Hi, > > This kind of issues are worse than flame wars! :-) > > I have yet another point of view that increases the confusion: PropEr is mostly compatible with Triq and QuickCheck (there's a free older version). Well, there is a version available in jungerl, but to my knowledge, it was not put there by the authors, and not actually intended to be free. Even so, one might consider it a lot less sensitive to copy that version than the later, proprietary, versions of QuickCheck. "Inspired by QuickCheck" is the wording in the README, or more specifically: "We have generally tried to keep PropEr's notation and output format as compatible as possible with QuviQ's QuickCheck, to allow for the reuse of existing testing code written for that tool." (https://github.com/manopapad/proper/blob/master/README.md) As far as I can tell, PropEr is an unauthorised reverse-engineering effort of the later commercial versions (the early version did not have shrinking or statem support, for example); even if it *had* been in line with EU copyright protections (which allow reverse-engineering for personal purposes, given that a legal copy has been obtained in the first place), publication of the resulting product is not permitted. I don't know EU copyright laws well enough to know how this affects the legal implications of any PropEr license. I know that if someone would steal a physical item from me, by law I cannot simply steal it back; legal action is required. BR, Ulf This is the Wikipedia version of what the EU has to say about reverse-engineering proprietary software: "Article 6 of the 1991 EU Computer Programs Directive allows reverse engineering for the purposes of interoperability, but prohibits it for the purposes of creating a competing product, and also prohibits the public release of information obtained through reverse engineering of software.[24][25][26] In 2009, the EU Computer Program Directive was superseded and the directive now states:[27] (15) The unauthorised reproduction, translation, adaptation or transformation of the form of the code in which a copy of a computer program has been made available constitutes an infringement of the exclusive rights of the author. Nevertheless, circumstances may exist when such a reproduction of the code and translation of its form are indispensable to obtain the necessary infor mation to achieve the interoperability of an indepen dently created program with other programs. It has therefore to be considered that, in these limited circum stances only, performance of the acts of reproduction and translation by or on behalf of a person having a right to use a copy of the program is legitimate and compatible with fair practice and must therefore be deemed not to require the authorisation of the right holder. An objective of this exception is to make it possible to connect all components of a computer system, including those of different manufacturers, so that they can work together. Such an exception to the author's exclusive rights may not be used in a way which prejudices the legitimate interests of the rightholder or which conflicts with a normal exploitation of the program." (http://en.wikipedia.org/wiki/Reverse_engineering) I assume that reverse-engineering in order to form a basis for research could be said to be in line with the above*, but if the expressed purpose is interoperability with QuickCheck, it seems logical that one should choose a license that encourages future interoperability as well, e.g. by making it easy for Quviq QuickCheck to incorporate innovations from the PropEr side, and contributing back. * Except the "prohibits the public release of information obtained through reverse engineering of software", which is a bit difficult to get around. Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Thu Jun 16 08:48:30 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 15 Jun 2011 23:48:30 -0700 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: As a commercial entity with (unrelated) IP to protect, I may be able to USE GPLv3 software, as long as I don't distribute to others (free or not), but there is no way I can CONTRIBUTE to GPL v3, because of the uncertainty around usage-based poisoning of patents and other IP. Regarding "others may make money if it's MIT" -- so what? Is the point to help the world in useful ways, or not? Useful things tend to make money. If there's an alternate, commercially supported, non-gpl license available for payment, that solves that problem, but if you won't make that available, choosing gplv3 just means you choose *not* to help a large set of software developers. Jw On Jun 15, 2011 11:13 PM, "Max Lapshin" wrote: > Damn, what a problem in GPL for a testing program? > Guys just want to protect their investment, but want to make a free > advertisement by offering their excelent product to community. > If it would be MIT, they can see their work in commercial product > without any profit for them. > > There is absolutely no problem in using GPL software to test your > products: just don't include it into distribution. > > Tests are not part of PropEr, but if any of you is afraid of GPL > virus, so don't include tests into your main repository: keep it > separate. > For example, I don't include test files for erlyvideo tests into > repository, because I don't want accidential discussions with their > potential copyright holders. > > > Btw, about PropEr. It is so widely told, that it is a magic tool, that > can remove all bugs in software, that I'm very interested to try it. > I will try to find time for it and tell if it helped me with erlyvideo or not. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Thu Jun 16 09:01:17 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jun 2011 11:01:17 +0400 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: <2FED89C1-D44D-4C89-B6F6-1488FD286CBB@gmail.com> On Jun 16, 2011, at 10:43 AM, Ulf Wiger wrote: > As far as I can tell, PropEr is an unauthorised reverse-engineering effort of the later commercial versions (the early version did not have shrinking or statem support, for example); How do you tell it's a reverse-engineering effort as opposed to a clean-room re-implementation? Is QuviQ QuickCheck rocket science that requires decompiling bytecode? There have been several release clean-room re-implementations of the JVM, how is QuickCheck different? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From overminddl1@REDACTED Thu Jun 16 09:09:23 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Thu, 16 Jun 2011 01:09:23 -0600 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: On Jun 16, 2011 12:48 AM, "Jon Watte" wrote: > > As a commercial entity with (unrelated) IP to protect, I may be able to USE GPLv3 software, as long as I don't distribute to others (free or not), but there is no way I can CONTRIBUTE to GPL v3, because of the uncertainty around usage-based poisoning of patents and other IP. > > Regarding "others may make money if it's MIT" -- so what? Is the point to help the world in useful ways, or not? Useful things tend to make money. If there's an alternate, commercially supported, non-gpl license available for payment, that solves that problem, but if you won't make that available, choosing gplv3 just means you choose *not* to help a large set of software developers. > > Jw > Exactly. I have contributed code to many non-gpl projects, but I cannot to gpl for the same reason, and likewise I cannot use gpl things either because I always release my code under more permissive licenses because I want it to be actually useful and used and not wage some holy war. Also, as a side point, in my experiences, when you have someone who is *forced* to release code it is always of such low quality so as to be unusable anyway, where the people who actually want to release code always tend to be much higher and of actually useful quality. As I see it gpl code not only reduces their audience but they also get fewer quality patches, which makes no sense to me at all. I make code to be used, not to make a worthless point. Now my original other question, how does this compare to the one at the afore-mentioned link? > On Jun 15, 2011 11:13 PM, "Max Lapshin" wrote: > > Damn, what a problem in GPL for a testing program? > > Guys just want to protect their investment, but want to make a free > > advertisement by offering their excelent product to community. > > If it would be MIT, they can see their work in commercial product > > without any profit for them. > > > > There is absolutely no problem in using GPL software to test your > > products: just don't include it into distribution. > > > > Tests are not part of PropEr, but if any of you is afraid of GPL > > virus, so don't include tests into your main repository: keep it > > separate. > > For example, I don't include test files for erlyvideo tests into > > repository, because I don't want accidential discussions with their > > potential copyright holders. > > > > > > Btw, about PropEr. It is so widely told, that it is a magic tool, that > > can remove all bugs in software, that I'm very interested to try it. > > I will try to find time for it and tell if it helped me with erlyvideo or not. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From overminddl1@REDACTED Thu Jun 16 09:11:36 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Thu, 16 Jun 2011 01:11:36 -0600 Subject: [erlang-questions] A PropEr announcement Message-ID: Oh, and another point, the only code I have ever been paid for is code that I have written for work, everything else is freely released under boost style licenses, and the gpl still makes no sense to me. On Jun 16, 2011 1:09 AM, "OvermindDL1" wrote: -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Thu Jun 16 09:28:47 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 16 Jun 2011 09:28:47 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <2FED89C1-D44D-4C89-B6F6-1488FD286CBB@gmail.com> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> <2FED89C1-D44D-4C89-B6F6-1488FD286CBB@gmail.com> Message-ID: On 16 Jun 2011, at 09:01, Joel Reymont wrote: > > On Jun 16, 2011, at 10:43 AM, Ulf Wiger wrote: > >> As far as I can tell, PropEr is an unauthorised reverse-engineering effort of the later commercial versions (the early version did not have shrinking or statem support, for example); > > How do you tell it's a reverse-engineering effort as opposed to a clean-room re-implementation? First of all, my intent was not to pass judgement (IANAL), only to point out that there are other legal subtleties regarding PropEr. But since the point of expressing an opinion is mainly to stimulate discussion, I dug into the legal text in order to try to answer your question. The relevant EU legal text [1] says, under Article 4 - restricted acts: "(a) the permanent or temporary reproduction of a computer program by any means and in any form, in part or in whole. Insofar as loading, displaying, running, transmision or storage of the computer program necessitate such reproduction, such acts shall be subject to authorization by the rightholder" This seems to imply that if it happened as Kostis said - that they wrote PropEr based on available tutorials, articles and documentation, never having been in possession of a copy of the software, this will not constitute a violation of said directive. > Is QuviQ QuickCheck rocket science that requires decompiling bytecode? It doesn't have to be. It only has to fall under the laws regarding copyright protection. "3. A computer program shall be protected if it is original in the sense that it is the author's own intellectual creation. No other criteria shall be applied to determine its eligibility for protection." (Article 1) But the more difficult part is perhaps what parts are protected in the first place: "2. Protection in accordance with this Directive shall apply to the expression in any form of a computer program. Ideas and principles which underlie any element of a computer program, including those which underlie its interfaces, are not protected by copyright under this Directive." A possible interpretation of this is that releasing any implementation that almost perfectly emulates the API of a protected program *does* violate the directive, much as a reproduction of a painting would. What is hard to reconcile is whether it is still infringement if the copy was made without ever having been in possession of the original. :) Presumably, this would also mean that the PropEr authors never used QuickCheck Mini - also a protected program. BR, Ulf [1] "Council Directive 91/250/EEC of 14 May 1991 on the legal protection of computer programs" http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:31991L0250:EN:HTML Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitrii@REDACTED Thu Jun 16 09:29:23 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Thu, 16 Jun 2011 10:29:23 +0300 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: > Damn, what a problem in GPL for a testing program? > Guys just want to protect their investment, but want to make a free > advertisement by offering their excelent product to community. > If it would be MIT, they can see their work in commercial product > without any profit for them. > > There is absolutely no problem in using GPL software to test your > products: just don't include it into distribution. > This won't work for a lot of opensource projects. GPL is simply not compatible with anything. And since most opensource include tests, the issue of GPL creeping up the chain and contaminating code becomes relevant. Since you distribute tests which link to both PropEr *and* your own code: - tests themselves must be GPL, since they link to GPL code - since tests are now GPL, they cannot link to GPL-incompatible code. Therefore your own code must now become GPL. I wonder why EPL/MPL is not an option for PropEr? > Tests are not part of PropEr, but if any of you is afraid of GPL > virus, so don't include tests into your main repository: keep it > separate. > For example, I don't include test files for erlyvideo tests into > repository, because I don't want accidential discussions with their > potential copyright holders. > > > Btw, about PropEr. It is so widely told, that it is a magic tool, that > can remove all bugs in software, that I'm very interested to try it. > I will try to find time for it and tell if it helped me with erlyvideo or not. See Kostis Sagonas' presentation on PropEr on erlang-factory.com. It's much better with video, but video isn't there yet From joelr1@REDACTED Thu Jun 16 09:55:06 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 16 Jun 2011 11:55:06 +0400 Subject: [erlang-questions] Kill process if message mailbox reaches a certain size (was discarding signals) In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A9F8@ESESSCMS0353.eemea.ericsson.se> Message-ID: <006A2298-6FBA-4F87-8AFF-010CE4DDABA7@gmail.com> Jon, On Jun 16, 2011, at 9:09 AM, Jon Watte wrote: > I would advise against single-core nodes or cloud-based nodes that don't have local networking, What do you mean by "don't have local networking"? I thought the loopback network interface is a given. > I also recommend modeling the traffic across the backplane of the Erlang nodes. How much data do you send per user "event" to other users, and how many other users? Broadcast or point-to-point? Sum it all up, double it, and see if you can still swing that on your current network backplane. If not, buy a bigger network, or start working on ways to compress/reduce the data stream :-) I would also keep in mind that messages between Erlang processes and inter-node "kernel pings" go over the same socket. This means that pings start to lag as the message traffic gets heavy. Erlang nodes split when pings are significantly delayed and then in your in deep doodoo. This used to be a case a couple of years ago, I don't think anything changed since. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From vladdu55@REDACTED Thu Jun 16 10:00:04 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 16 Jun 2011 10:00:04 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: On Thu, Jun 16, 2011 at 08:43, Max Lapshin wrote: > Of course, tests aren't covered with the same license. > > GCC is licensed under GPL. You are writing C code, which has ifdefs > for GCC and for MSVC. How can you imagine, that someone will order you > to release your program under GPL? > I don't know. But I've seen stranger things in my life :-) > PropEr is just a tool. If you distribute this tool, you must share > sources. You may sell it, but with opened sources. But you don't have > any restrictions about files, that this tool works with. > Apparently, that the tests are linked with the PropEr runtime code is enough to contaminate them with GPL. regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu Jun 16 10:01:26 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 16 Jun 2011 10:01:26 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: On Thu, Jun 16, 2011 at 08:43, Ulf Wiger wrote: > > On 16 Jun 2011, at 08:30, Vlad Dumitrescu wrote: > > Hi, > > This kind of issues are worse than flame wars! :-) > > I have yet another point of view that increases the confusion: PropEr is > mostly compatible with Triq and QuickCheck (there's a free older version). > > > Well, there is a version available in jungerl, but to my knowledge, it was > not put there by the authors, and not actually intended to be free. Even so, > one might consider it a lot less sensitive to copy that version than the > later, proprietary, versions of QuickCheck. > Quviq released an official mini-version last year. http://www.quviq.com/news100621.html regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Thu Jun 16 10:23:04 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 16 Jun 2011 10:23:04 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: On 16 Jun 2011, at 10:01, Vlad Dumitrescu wrote: > On Thu, Jun 16, 2011 at 08:43, Ulf Wiger wrote: > > On 16 Jun 2011, at 08:30, Vlad Dumitrescu wrote: > >> Hi, >> >> This kind of issues are worse than flame wars! :-) >> >> I have yet another point of view that increases the confusion: PropEr is mostly compatible with Triq and QuickCheck (there's a free older version). > > Well, there is a version available in jungerl, but to my knowledge, it was not put there by the authors, and not actually intended to be free. Even so, one might consider it a lot less sensitive to copy that version than the later, proprietary, versions of QuickCheck. > > Quviq released an official mini-version last year. > > http://www.quviq.com/news100621.html Yes, but the QuickCheck Mini license agreement, while permitting free use and re-distribution, specifically forbids reverse-engineering. Thus, we should hope that the PropEr developers did not obtain a copy of QuickCheck Mini and ran it in order to learn how it worked, as this would put them in violation of the EU Directives regarding software copyright. For those who are inspired by PropEr, but are put off by the GPL, using and re-distributing QuickCheck Mini is of course unproblematic. ;-) BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Thu Jun 16 10:30:59 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 16 Jun 2011 10:30:59 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: <4DF9BF43.7080508@gmail.com> On 2011-06-16 08:43, Max Lapshin wrote: > Of course, tests aren't covered with the same license. > > GCC is licensed under GPL. You are writing C code, which has ifdefs > for GCC and for MSVC. How can you imagine, that someone will order you > to release your program under GPL? > > PropEr is just a tool. If you distribute this tool, you must share > sources. You may sell it, but with opened sources. But you don't have > any restrictions about files, that this tool works with. This is pretty simple: to a compiler, the source code you write are just input, and the resulting object files are just output, and these are not affected in any way by the licensing of the compiler itself, just like editing a photo in GIMP has no legal implications on the resulting image file. There is one exception: if the generated object code contains significant portions of code copied from templates that belong to the compiler - then the license of the templates applies to the generated object code. The GNU Bison parser is such a case: see http://www.gnu.org/software/bison/manual/html_node/Conditions.html#Conditions for their explanation of how they originally used GPL for the template code as well, but later dropped that so it could be used freely by anyone. (This also applies to code fragments in included header files - the guideline is that macros of up to 10 lines of code are not considered significant, but if they are larger, the author of the header file should consider adding an exception to the license - I have done that in EUnit - or use different licensing for the header files in order to avoid tainting users' code.) But in a test framework, you often write code (your tests) which calls out to the test framework library code. In this respect, test code is no different from, say, scientific code that calls a math library. The resulting code, after linking (statically or at run-time) is a work based on the code of the test framework. I guess that if the test code is written solely as callbacks, as done in the Common Test style of test suites, the situation is less clear. /Richard From carlsson.richard@REDACTED Thu Jun 16 11:44:13 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 16 Jun 2011 11:44:13 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: <4DF9D06D.3010903@gmail.com> On 2011-06-16 03:18, James Churchman wrote: > Is this a fact or a stated FSF opinion? seems counter intuitive that > something that writes to an api ( even if it's dynamically linked ) can > be considered an alteration of derived work, as what is to stop someone > producing an identical api with the same, or indeed a different set of > behaviour, does the existence of this "new" api now change things. > Better still what if it targets an intermediate api. In that case if all > calls in your code go to notgplmodule:do() which is a dual licensed > opensource project, which calls gplv3 code, what happens then?? Here's a good summary of the FSF's thoughts about combined works, from the GPL FAQ: "Where's the line between two separate programs, and one program with two parts? This is a legal question, which ultimately judges will decide. We believe that a proper criterion depends both on the mechanism of communication (exec, pipes, rpc, function calls within a shared address space, etc.) and the semantics of the communication (what kinds of information are interchanged). If the modules are included in the same executable file, they are definitely combined in one program. If modules are designed to run linked together in a shared address space, that almost surely means combining them into one program. By contrast, pipes, sockets and command-line arguments are communication mechanisms normally used between two separate programs. So when they are used for communication, the modules normally are separate programs. But if the semantics of the communication are intimate enough, exchanging complex internal data structures, that too could be a basis to consider the two parts as combined into a larger program." By this reasoning, it is for example not sufficient separation to use two Erlang processes and use message passing to avoid direct linkage - the connection is still too "intimate". Using an intermediate/wrapper API doesn't work either; the linked combination of the GPL code, the wrapper code, and your code, becomes a single larger work, and the GPL will apply to all of it (also covered in the FAQ). An API in itself (a list of function names, type signatures and other calling conventions) cannot to my knowledge be copyrighted, because it's simply not considered a "work" that can be protected by copyright laws - it's too trivial. The question is: if your code is written to use a particular API (like the one in your wrapper example), can you reasonably claim that your code is not designed to use a particular implementation of that API? I.e., do there exist any real alternatives? The (very) little I know of law is that courts tend to throw out reasoning that is pretty obviously only trying to create a smokescreen, like "I wasn't being paid, I just happened to pick up a bunch of dollars dropped by someone who turned out to be the guy who asked me to do him a favour yesterday." If the only existing, working, implementation of your neat wrapper API, is one that just forwards calls to some GPL'ed library, the "intermediate API" argument won't hold. But reversely, if you do write a program targeting an API with several available implementations, then if one of those happens to be a GPL'ed library, it doesn't affect your code, because it wasn't designed to use that library specifially, and it's up to your users if they choose that implementation. (I know I've seen a good summary of this stuff once, but I can't seem to find a reference right now.) So, if there was once just one implementation, using GPL, of a particular API, and someone does a clean-room implementation of that same API but under a different license like MIT, then as far as I can tell, that does change the situation, at least as long as the non-GPL implementation is somehow available to the users of your code. I.e., I don't think you can create an alternative and buggy implementation, lock it up in your vault, and then claim that your program isn't necessarily designed to use that GPL'ed library that actually works. It's of course also possible for the author of a library to use the GPL with an exception that says something like "as long as you as a user just call these API functions, the GPL licensing doesn't affect your own code even if you link with this library". But if there is no such exception, you won't get around the GPL by a simple wrapper. Using separate OS processes and pipes for communication _might_ be enough, but probably not if you're marshalling complex data structures in an obvious attempt to get around the GPL. /Richard From roberto.aloi@REDACTED Thu Jun 16 13:05:46 2011 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Thu, 16 Jun 2011 11:05:46 +0000 (GMT) Subject: [erlang-questions] Photos from the Erlang Factory In-Reply-To: <433360315.306451308222288741.JavaMail.root@zimbra> Message-ID: <860089743.306471308222346212.JavaMail.root@zimbra> Hello everyone, sorry if I misuse this mailinglist but I know that most (if not all of them) of the delegates and speakers at the latest Erlang Factory conference are listening here. The photos from the conference are finally online: http://www.flickr.com/photos/erlang-consulting/sets/72157626849482931/ I promise I won't blackmail Kostis, featured in a PropEr hug... Regards, Roberto Aloi -- University of Kent - Erlang Solutions Ltd. Blog: http://aloiroberto.wordpress.com Twitter: @robertoaloi From jameschurchman@REDACTED Thu Jun 16 14:10:40 2011 From: jameschurchman@REDACTED (James Churchman) Date: Thu, 16 Jun 2011 13:10:40 +0100 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: <99E9CAB8-356C-4D87-8318-450EE4785847@gmail.com> > Magical Things Happen. The tendency for linking to GPL works being > required to be GPL works is a complicated thing that gets contested in > different ways in different places. > hey amy, thanks for your reply > Making an intermediate API to present a non-GPL interface to > proprietary code is a pretty well established technique. See: some > video card drivers in Linux, with having a GPL compatible bit that > just creates an interface for the non-GPL compatible bits to later > use. > well then, an intermediate (mit+gpl) module, lets call it "prop", fixes all the issues then :-) add in a few cleanroom macros and that sounds like an hour long job! anybody willing? ...no need to panic guys... :-) James > Making an identical API to an existing GPL based API also complicates things. > > From here, the only correct answer is "Consult a lawyer in your > jurisdiction." Unfortunate, but that's how it goes. From mihai@REDACTED Thu Jun 16 14:43:39 2011 From: mihai@REDACTED (Mihai Balea) Date: Thu, 16 Jun 2011 08:43:39 -0400 Subject: [erlang-questions] Kill process if message mailbox reaches a certain size (was discarding signals) In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A9F8@ESESSCMS0353.eemea.ericsson.se> Message-ID: On Jun 16, 2011, at 1:09 AM, Jon Watte wrote: > First, I assume that the proposal for "kill and restart" is to kill and restart the particular process that has an inbox that is "too full." This does not mean that the client crashes -- it means that those messages are lost. This is no different to how Erlang systems are robust in the face of rare crashing bugs (as opposed to, say, C++, where generally the entire system goes down because of a rare stray pointer bug, for example). Basically, crashing and re-starting an Erlang worker process is just one way of clearing out the message queue, and also making sure that any possible state corruption goes away because the process re-starts "afresh." The Erlang/OTP supervision tree is designed to work in this mode. That is exactly what I was proposing. When you kill just one offending process, you will probably lose one call (or transaction, etc ) but you don't bog down a scheduler or even an entire VM. Especially considering that processes with growing message queues tend to exhibit runaway memory consumption as well, which will eventually bring down the VM (see the thread Max mentioned). Mihai From jameschurchman@REDACTED Thu Jun 16 14:48:15 2011 From: jameschurchman@REDACTED (James Churchman) Date: Thu, 16 Jun 2011 13:48:15 +0100 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <99E9CAB8-356C-4D87-8318-450EE4785847@gmail.com> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> <99E9CAB8-356C-4D87-8318-450EE4785847@gmail.com> Message-ID: <8FDA967B-09E5-490A-89A1-01F975B2DC04@gmail.com> Part 1) Ok better still, if the PropEr team really do feel that this is the kind of project that would benefit from begin gpl licences ( it's not... tho i can see the benefits for something like the linux kernel of being gpl ) then if i understand correctly to fix all theses problems all they need to do is : The interface module & the macro header to PropEr needs to be dual licensed, All the clever code that does all the property testing can still be put as gpl. Part 2) To ask the wider question, is the worry of some commercial company grabbing the source and making a heavily modified close source version that is better, then attempting to *sell it*, really an issue: There *already was* a closed source version, called "QuickCheck" and this got, in its near entirety, fully cloned, and in a relatively short space of time. I really cant see a few additions made by some unknown erlang company ever working : A) There are not that many companies that know erlang well enough: >> Even if for some unknown reason erlang solutions tried to do this, not that they would, there are still enough erlang coders out in the world to look at the version being sold, and to very very very very very very rapidly, if the additions are good, clone them again.. as i said before the entirety of PropEr is a clone to start with >> There is no other erlang company that has even 50 % of the erlang talent, even dare i say erricson ? so not other company could even come close.. B) The additions made to PropEr are unlikely to be so substantial that people would pay thousands for these additions... C) What any company would really pay for is the commercial support, the brains to produce very *good* tests, training, and if they really needed their own additions to the property based testing tool, these to be written by the people who know along with a lot more, none of which is the tool its self, but the brains who *understand* the tool In *all* these cases they would go to QuviQ to have them do it, not to some odd startup that has a mildly bastardised PropEr that they refuse to open source In this case ( and not the case of say linux etc..) i can only see the gpl getting in the way of the very purpose of the tool, and think that in the long run there will be a total NEED for the erlang community to produce a testing tool that is truly "free". "triq" should be possible to "trick" out, as it is apache 2, everybody can comfortably contribute, has the basic underpinning, and would be worth the effort in the medium, and definitely long term, to everybody. James On 16 Jun 2011, at 13:10, James Churchman wrote: >> Magical Things Happen. The tendency for linking to GPL works being >> required to be GPL works is a complicated thing that gets contested in >> different ways in different places. >> > > hey amy, thanks for your reply > >> Making an intermediate API to present a non-GPL interface to >> proprietary code is a pretty well established technique. See: some >> video card drivers in Linux, with having a GPL compatible bit that >> just creates an interface for the non-GPL compatible bits to later >> use. >> > > well then, an intermediate (mit+gpl) module, lets call it "prop", fixes all the issues then :-) > add in a few cleanroom macros and that sounds like an hour long job! anybody willing? > > ...no need to panic guys... :-) > > James > >> Making an identical API to an existing GPL based API also complicates things. >> >> From here, the only correct answer is "Consult a lawyer in your >> jurisdiction." Unfortunate, but that's how it goes. > From dmitrii@REDACTED Thu Jun 16 14:58:30 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Thu, 16 Jun 2011 15:58:30 +0300 Subject: [erlang-questions] Photos from the Erlang Factory In-Reply-To: <860089743.306471308222346212.JavaMail.root@zimbra> References: <860089743.306471308222346212.JavaMail.root@zimbra> Message-ID: <35CE1293-B491-4AC4-8158-01C783601316@dmitriid.com> To Roberto: Hurrah!!! To others: keep photos coming :) I know there've been a few people with cameras :) > Hello everyone, > > sorry if I misuse this mailinglist but I know that most (if not all of them) of the delegates and speakers at the latest Erlang Factory conference are listening here. > > The photos from the conference are finally online: > > http://www.flickr.com/photos/erlang-consulting/sets/72157626849482931/ > > I promise I won't blackmail Kostis, featured in a PropEr hug... > > Regards, > > Roberto Aloi > -- > University of Kent - Erlang Solutions Ltd. > Blog: http://aloiroberto.wordpress.com > Twitter: @robertoaloi > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Thu Jun 16 14:59:23 2011 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 16 Jun 2011 14:59:23 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <4DF9D06D.3010903@gmail.com> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> <4DF9D06D.3010903@gmail.com> Message-ID: On Thu, Jun 16, 2011 at 11:44 AM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > On 2011-06-16 03:18, James Churchman wrote: > >> Is this a fact or a stated FSF opinion? seems counter intuitive that >> something that writes to an api ( even if it's dynamically linked ) can >> be considered an alteration of derived work, as what is to stop someone >> producing an identical api with the same, or indeed a different set of >> behaviour, does the existence of this "new" api now change things. >> Better still what if it targets an intermediate api. In that case if all >> calls in your code go to notgplmodule:do() which is a dual licensed >> opensource project, which calls gplv3 code, what happens then?? >> > > Here's a good summary of the FSF's thoughts about combined works, from the > GPL FAQ: > > "Where's the line between two separate programs, and one program with two > parts? This is a legal question, which ultimately judges will decide. We > believe that a proper criterion depends both on the mechanism of > communication (exec, pipes, rpc, function calls within a shared address > space, etc.) and the semantics of the communication (what kinds of > information are interchanged). > If the modules are included in the same executable file, they are > definitely combined in one program. If modules are designed to run linked > together in a shared address space, that almost surely means combining them > into one program. > By contrast, pipes, sockets and command-line arguments are communication > mechanisms normally used between two separate programs. So when they are > used for communication, the modules normally are separate programs. But if > the semantics of the communication are intimate enough, exchanging complex > internal data structures, that too could be a basis to consider the two > parts as combined into a larger program." > What a load of complete and utter nonsense - what does "are intimate enough mean" etc. What about systems that are non-deterministic? If I send a message to a black box and it tosses a dice to see if it will reply then surely this is not an "intimate communication" Black box abstract says that two systems are identical if their observed I/O behavior is identical. But if I open the black box and discover either one or two communicating things inside then the law is *different* since in the one case I do not have to consider the intimacy of the relationship and the other I do. The basic idea in object-oriented programming is *independent* objects communicating by message passing. Would a lawyer in a court of law have to decide if a collection of communicating objects constituted one program or many - what happens when you put them all in the same address space like smalltalk - is this one program or two thousand small programs each with it's own license. > > By this reasoning, it is for example not sufficient separation to use two > Erlang processes and use message passing to avoid direct linkage - the > connection is still too "intimate". > No if you send a process it might on good day if it feels like it send a reply. Image two nodes one in Sweden the other on the moon with flaky communication - sometimes a reply to a message would be received sometimes not - anyway in the absence of a formal proof one would never know if a message will be returned unless you can solve the halting problem. /Joe > > Using an intermediate/wrapper API doesn't work either; the linked > combination of the GPL code, the wrapper code, and your code, becomes a > single larger work, and the GPL will apply to all of it (also covered in the > FAQ). > > An API in itself (a list of function names, type signatures and other > calling conventions) cannot to my knowledge be copyrighted, because it's > simply not considered a "work" that can be protected by copyright laws - > it's too trivial. The question is: if your code is written to use a > particular API (like the one in your wrapper example), can you reasonably > claim that your code is not designed to use a particular implementation of > that API? I.e., do there exist any real alternatives? > > The (very) little I know of law is that courts tend to throw out reasoning > that is pretty obviously only trying to create a smokescreen, like "I wasn't > being paid, I just happened to pick up a bunch of dollars dropped by someone > who turned out to be the guy who asked me to do him a favour yesterday." If > the only existing, working, implementation of your neat wrapper API, is one > that just forwards calls to some GPL'ed library, the "intermediate API" > argument won't hold. But reversely, if you do write a program targeting an > API with several available implementations, then if one of those happens to > be a GPL'ed library, it doesn't affect your code, because it wasn't designed > to use that library specifially, and it's up to your users if they choose > that implementation. (I know I've seen a good summary of this stuff once, > but I can't seem to find a reference right now.) > > So, if there was once just one implementation, using GPL, of a particular > API, and someone does a clean-room implementation of that same API but under > a different license like MIT, then as far as I can tell, that does change > the situation, at least as long as the non-GPL implementation is somehow > available to the users of your code. I.e., I don't think you can create an > alternative and buggy implementation, lock it up in your vault, and then > claim that your program isn't necessarily designed to use that GPL'ed > library that actually works. > > It's of course also possible for the author of a library to use the GPL > with an exception that says something like "as long as you as a user just > call these API functions, the GPL licensing doesn't affect your own code > even if you link with this library". But if there is no such exception, you > won't get around the GPL by a simple wrapper. Using separate OS processes > and pipes for communication _might_ be enough, but probably not if you're > marshalling complex data structures in an obvious attempt to get around the > GPL. > > /Richard > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Thu Jun 16 15:39:12 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 16 Jun 2011 15:39:12 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> <4DF9D06D.3010903@gmail.com> Message-ID: <4DFA0780.4010800@gmail.com> On 06/16/2011 02:59 PM, Joe Armstrong wrote: > What a load of complete and utter nonsense - what does "are intimate > enough mean" etc. > > What about systems that are non-deterministic? If I send a message to a > black box > and it tosses a dice to see if it will reply then surely this is not an > "intimate communication" > > Black box abstract says that two systems are identical if their observed > I/O behavior is identical. > But if I open the black box and discover either one or two communicating > things inside then > the law is *different* since in the one case I do not have to consider > the intimacy of the relationship and the > other I do. > > The basic idea in object-oriented programming is *independent* objects > communicating by > message passing. Would a lawyer in a court of law have to decide if a > collection of communicating objects > constituted one program or many - what happens when you put them all in > the same address space > like smalltalk - is this one program or two thousand small programs each > with it's own license. Basically, yes, if it came to a court case, it would be up to lawyers. The little I know of law is that the kind of abstract, down-to-first-principles reasoning that we programmers like to entertain doesn't usually impress the courts. Law is pretty different, and is very much about intentionality - both the intentions of the lawmakers (even when the paragraphs aren't totally clear, there may be other background material, previous cases etc., that are used as guidelines), and the apparent intentions of the accused: it's hard to get away on an abstract technical point if it's rather obvious that you were trying to do something against the intention of the law - see for example the case against the Pirate Bay where the defendants were found guilty of "assisting in making copyrighted content available", even though they claimed they were only running a kind of billboard service and it was down to the users what they decided to publish on it. When it comes to software and licenses, it all boils down to the question "did the defendant intentionally create program B in such a way that it builds on program A, without complying with the licensing terms of program A", and I don't think the defense "but it's all just objects" is likely to work. But the exact boundaries of how B may or may not build on A have never been extensively tried in court as far as I know, so there is definitely some uncertainty here. (And if they ever are tried, I pity the judge and jury who will have to listen through hours of geeky arguments about static and dynamic linkage, separate address spaces, marshalling, etc., and try to make any sense of it.) /Richard From jameschurchman@REDACTED Thu Jun 16 17:36:48 2011 From: jameschurchman@REDACTED (James Churchman) Date: Thu, 16 Jun 2011 16:36:48 +0100 Subject: [erlang-questions] erlang property testing Message-ID: The other thread seems totally off topic to me, missing the wider point of "testing in erlang" so I thought I would start a second that does not contain intimacies the gpl, however fascinating and leave that for the other thread. As I see it : ] In the near past there was eunit etc.. but it has limitations ] QuickCheck came along and shines a "new light" on the world of testing Unfortunately it's closed source and costs a lot of money, this is great for the likes of big businesses creating custom software : They pay in the big picture a small amount of money, get fantastic support, find bugs quickly; it's proven the quicker you find a bug the it costs dramatically less For opensource projects it was a problem; anyone with out a licence could not even test the code or run the suite, so cant test their pull requests first For startups with limited cash, it was a problem For people something that is not 100 % vanilla OTP/Erlang this is a problem as its suppled as complied beam code.. (eg for testing erjang compiler etc.. etc.. etc.. ) a big problems for a "testing tool" For people first investigating and learning erlang it was a problem For people with a want to extend its functionality etc.. it was a problem; it was closed source.. all the amazing minds in the erlang world were locked out of making improvements they saw fit ] PropEr (unannounced) arrives on the scene, is 0 cost and the source is available to tinker with Modifications to the source can be made It also comes with the benefit that it can test the specs** ( should be easy deriving the type generators from the spec AST tho) ] quickcheck mini comes out, a bit late and with less functionality than the other two and it's still closed source, but is available for 0 cost This is roughly the situation we are right now and there seems a few problems: a) QuickCheck is not really compatible with PropEr This seems to be a problem, and often choice is good. You can pick the tool that suites your needs.. eg mochiweb or yaws, totally different really. In this case however its just plain incompatibility and bad. It means that if i start on PropEr, and decide i want someone else to build me a more intensive test suite; this seems a good idea.. if someone else tests your code.. specially someone with more experience; All tests have to be modified and moved over In many cases there will be two camps, the PropEr and the QQ camp. Over time the two will divider further. b) The QuickCheck product and the ideas behind property based testing come from QuviQ and its employes. I think that everybody own them a large degree of gratitude, tho i also think that their business naivety does show threw here. It was just 100 % obvious to I think me and many other erlang dev's that a tool *this* good and *this* powerful was always going to be cloned, and the clone was never going to be 100 % compatible. Whether it was PropE or Triq or one written by someone else this was always going to happen & happen quickly. Also the majority of the money is charging the big companies, with ALL the money, for custom service. I think that everybody appreciates the presence of QuviQ & co in the erlang community, but there was a desire and plain need for a better unit testing tool that can be modified, used by anybody with out licence etc.. c) A testing tool is very different. I think the gpl has strengths, in that it stops a huge company grabbing your code and selling a better version, and in many ways it protects the original authors, but it also comes at great cost. A testing tool is by its very nature something to test your code, it needs to be "embedded" in your product if it to do anything but test the IO of the entire system. In this case the gpl is a poor fit.. PropEr will never be the product it intended to be whilst under the GPL... in many ways its easier to embed beam code for an entirely closed testing tool. So the obvious questions is where does this near future train crash end up? In my dream land, the original QQ goes MIT licence and we all just contribute to that, the obvious problem I see is that it's lightly not the business plan that QuviQ ever dreamed of. Tho lightly neither was the arrival of clone of QQ , which will lightly be far more damaging to their biz as these clones are not fully compatible. Then PropEr can be seen in the future as an instigator of great change, but not really used The other options are : PropEr Moves to a more sensible "for a testing tool " licence than the gpl. As the other thread reveals this is going to be a recurring nightmare for ever on, until of course a non gpl one arrives. Then possibly, if the entire erlang testing ecosystem, including maybe larges sections of OTP move to PropEr, then it may become untenable to have a closed alternative. maybe in this case, as unusual as it may seem QuviQ may want to offer commercial support for PropEr. If it was me in that position personally, I would be rubbing my hands together. The thought that an entire industry may take up my idea would be great, as the saying goes it's usually better to have a small % of something huge than a large % of something tiny, especially when it's your expert domain that goes mainstream. I think its just more complex biz model than selling. A legal & tecnical workaround. This is my least favroable option, but if there is an intermediary that can run either QQ or PropEr that is dual licences then its licence Ok and API differences ok. The problem with this is its extra work and ends up with the usual "least common denominator". If we liked the solution we would all be writing java. We all finish Triq. I am fairly confidant i could spin up the spec testing quite fast. I understand the spec AST's for transforming into something else I use. Maybe a few others could get the shrinking code more up-to-speed. State machines etc.. can come later. I think this is very feasible. A few more I invite people to propose. I also welcome replies that are less a minutiae of a minutiae but more *broad* in nature of what is the best end goal. Sincere apologies if this trampled on a whole heap of clever and wise peoples toes, but better in the clear now i feel than have undisclosed problems in the future J From co7eb@REDACTED Thu Jun 16 19:21:46 2011 From: co7eb@REDACTED (=?iso-8859-1?Q?Gilberio_Carmenates_Garc=EDa?=) Date: Thu, 16 Jun 2011 13:21:46 -0400 Subject: [erlang-questions] A big favor Message-ID: <000001cc2c49$e0f55130$a2dff390$@frcuba.co.cu> Hi all, I now this is a bit wrong but some time we have no so much choices, I have a little problem, I took on credit some money to improve my pc because I needed it, 130 dollars, from a friend but he leaves the town soon and I now he won?t took that back from me because he my friend but I don?t want to be like that with him, I thought I could pay that soon but the trust is that here is a bit hard to collect that count of money in less than four month, I get 17 dollars at month and that is very difficult. What I wish is if some of you can help me, I already have 30 dollars but miss 100, I need help from some of you, it will very nice and I will be grateful all my live and I hope to better my job and have better salary and someday help some of you. Best, Ivan, the Cuban Erlang boy. ======================================================================= Este mensaje ha sido enviado mediante el servicio de correo electr?nico que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para respaldar el cumplimiento de los objetivos de la organizaci?n y su pol?tica informativa. La persona que env?a este correo asume el compromiso de usar el servicio a tales fines y cumplir con las regulaciones establecidas. -------------- next part -------------- An HTML attachment was scrubbed... URL: From djk121@REDACTED Thu Jun 16 21:48:50 2011 From: djk121@REDACTED (Dan Kelley) Date: Thu, 16 Jun 2011 15:48:50 -0400 Subject: [erlang-questions] erl_scan, erl_parse, macros and records Message-ID: Hi, I'm working on a router of sorts. I'd like to have a config file which specifies how to classify messages, what channels are available, and routing rules based on classifications and channels. When my application starts, I'd like to transform the configuration into a module (let's call it 'router') so I can do someting like 'router:route(Message)'. Message will be a record that encapsulates a bunch of things about the message, and the returned value will be the channel to which the message should be delivered. The Message record is reasonably complicated - the top level thing contains several layers of different records. I also have a large number of macros that I use to extract different fields/subrecords of a top-level Message. When I started looking at how to do this, I quickly found erl_scan, erl_parse, and friends. I can easily create a module on the fly with these, but everything falls apart when I need to get at the Message record and macro definitions. Despite the access provided to compiler-ish things by erl_scan, erl_parse and friends, it looks like the only way to get at the preprocessor is to use epp on files. This is a problem because the code I want to dynamically create needs to use the record definitions and macros from the Message header file. Trying to figure a way around this led me to aleppo, which neatly solved the problem of including the header file, but aleppo doesn't process records. For a brief and exciting moment, it looked like erl_expand_records might save the day, but it expects forms, and I can't get anything with records through erl_parse:parse_form. Am I missing an easier way to dynamically create code that references macros and records? Thanks, Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Thu Jun 16 22:12:02 2011 From: mihai@REDACTED (Mihai Balea) Date: Thu, 16 Jun 2011 16:12:02 -0400 Subject: [erlang-questions] erl_scan, erl_parse, macros and records In-Reply-To: References: Message-ID: <5710622D-C410-4209-875E-EB70D8C58FD8@hates.ms> On Jun 16, 2011, at 3:48 PM, Dan Kelley wrote: > > Hi, > > I'm working on a router of sorts. I'd like to have a config file which specifies how to classify messages, what channels are available, and routing rules based on classifications and channels. > > When my application starts, I'd like to transform the configuration into a module (let's call it 'router') so I can do someting like 'router:route(Message)'. Message will be a record that encapsulates a bunch of things about the message, and the returned value will be the channel to which the message should be delivered. The Message record is reasonably complicated - the top level thing contains several layers of different records. I also have a large number of macros that I use to extract different fields/subrecords of a top-level Message. > > When I started looking at how to do this, I quickly found erl_scan, erl_parse, and friends. I can easily create a module on the fly with these, but everything falls apart when I need to get at the Message record and macro definitions. Despite the access provided to compiler-ish things by erl_scan, erl_parse and friends, it looks like the only way to get at the preprocessor is to use epp on files. This is a problem because the code I want to dynamically create needs to use the record definitions and macros from the Message header file. > > Trying to figure a way around this led me to aleppo, which neatly solved the problem of including the header file, but aleppo doesn't process records. For a brief and exciting moment, it looked like erl_expand_records might save the day, but it expects forms, and I can't get anything with records through erl_parse:parse_form. > > Am I missing an easier way to dynamically create code that references macros and records? > > Thanks, Why not generate a temporary file and apply compile:file/2 on it? Mihai From djk121@REDACTED Thu Jun 16 22:19:16 2011 From: djk121@REDACTED (Dan Kelley) Date: Thu, 16 Jun 2011 16:19:16 -0400 Subject: [erlang-questions] erl_scan, erl_parse, macros and records In-Reply-To: <5710622D-C410-4209-875E-EB70D8C58FD8@hates.ms> References: <5710622D-C410-4209-875E-EB70D8C58FD8@hates.ms> Message-ID: On Thu, Jun 16, 2011 at 4:12 PM, Mihai Balea wrote: > > On Jun 16, 2011, at 3:48 PM, Dan Kelley wrote: > > > > > Hi, > > > > I'm working on a router of sorts. I'd like to have a config file which > specifies how to classify messages, what channels are available, and routing > rules based on classifications and channels. > > > > When my application starts, I'd like to transform the configuration into > a module (let's call it 'router') so I can do someting like > 'router:route(Message)'. Message will be a record that encapsulates a bunch > of things about the message, and the returned value will be the channel to > which the message should be delivered. The Message record is reasonably > complicated - the top level thing contains several layers of different > records. I also have a large number of macros that I use to extract > different fields/subrecords of a top-level Message. > > > > When I started looking at how to do this, I quickly found erl_scan, > erl_parse, and friends. I can easily create a module on the fly with these, > but everything falls apart when I need to get at the Message record and > macro definitions. Despite the access provided to compiler-ish things by > erl_scan, erl_parse and friends, it looks like the only way to get at the > preprocessor is to use epp on files. This is a problem because the code I > want to dynamically create needs to use the record definitions and macros > from the Message header file. > > > > Trying to figure a way around this led me to aleppo, which neatly solved > the problem of including the header file, but aleppo doesn't process > records. For a brief and exciting moment, it looked like erl_expand_records > might save the day, but it expects forms, and I can't get anything with > records through erl_parse:parse_form. > > > > Am I missing an easier way to dynamically create code that references > macros and records? > > > > Thanks, > > Why not generate a temporary file and apply compile:file/2 on it? Two reasons: First, I can provide better error messages, in the sense that I'll have smaller compile units that I'll trace back to specific parts of the config. Second, I'm not sure that I can make that non-ugly. I had hoped to write my dynamic module to disk using erl_pp so it'd be human-readable. But yeah, that's my fallback position. Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From anders.nygren@REDACTED Thu Jun 16 22:42:55 2011 From: anders.nygren@REDACTED (Anders Nygren) Date: Thu, 16 Jun 2011 15:42:55 -0500 Subject: [erlang-questions] erl_scan, erl_parse, macros and records In-Reply-To: References: <5710622D-C410-4209-875E-EB70D8C58FD8@hates.ms> Message-ID: On Thu, Jun 16, 2011 at 3:19 PM, Dan Kelley wrote: > > > On Thu, Jun 16, 2011 at 4:12 PM, Mihai Balea wrote: >> >> On Jun 16, 2011, at 3:48 PM, Dan Kelley wrote: >> >> > >> > Hi, >> > >> > I'm working on a router of sorts. ?I'd like to have a config file which >> > specifies how to classify messages, what channels are available, and routing >> > rules based on classifications and channels. >> > >> > When my application starts, I'd like to transform the configuration into >> > a module (let's call it 'router') so I can do someting like >> > 'router:route(Message)'. ?Message will be a record that encapsulates a bunch >> > of things about the message, and the returned value will be the channel to >> > which the message should be delivered. ?The Message record is reasonably >> > complicated - the top level thing contains several layers of different >> > records. ?I also have a large number of macros that I use to extract >> > different fields/subrecords of a top-level Message. >> > >> > When I started looking at how to do this, I quickly found erl_scan, >> > erl_parse, and friends. ?I can easily create a module on the fly with these, >> > but everything falls apart when I need to get at the Message record and >> > macro definitions. ?Despite the access provided to compiler-ish things by >> > erl_scan, erl_parse and friends, it looks like the only way to get at the >> > preprocessor is to use epp on files. ?This is a problem because the code I >> > want to dynamically create needs to use the record definitions and macros >> > from the Message header file. >> > >> > Trying to figure a way around this led me to aleppo, which neatly solved >> > the problem of including the header file, but aleppo doesn't process >> > records. ?For a brief and exciting moment, it looked like erl_expand_records >> > might save the day, but it expects forms, and I can't get anything with >> > records through erl_parse:parse_form. >> > >> > Am I missing an easier way to dynamically create code that references >> > macros and records? >> > >> > Thanks, >> >> Why not generate a temporary file and apply compile:file/2 on it? > > Two reasons: > First, I can provide better error messages, in the sense that I'll have > smaller compile units that I'll trace back to specific parts of the config. > Second, ?I'm not sure that I can make that non-ugly. ?I had hoped to write > my dynamic module to disk using erl_pp so it'd be human-readable. > But yeah, that's my fallback position. > Dan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > Hi Take a look at erl_syntax. /Anders From zsolt.erl@REDACTED Fri Jun 17 00:26:36 2011 From: zsolt.erl@REDACTED (Zsolt Keszthelyi) Date: Thu, 16 Jun 2011 18:26:36 -0400 Subject: [erlang-questions] erl_scan, erl_parse, macros and records In-Reply-To: References: <5710622D-C410-4209-875E-EB70D8C58FD8@hates.ms> Message-ID: Have you tried smerl? http://code.google.com/p/smerl/ I used it for something very similar (dynamically generating email sorting functions from a text file containing rules) Zsolt On Thu, Jun 16, 2011 at 4:19 PM, Dan Kelley wrote: > > > On Thu, Jun 16, 2011 at 4:12 PM, Mihai Balea wrote: > >> >> On Jun 16, 2011, at 3:48 PM, Dan Kelley wrote: >> >> > >> > Hi, >> > >> > I'm working on a router of sorts. I'd like to have a config file which >> specifies how to classify messages, what channels are available, and routing >> rules based on classifications and channels. >> > >> > When my application starts, I'd like to transform the configuration into >> a module (let's call it 'router') so I can do someting like >> 'router:route(Message)'. Message will be a record that encapsulates a bunch >> of things about the message, and the returned value will be the channel to >> which the message should be delivered. The Message record is reasonably >> complicated - the top level thing contains several layers of different >> records. I also have a large number of macros that I use to extract >> different fields/subrecords of a top-level Message. >> > >> > When I started looking at how to do this, I quickly found erl_scan, >> erl_parse, and friends. I can easily create a module on the fly with these, >> but everything falls apart when I need to get at the Message record and >> macro definitions. Despite the access provided to compiler-ish things by >> erl_scan, erl_parse and friends, it looks like the only way to get at the >> preprocessor is to use epp on files. This is a problem because the code I >> want to dynamically create needs to use the record definitions and macros >> from the Message header file. >> > >> > Trying to figure a way around this led me to aleppo, which neatly solved >> the problem of including the header file, but aleppo doesn't process >> records. For a brief and exciting moment, it looked like erl_expand_records >> might save the day, but it expects forms, and I can't get anything with >> records through erl_parse:parse_form. >> > >> > Am I missing an easier way to dynamically create code that references >> macros and records? >> > >> > Thanks, >> >> Why not generate a temporary file and apply compile:file/2 on it? > > > Two reasons: > > First, I can provide better error messages, in the sense that I'll have > smaller compile units that I'll trace back to specific parts of the config. > > Second, I'm not sure that I can make that non-ugly. I had hoped to write > my dynamic module to disk using erl_pp so it'd be human-readable. > > But yeah, that's my fallback position. > > Dan > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ebegumisa@REDACTED Fri Jun 17 04:56:59 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Fri, 17 Jun 2011 12:56:59 +1000 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: Just to clarify/reinforce, what gets a little confusing in these discussions are the rights and obligations between licensor and licensee, which is a contractual arrangement and subject to the doctrine of privity. [Continuing with the mentioned hypothetical] * If the PropEr developers have copies of QuickCheck Mini, then they bind themselves to a license agreement with Quviq, including the promise not to reverse engineer. * If PropEr is GPL'ed, and Vlad uses it, he binds himself to an agreement with the PropEr developers, and the terms thereunder. Vlad is not privy to the first agreement, and thus can never be held liable for its breach, if there is such a breach. And he cannot be expected to stop using a version he already has. (Though he might have the rug pulled out from under him, should PropEr development cease and future versions stop being made.) However, assuming all parties are in Europe, and although the concept of freedom of contract generally applies, they also bind themselves doctrine that they can never make an agreement that contravenes codified EU law - including said reverse-engineering law in said copyright statute. Then the aggrieved party need not be privy to an agreement to have their rights enforced since these rights are not established by an agreement, but by a statute. In this case, Vlad and all PropEr users in Europe could well be ordered by a court to cease using the program regardless of any agreement they have with it's developers (since such an agreement would be considered invalidated.) Hypothetically speaking. That said: Erlang being a small-friendly community -- I assumed (and like to continue to a assume) that PropEr and QuickCheck developers have not only been aware of each others projects, but I also like to think that they've been in touch with each other and sorted out any such concerns before version 1 of PropEr was announced. Like Erlang, I prefer to think of the happy case! - Edmond - On Thu, 16 Jun 2011 18:23:04 +1000, Ulf Wiger wrote: > > On 16 Jun 2011, at 10:01, Vlad Dumitrescu wrote: > >> On Thu, Jun 16, 2011 at 08:43, Ulf Wiger >> wrote: >> >> On 16 Jun 2011, at 08:30, Vlad Dumitrescu wrote: >> >>> Hi, >>> >>> This kind of issues are worse than flame wars! :-) >>> >>> I have yet another point of view that increases the confusion: PropEr >>> is mostly compatible with Triq and QuickCheck (there's a free older >>> version). >> >> Well, there is a version available in jungerl, but to my knowledge, it >> was not put there by the authors, and not actually intended to be free. >> Even so, one might consider it a lot less sensitive to copy that >> version than the later, proprietary, versions of QuickCheck. >> >> Quviq released an official mini-version last year. >> >> http://www.quviq.com/news100621.html > > Yes, but the QuickCheck Mini license agreement, while permitting free > use and re-distribution, specifically forbids reverse-engineering. Thus, > we should hope that the PropEr developers did not obtain a copy of > QuickCheck Mini and ran it in order to learn how it worked, as this > would put them in violation of the EU Directives regarding software > copyright. > > For those who are inspired by PropEr, but are put off by the GPL, using > and re-distributing QuickCheck Mini is of course unproblematic. ;-) > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From peralta.alejandro@REDACTED Fri Jun 17 05:07:21 2011 From: peralta.alejandro@REDACTED (Ale) Date: Fri, 17 Jun 2011 00:07:21 -0300 Subject: [erlang-questions] MPI and Erlang. (fun experiment) Message-ID: Hello all, Just sharing with whom might ever be interested, a fun (for me, might not be for you) experiment. I tried to port a MPI implementation of Simpson's rule[0], I had made for a course in HPC, to Erlang. I wasn't expecting Erlang to beat C... but, well see the results: MPI: https://github.com/alep/Experiments/blob/master/simpson/c-mpi/simpsons_rule_mpi.c $ time mpirun -np 2 mpisimpson 0 10 100000000 ale@REDACTED's password: proc[0]: iterating from: 0 to: 50000000 proc[0]: result: 1.83907152907611348702 proc[1]: iterating from: 50000000 to: 100000000 real 0m7.016s user 0m3.288s sys 0m0.072s Erlang: https://github.com/alep/Experiments/blob/master/simpson/erlang/simpsonrule.erl $ time ./run.sh Hostname: dell-desktop Proc[0]: iterating from 0 to 50000000 Proc[1]: iterating from 50000000 to 100000000 Result: 1.8390715290761146 real 0m25.047s user 0m22.713s sys 0m0.116s FAIL! :-D But it was fun. Now is there a way to make it a bit faster? What am I doing wrong? How can I improve it? Ideas....? Thanks [0] http://en.wikipedia.org/wiki/Simpson%27s_rule -- Ale. From ebegumisa@REDACTED Fri Jun 17 05:08:42 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Fri, 17 Jun 2011 13:08:42 +1000 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> <4DF9D06D.3010903@gmail.com> Message-ID: Aaaaa ... the all to common Legislature vs Judiciary problem. * Legislators make laws. * These laws are made sufficiently vague (usually due to "legislative compromise" or plain incompetence) * Courts have to interpret vagueness, judges make precedents to do so. * Legislators then complain that judges are making their own laws... "Not their mandate! The Judiciary should only enforce laws that *we* make," they scream. Maybe engineers should make laws that pertain to engineering! Heck, maybe engineers should be on the bench for enforcing those laws! - Edmond - On Thu, 16 Jun 2011 22:59:23 +1000, Joe Armstrong wrote: > On Thu, Jun 16, 2011 at 11:44 AM, Richard Carlsson < > carlsson.richard@REDACTED> wrote: > >> On 2011-06-16 03:18, James Churchman wrote: >> >>> Is this a fact or a stated FSF opinion? seems counter intuitive that >>> something that writes to an api ( even if it's dynamically linked ) can >>> be considered an alteration of derived work, as what is to stop someone >>> producing an identical api with the same, or indeed a different set of >>> behaviour, does the existence of this "new" api now change things. >>> Better still what if it targets an intermediate api. In that case if >>> all >>> calls in your code go to notgplmodule:do() which is a dual licensed >>> opensource project, which calls gplv3 code, what happens then?? >>> >> >> Here's a good summary of the FSF's thoughts about combined works, from >> the >> GPL FAQ: >> >> "Where's the line between two separate programs, and one program with >> two >> parts? This is a legal question, which ultimately judges will decide. We >> believe that a proper criterion depends both on the mechanism of >> communication (exec, pipes, rpc, function calls within a shared address >> space, etc.) and the semantics of the communication (what kinds of >> information are interchanged). >> If the modules are included in the same executable file, they are >> definitely combined in one program. If modules are designed to run >> linked >> together in a shared address space, that almost surely means combining >> them >> into one program. >> By contrast, pipes, sockets and command-line arguments are >> communication >> mechanisms normally used between two separate programs. So when they are >> used for communication, the modules normally are separate programs. But >> if >> the semantics of the communication are intimate enough, exchanging >> complex >> internal data structures, that too could be a basis to consider the two >> parts as combined into a larger program." >> > > What a load of complete and utter nonsense - what does "are intimate > enough > mean" etc. > > What about systems that are non-deterministic? If I send a message to a > black box > and it tosses a dice to see if it will reply then surely this is not an > "intimate communication" > > Black box abstract says that two systems are identical if their observed > I/O > behavior is identical. > But if I open the black box and discover either one or two communicating > things inside then > the law is *different* since in the one case I do not have to consider > the > intimacy of the relationship and the > other I do. > > The basic idea in object-oriented programming is *independent* objects > communicating by > message passing. Would a lawyer in a court of law have to decide if a > collection of communicating objects > constituted one program or many - what happens when you put them all in > the > same address space > like smalltalk - is this one program or two thousand small programs each > with it's own license. > > >> >> By this reasoning, it is for example not sufficient separation to use >> two >> Erlang processes and use message passing to avoid direct linkage - the >> connection is still too "intimate". >> > > No if you send a process it might on good day if it feels like it send a > reply. > > Image two nodes one in Sweden the other on the moon with flaky > communication > - sometimes a reply > to a message would be received sometimes not - anyway in the absence of a > formal proof one would never > know if a message will be returned unless you can solve the halting > problem. > > > /Joe > > > >> >> Using an intermediate/wrapper API doesn't work either; the linked >> combination of the GPL code, the wrapper code, and your code, becomes a >> single larger work, and the GPL will apply to all of it (also covered >> in the >> FAQ). >> >> An API in itself (a list of function names, type signatures and other >> calling conventions) cannot to my knowledge be copyrighted, because it's >> simply not considered a "work" that can be protected by copyright laws - >> it's too trivial. The question is: if your code is written to use a >> particular API (like the one in your wrapper example), can you >> reasonably >> claim that your code is not designed to use a particular implementation >> of >> that API? I.e., do there exist any real alternatives? >> >> The (very) little I know of law is that courts tend to throw out >> reasoning >> that is pretty obviously only trying to create a smokescreen, like "I >> wasn't >> being paid, I just happened to pick up a bunch of dollars dropped by >> someone >> who turned out to be the guy who asked me to do him a favour >> yesterday." If >> the only existing, working, implementation of your neat wrapper API, is >> one >> that just forwards calls to some GPL'ed library, the "intermediate API" >> argument won't hold. But reversely, if you do write a program targeting >> an >> API with several available implementations, then if one of those >> happens to >> be a GPL'ed library, it doesn't affect your code, because it wasn't >> designed >> to use that library specifially, and it's up to your users if they >> choose >> that implementation. (I know I've seen a good summary of this stuff >> once, >> but I can't seem to find a reference right now.) >> >> So, if there was once just one implementation, using GPL, of a >> particular >> API, and someone does a clean-room implementation of that same API but >> under >> a different license like MIT, then as far as I can tell, that does >> change >> the situation, at least as long as the non-GPL implementation is somehow >> available to the users of your code. I.e., I don't think you can create >> an >> alternative and buggy implementation, lock it up in your vault, and then >> claim that your program isn't necessarily designed to use that GPL'ed >> library that actually works. >> >> It's of course also possible for the author of a library to use the GPL >> with an exception that says something like "as long as you as a user >> just >> call these API functions, the GPL licensing doesn't affect your own code >> even if you link with this library". But if there is no such exception, >> you >> won't get around the GPL by a simple wrapper. Using separate OS >> processes >> and pipes for communication _might_ be enough, but probably not if >> you're >> marshalling complex data structures in an obvious attempt to get around >> the >> GPL. >> >> /Richard >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Fri Jun 17 05:30:29 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Fri, 17 Jun 2011 13:30:29 +1000 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: On Fri, 17 Jun 2011 12:56:59 +1000, Edmond Begumisa wrote: > Just to clarify/reinforce, what gets a little confusing in these > discussions are the rights and obligations between licensor and > licensee, which is a contractual arrangement and subject to the doctrine > of privity. > CORRECTION: This was inaccurate. A licensee agreement is not strictly contractual agreement, but a license may be passed as consideration exchanged as part of a contractual agreement. Nevertheless, the privy doctrine usually applies to agreements in which licenses are past in most jurisdictions. > [Continuing with the mentioned hypothetical] > > * If the PropEr developers have copies of QuickCheck Mini, then they > bind themselves to a license agreement with Quviq, including the promise > not to reverse engineer. > * If PropEr is GPL'ed, and Vlad uses it, he binds himself to an > agreement with the PropEr developers, and the terms thereunder. > > Vlad is not privy to the first agreement, and thus can never be held > liable for its breach, if there is such a breach. And he cannot be > expected to stop using a version he already has. (Though he might have > the rug pulled out from under him, should PropEr development cease and > future versions stop being made.) > > However, assuming all parties are in Europe, and although the concept of > freedom of contract generally applies, they also bind themselves > doctrine that they can never make an agreement that contravenes codified > EU law - including said reverse-engineering law in said copyright > statute. Then the aggrieved party need not be privy to an agreement to > have their rights enforced since these rights are not established by an > agreement, but by a statute. In this case, Vlad and all PropEr users in > Europe could well be ordered by a court to cease using the program > regardless of any agreement they have with it's developers (since such > an agreement would be considered invalidated.) Hypothetically speaking. > > That said: Erlang being a small-friendly community -- I assumed (and > like to continue to a assume) that PropEr and QuickCheck developers have > not only been aware of each others projects, but I also like to think > that they've been in touch with each other and sorted out any such > concerns before version 1 of PropEr was announced. Like Erlang, I prefer > to think of the happy case! > > - Edmond - > > > On Thu, 16 Jun 2011 18:23:04 +1000, Ulf Wiger > wrote: > >> >> On 16 Jun 2011, at 10:01, Vlad Dumitrescu wrote: >> >>> On Thu, Jun 16, 2011 at 08:43, Ulf Wiger >>> wrote: >>> >>> On 16 Jun 2011, at 08:30, Vlad Dumitrescu wrote: >>> >>>> Hi, >>>> >>>> This kind of issues are worse than flame wars! :-) >>>> >>>> I have yet another point of view that increases the confusion: PropEr >>>> is mostly compatible with Triq and QuickCheck (there's a free older >>>> version). >>> >>> Well, there is a version available in jungerl, but to my knowledge, it >>> was not put there by the authors, and not actually intended to be >>> free. Even so, one might consider it a lot less sensitive to copy that >>> version than the later, proprietary, versions of QuickCheck. >>> >>> Quviq released an official mini-version last year. >>> >>> http://www.quviq.com/news100621.html >> >> Yes, but the QuickCheck Mini license agreement, while permitting free >> use and re-distribution, specifically forbids reverse-engineering. >> Thus, we should hope that the PropEr developers did not obtain a copy >> of QuickCheck Mini and ran it in order to learn how it worked, as this >> would put them in violation of the EU Directives regarding software >> copyright. >> >> For those who are inspired by PropEr, but are put off by the GPL, using >> and re-distributing QuickCheck Mini is of course unproblematic. ;-) >> >> BR, >> Ulf W >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ulf.wiger@REDACTED Fri Jun 17 08:42:50 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 17 Jun 2011 08:42:50 +0200 Subject: [erlang-questions] erl_scan, erl_parse, macros and records In-Reply-To: References: Message-ID: <3FC8B04D-7B55-4FFB-9EE3-306968CDDC56@erlang-solutions.com> This sounds in part similar to what the Diameter application (part of OTP R14B03) does. The Diameter application uses a parse_transform called diameter_exprecs, which is essentially the same as the exprecs module in http://github.com/esl/parse_trans For one thing, this allows users to work with record definitions generated at compile-time, without having to deal with include files. It also allows the diameter_service agent to check ubiquitous attributes like 'Destination-Host', 'Destination-Realm' in any record, as long as that record is known by the encode-decode ("dictionary") module: get_avp_value(Dict, Name, Rec) when is_tuple(Rec) -> try Dict:'#get-'(Name, Rec) catch error:_ -> undefined end. (from diameter/src/app/diameter_service.erl) Another thing that might help is parse_trans_codegen.erl, which provides a (IMHO) more readable way to generate code. The idea is that you simply wrap the code you want to generate in a fun(), possibly using some simple expansion techniques to import values, etc. If records are known at generation time, they can simply be included and used as-is, with normal record syntax. https://github.com/esl/parse_trans/blob/master/doc/parse_trans_codegen.md See also https://github.com/esl/parse_trans/blob/master/examples/ex_codegen.erl for some (admittedly pretty corny) examples. BR, Ulf W On 16 Jun 2011, at 21:48, Dan Kelley wrote: > > Hi, > > I'm working on a router of sorts. I'd like to have a config file which specifies how to classify messages, what channels are available, and routing rules based on classifications and channels. > > When my application starts, I'd like to transform the configuration into a module (let's call it 'router') so I can do someting like 'router:route(Message)'. Message will be a record that encapsulates a bunch of things about the message, and the returned value will be the channel to which the message should be delivered. The Message record is reasonably complicated - the top level thing contains several layers of different records. I also have a large number of macros that I use to extract different fields/subrecords of a top-level Message. > > When I started looking at how to do this, I quickly found erl_scan, erl_parse, and friends. I can easily create a module on the fly with these, but everything falls apart when I need to get at the Message record and macro definitions. Despite the access provided to compiler-ish things by erl_scan, erl_parse and friends, it looks like the only way to get at the preprocessor is to use epp on files. This is a problem because the code I want to dynamically create needs to use the record definitions and macros from the Message header file. > > Trying to figure a way around this led me to aleppo, which neatly solved the problem of including the header file, but aleppo doesn't process records. For a brief and exciting moment, it looked like erl_expand_records might save the day, but it expects forms, and I can't get anything with records through erl_parse:parse_form. > > Am I missing an easier way to dynamically create code that references macros and records? > > Thanks, > > Dan > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Fri Jun 17 08:46:24 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 17 Jun 2011 08:46:24 +0200 Subject: [erlang-questions] MPI and Erlang. (fun experiment) In-Reply-To: References: Message-ID: <1308293184.5047.61.camel@seasc1137> Greetings, Thank you for supplying the example. Would it be possible for you to split parts of simpsonrule.erl into one mpi.erl where you have the C examples MPI library? It would make the comparison easier. Moreover, the C code has hard coded sin(). In Erlang you send a fun. Try to hard code sin() in Erlang too, it might speed things up a little. Finally, you have apply/2 in compute/5 which I think could be written as Fun(X), it might speed things up a little. bengt On Fri, 2011-06-17 at 05:07 +0200, Ale wrote: > Hello all, > > Just sharing with whom might ever be interested, a fun (for me, might > not be for you) experiment. I tried to port a MPI implementation of > Simpson's rule[0], I had made for a course in HPC, to Erlang. I wasn't > expecting Erlang to beat C... but, well see the results: > > MPI: https://github.com/alep/Experiments/blob/master/simpson/c-mpi/simpsons_rule_mpi.c > $ time mpirun -np 2 mpisimpson 0 10 100000000 > ale@REDACTED's password: > proc[0]: iterating from: 0 to: 50000000 > proc[0]: result: 1.83907152907611348702 > proc[1]: iterating from: 50000000 to: 100000000 > > real 0m7.016s > user 0m3.288s > sys 0m0.072s > > Erlang: https://github.com/alep/Experiments/blob/master/simpson/erlang/simpsonrule.erl > $ time ./run.sh > Hostname: dell-desktop > Proc[0]: iterating from 0 to 50000000 > Proc[1]: iterating from 50000000 to 100000000 > Result: 1.8390715290761146 > > real 0m25.047s > user 0m22.713s > sys 0m0.116s > > FAIL! :-D But it was fun. > > Now is there a way to make it a bit faster? What am I doing wrong? How > can I improve it? Ideas....? Thanks > > [0] http://en.wikipedia.org/wiki/Simpson%27s_rule From ivan@REDACTED Fri Jun 17 09:34:39 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Fri, 17 Jun 2011 08:34:39 +0100 Subject: [erlang-questions] Everything Everywhere receives marketing award Message-ID: <4DFB038F.4080609@llaisdy.com> Dear All I hope this isn't too off-topic. I just saw Everything Everywhere received an award from the Institute of Promotional Marketing: Gold for Employee Recognition. http://www.theipm.org.uk/The_Institute_of_Promotional_Marketing_Awards/list_of_IPM_Awards_2011_winners.aspx Their campaign was called "The Bee's Knees" and the blurb says: A range of bee assets, a bee lexicon and bee-haviours were developed ... OTP bee-haviours perhaps? Best wishes (and congrats to Everything Everywhere people if any are reading) Ivan -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From ok@REDACTED Fri Jun 17 10:33:18 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 17 Jun 2011 20:33:18 +1200 Subject: [erlang-questions] MPI and Erlang. (fun experiment) In-Reply-To: <1308293184.5047.61.camel@seasc1137> References: <1308293184.5047.61.camel@seasc1137> Message-ID: <375EEF4D-4315-4FEB-B53F-280DE9965CC8@cs.otago.ac.nz> > On Fri, 2011-06-17 at 05:07 +0200, Ale wrote: >> Hello all, >> >> Just sharing with whom might ever be interested, a fun (for me, might >> not be for you) experiment. I tried to port a MPI implementation of >> Simpson's rule[0], I had made for a course in HPC, to Erlang. I wasn't >> expecting Erlang to beat C... but, well see the results: (1) You are not comparing apples with apples: in the C code the 'sin' function is provided to the kernel code via a macro definition, which means that on an X86 a compiler might (with certain command line parameters) inline that and just generate the x87 'sin' instruction; in the Erlang code math:sin/1 is passed as a parameter and invoked in what I believe to be the slowest available way to invoke a function. (2) You tell us nothing about how the C code was compiled. Between different C compilers and different compilation options, a factor of 3 on the same machine is possible. (3) You tell us nothing about how the Erlang code was compiled. (4) You have told the C compiler all about the relevant types; you have told the Erlang compiler nothing about them. This is the kind of code where HiPE can make quite a difference, *if* you give it a few clues. Take compute(Idx, Acc, A, H, Fun) -> X = A + Idx * H, case Idx rem 2 of 0 -> NewAcc = Acc + apply(Fun, [X]) * 2; _ -> NewAcc = Acc + apply(Fun, [X]) * 4 end, NewAcc. as an example. First off, what _is_ the point of NewAcc? Why not write this as compute(Idx, Acc, A, H, Fun) -> Acc + Fun(A + Idx*H) * ((Idx band 1)*2 + 2). and then give HiPE a hint by writing compute(Idx, Acc, A, H, Fun) when is_integer(Idx), is_float(Acc), is_float(A), is_float(H), is_function(Fun, 1) -> Acc + Fun(A + Idx*H) * ((Idx band 1)*2 + 2). Of course the most help would come from a -spec, in which you can tell HiPe that Fun takes float arguments and delivers a float result. cycle(Start, End, Step, Fun, Args, Acc) -> NextStep = Start + Step, case NextStep > End of true -> Acc; false -> NewAcc = apply(Fun, [Start, Acc|Args]), cycle(NextStep, End, Step, Fun, Args, NewAcc) end. cycle(Start, End, Args, Acc) -> cycle(Start, End, 1, fun compute/5, Args, Acc). Here I note that Fun has arity 5 and is called with [Start,Acc|Args], so Args must have 3 elements. We could do cycle(Start, End, [A,H,Fun], Acc) -> cycle(Start, End, 1, Acc, fun (S, R) -> compute(S, R, A, H, Fun) end). cycle(Start, End, Step, Acc, Fun) -> Next_Step = Start + Step, if Next_Step > End -> Acc ; true -> cycle(Next_Step, End, Step, Fun(Start, Acc), Fun) end. and it's natural to revise this to cycle(Start, End, [A,H,Fun], Acc) when is_integer(Start), is_integer(End), is_float(A), is_float(H), is_float(Acc), is_function(Fun, 1) -> cycle(Start, End, 1, Acc, fun (I, R) -> R + Fun(A + I*H)*((I band 1)*2 + 2) end). cycle(Start, End, Step, Acc, Update) when is_integer(Start), is_integer(End), is_integer(Step), is_float(Acc), is_function(Update, 2) -> Next_Step = Start + Step, if Next_Step > End -> Acc ; true -> cycle(Next_Step, End, Step, Update(Start, Acc), Fun) end. As before, a more up to date way to help HiPe is to use -spec declarations, which can tell the compiler what the arguments of Update are and what the result is. The big issue here is that with adequate type information, HiPe can avoid boxing and unboxing a lot of floating point numbers, so it's quite likely that you can get a factor of 2 to 4 speedup fairly easily. From bob@REDACTED Fri Jun 17 11:05:00 2011 From: bob@REDACTED (Bob Cowdery) Date: Fri, 17 Jun 2011 10:05:00 +0100 Subject: [erlang-questions] wx-erlang question Message-ID: <4DFB18BC.8070906@bobcowdery.plus.com> Hi Are there any examples of using AUI where the size settings of the panes works. minSize, maxSize and bestSize seem to have limited effect, especially on the X axis where it is completely ignored. I've also tried savePerspective and loadPerspective which look good when I look at the string but attempting to load it back in gives a completely empty window. I appreciate this is probably a wx rather than erlang problem although I can't find anything that says perspectives don't work. Should I be doing anything special with the string or doing something special when I restore a perspective? I simply save it in an ETS table which is saved as a file. I have set the {update, true} option. Regards Bob From dangud@REDACTED Fri Jun 17 11:19:40 2011 From: dangud@REDACTED (Dan Gudmundsson) Date: Fri, 17 Jun 2011 11:19:40 +0200 Subject: [erlang-questions] wx-erlang question In-Reply-To: <4DFB18BC.8070906@bobcowdery.plus.com> References: <4DFB18BC.8070906@bobcowdery.plus.com> Message-ID: I don't know anything about it, I only wrote the wx<->wxWidgets mapping, you will have to search/ask in the wxWidgets forum/wiki directly. /Dan On Fri, Jun 17, 2011 at 11:05 AM, Bob Cowdery wrote: > Hi > > Are there any examples of using AUI where the size settings of the panes > works. minSize, maxSize and bestSize seem to have limited effect, > especially on the X axis where it is completely ignored. I've also tried > savePerspective and loadPerspective which look good when I look at the > string but attempting to load it back in gives a completely empty window. > > I appreciate this is probably a wx rather than erlang problem although I > can't find anything that says perspectives don't work. Should I be doing > anything special with the string or doing something special when I > restore a perspective? I simply save it in an ETS table which is saved > as a file. I have set the {update, true} option. > > Regards > Bob > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Fri Jun 17 11:57:38 2011 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 17 Jun 2011 10:57:38 +0100 Subject: [erlang-questions] Everything Everywhere receives marketing award In-Reply-To: <4DFB038F.4080609@llaisdy.com> References: <4DFB038F.4080609@llaisdy.com> Message-ID: Thank you Ivan. We certainly use a lot of Erlang and OTP behaviours in Everything Everywhere, and it is great fun working here! So if anyone is looking for a job hacking Erlang in a telecoms environment, please drop me a mail offline. I currently have one permanent and two contracting positions open. cheers Chandru On 17 June 2011 08:34, Ivan Uemlianin wrote: > Dear All > > I hope this isn't too off-topic. > > I just saw Everything Everywhere received an award from the Institute of > Promotional Marketing: Gold for Employee Recognition. > > http://www.theipm.org.uk/The_**Institute_of_Promotional_** > Marketing_Awards/list_of_IPM_**Awards_2011_winners.aspx > > Their campaign was called "The Bee's Knees" and the blurb says: > > A range of bee assets, a bee lexicon and bee-haviours > were developed ... > > OTP bee-haviours perhaps? > > Best wishes (and congrats to Everything Everywhere people if any are > reading) > > Ivan > > > -- > ==============================**============================== > Ivan A. Uemlianin > Speech Technology Research and Development > > ivan@REDACTED > www.llaisdy.com > llaisdy.wordpress.com > www.linkedin.com/in/**ivanuemlianin > > "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" > (Schiller, Beethoven) > ==============================**============================== > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Fri Jun 17 12:43:01 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 17 Jun 2011 12:43:01 +0200 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> <2FED89C1-D44D-4C89-B6F6-1488FD286CBB@gmail.com> Message-ID: <723AE2BF-5F9E-4B27-9265-11B0BC5E41DB@erlang-solutions.com> It has been brought to my attention that some have interpreted my posts in this thread as gunning for Kostis. I am not. I sincerely hope that no one has been on the wrong side of the law here (and even more that there will be no attempt at legal action). Knowing Kostis, I am pretty certain that he had very carefully considered what he was doing. I think it is extremely important to have a discussion about not just what is strictly legal, but also how we as a community act to show respect for other people's right to innovate and make money on their innovation. OTOH, Kostis definitely could have written a quickcheck-like tool with a different interface, staying absolutely clear of even the slightest risk of being accused of infringement. Safer for Kostis, but actually worse for the community. I believe it is better, ultimately for all parties involved, that the APIs are almost fully compatible. I hope that Quviq will find a way to profit from this too, not least since they have done a tremendous amount of good for the Erlang community. Uncritically applauding acts of essentially cloning other people's IPR may have unwanted consequences. Let's consider for a while that one of the main driving forces behind the current patent system is the Pharmaceutical industry. Why? Much because it is extremely expensive to develop and approve a new drug, but very cheap to copy it. It's clear that OSS is pushing the point at which one can charge for a software component higher and higher. Today, it seems, if you don't have several man-years of effort invested in your software, you might as well not even bother, since as soon as you launch, chances are that someone in the OSS community might decide to build a free version just as a fun exercise. I can't help thinking that this sort of activity is mostly self-gratifying, and doesn't take into account the fact that the Open Source community has yet to figure out how to be self-sustaining. Much of the money that funds Open Source (or at least pays the rent for those who contribute to it) comes from largely proprietary companies. What OSS has so far demonstrated is that giving things away for free can be extremely beneficial for innovation and quality. On the topic of ensuring steady revenue and managing to pay people's salaries at the end of each month, it has some way to go still. BR, Ulf W On 16 Jun 2011, at 09:28, Ulf Wiger wrote: > > On 16 Jun 2011, at 09:01, Joel Reymont wrote: > >> >> On Jun 16, 2011, at 10:43 AM, Ulf Wiger wrote: >> >>> As far as I can tell, PropEr is an unauthorised reverse-engineering effort of the later commercial versions (the early version did not have shrinking or statem support, for example); >> >> How do you tell it's a reverse-engineering effort as opposed to a clean-room re-implementation? > > First of all, my intent was not to pass judgement (IANAL), only to point out that there are other legal subtleties regarding PropEr. But since the point of expressing an opinion is mainly to stimulate discussion, I dug into the legal text in order to try to answer your question. > > The relevant EU legal text [1] says, under Article 4 - restricted acts: > > "(a) the permanent or temporary reproduction of a computer program by any means and in any form, in part or in whole. Insofar as loading, displaying, running, transmision or storage of the computer program necessitate such reproduction, such acts shall be subject to authorization by the rightholder" > > This seems to imply that if it happened as Kostis said - that they wrote PropEr based on available tutorials, articles and documentation, never having been in possession of a copy of the software, this will not constitute a violation of said directive. > >> Is QuviQ QuickCheck rocket science that requires decompiling bytecode? > > It doesn't have to be. It only has to fall under the laws regarding copyright protection. > > "3. A computer program shall be protected if it is original in the sense that it is the author's own intellectual creation. No other criteria shall be applied to determine its eligibility for protection." > (Article 1) > > But the more difficult part is perhaps what parts are protected in the first place: > > "2. Protection in accordance with this Directive shall apply to the expression in any form of a computer program. Ideas and principles which underlie any element of a computer program, including those which underlie its interfaces, are not protected by copyright under this Directive." > > A possible interpretation of this is that releasing any implementation that almost perfectly emulates the API of a protected program *does* violate the directive, much as a reproduction of a painting would. What is hard to reconcile is whether it is still infringement if the copy was made without ever having been in possession of the original. :) > > Presumably, this would also mean that the PropEr authors never used QuickCheck Mini - also a protected program. > > BR, > Ulf > > [1] "Council Directive 91/250/EEC of 14 May 1991 on the legal protection of computer programs" > http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:31991L0250:EN:HTML > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jameschurchman@REDACTED Fri Jun 17 12:43:51 2011 From: jameschurchman@REDACTED (James Churchman) Date: Fri, 17 Jun 2011 11:43:51 +0100 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: making something with a similar-ish api to something else is not reverse engineering. its a rewrite from scratch. reverse engineering would be disassembling the beam files to discover how they work, simply reimplementing them it is not. if that was the case there would be one cd player, one car, one bike, one washing machine, one fridge, one operating system etc.. in the world and never ever a second version, and having a copy of the original will make no difference, if you clean room something, as long as you don't break any patents in the process. On 17 Jun 2011, at 04:30, Edmond Begumisa wrote: > On Fri, 17 Jun 2011 12:56:59 +1000, Edmond Begumisa wrote: > >> Just to clarify/reinforce, what gets a little confusing in these discussions are the rights and obligations between licensor and licensee, which is a contractual arrangement and subject to the doctrine of privity. >> > > CORRECTION: This was inaccurate. A licensee agreement is not strictly contractual agreement, but a license may be passed as consideration exchanged as part of a contractual agreement. Nevertheless, the privy doctrine usually applies to agreements in which licenses are past in most jurisdictions. > >> [Continuing with the mentioned hypothetical] >> >> * If the PropEr developers have copies of QuickCheck Mini, then they bind themselves to a license agreement with Quviq, including the promise not to reverse engineer. >> * If PropEr is GPL'ed, and Vlad uses it, he binds himself to an agreement with the PropEr developers, and the terms thereunder. >> >> Vlad is not privy to the first agreement, and thus can never be held liable for its breach, if there is such a breach. And he cannot be expected to stop using a version he already has. (Though he might have the rug pulled out from under him, should PropEr development cease and future versions stop being made.) >> >> However, assuming all parties are in Europe, and although the concept of freedom of contract generally applies, they also bind themselves doctrine that they can never make an agreement that contravenes codified EU law - including said reverse-engineering law in said copyright statute. Then the aggrieved party need not be privy to an agreement to have their rights enforced since these rights are not established by an agreement, but by a statute. In this case, Vlad and all PropEr users in Europe could well be ordered by a court to cease using the program regardless of any agreement they have with it's developers (since such an agreement would be considered invalidated.) Hypothetically speaking. >> >> That said: Erlang being a small-friendly community -- I assumed (and like to continue to a assume) that PropEr and QuickCheck developers have not only been aware of each others projects, but I also like to think that they've been in touch with each other and sorted out any such concerns before version 1 of PropEr was announced. Like Erlang, I prefer to think of the happy case! >> >> - Edmond - >> >> >> On Thu, 16 Jun 2011 18:23:04 +1000, Ulf Wiger wrote: >> >>> >>> On 16 Jun 2011, at 10:01, Vlad Dumitrescu wrote: >>> >>>> On Thu, Jun 16, 2011 at 08:43, Ulf Wiger wrote: >>>> >>>> On 16 Jun 2011, at 08:30, Vlad Dumitrescu wrote: >>>> >>>>> Hi, >>>>> >>>>> This kind of issues are worse than flame wars! :-) >>>>> >>>>> I have yet another point of view that increases the confusion: PropEr is mostly compatible with Triq and QuickCheck (there's a free older version). >>>> >>>> Well, there is a version available in jungerl, but to my knowledge, it was not put there by the authors, and not actually intended to be free. Even so, one might consider it a lot less sensitive to copy that version than the later, proprietary, versions of QuickCheck. >>>> >>>> Quviq released an official mini-version last year. >>>> >>>> http://www.quviq.com/news100621.html >>> >>> Yes, but the QuickCheck Mini license agreement, while permitting free use and re-distribution, specifically forbids reverse-engineering. Thus, we should hope that the PropEr developers did not obtain a copy of QuickCheck Mini and ran it in order to learn how it worked, as this would put them in violation of the EU Directives regarding software copyright. >>> >>> For those who are inspired by PropEr, but are put off by the GPL, using and re-distributing QuickCheck Mini is of course unproblematic. ;-) >>> >>> BR, >>> Ulf W >>> >>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>> http://erlang-solutions.com >>> >>> >>> >> >> > > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ivan@REDACTED Fri Jun 17 13:10:12 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Fri, 17 Jun 2011 12:10:12 +0100 Subject: [erlang-questions] how to use sockets in erlang Message-ID: <4DFB3614.9080406@llaisdy.com> Dear All I have a Python application I'm porting to Erlang (I'm expecting the Erlang version to be a bit faster, but much more robust and easier to scale up). The central function is to jsonify some data, send it over a socket to a remote (third party) server, and receive a reply. Thanks to mochijson2, the jsonifying is sorted, but something in the socketing is not going well. In the excerpts below I abstract the socket connect/send/receive code. The python works as expected, the erlang gen_tcp:recv returns {error, closed}. I suspect I am missing something basic about sockets in Erlang. Please can someone point out what it is? n.b. jsondata (python) and JsonData (erlang) are identical --- I have even copied the list of integers in JsonData and pasted them into the python and sent them. Python version --- works: jsondata = json.dumps(data) + "\n" s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((appliance_host, appliance_port)) s.send(jsondata) returndata = s.recv(4096) Erlang version --- doesn't work: JsonData = list_to_binary(mochijson2:encode(Data) ++ <<10>>), {ok, Socket} = gen_tcp:connect(?APPLIANCE_HOST, ?APPLIANCE_PORT, [inet, binary, {active, false}, {packet, 0}]), ok = gen_tcp:send(Socket, JsonData), Result = gen_tcp:recv(Socket, 4096, 5000), % Result = {error, closed} With thanks and best wishes Ivan -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From watson.timothy@REDACTED Fri Jun 17 13:11:01 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 17 Jun 2011 12:11:01 +0100 Subject: [erlang-questions] erl_scan, erl_parse, macros and records In-Reply-To: <3FC8B04D-7B55-4FFB-9EE3-306968CDDC56@erlang-solutions.com> References: <3FC8B04D-7B55-4FFB-9EE3-306968CDDC56@erlang-solutions.com> Message-ID: I can also whole-heartedly recommend parse_trans, and in particular the exprecs module. You can see another usage of this in https://github.com/hyperthunk/nodewatch/blob/master/dxcommon/src/dx_subscription.erl and of the API for accessing record info in https://github.com/hyperthunk/nodewatch/blob/master/dxcommon/src/dxcommon.erl - look at the function record_to_proplist/1, which is completely generic in the type of the record and works with recursive record structures. Very useful stuff! On 17 June 2011 07:42, Ulf Wiger wrote: > > This sounds in part similar to what the Diameter application (part of OTP > R14B03) does. > The Diameter application uses a parse_transform called diameter_exprecs, > which is essentially?the same as the exprecs module in > http://github.com/esl/parse_trans > For one thing, this allows users to work with record definitions generated > at compile-time, without having to deal with include files. It also allows > the diameter_service agent to check ubiquitous attributes like > 'Destination-Host', 'Destination-Realm' in any record, as long as that > record is known by the encode-decode ("dictionary") module: > get_avp_value(Dict, Name, Rec) > ??when is_tuple(Rec) -> > ?? ?try > ?? ? ? ?Dict:'#get-'(Name, Rec) > ?? ?catch > ?? ? ? ?error:_ -> > ?? ? ? ? ? ?undefined > ?? ?end. > (from diameter/src/app/diameter_service.erl) > Another thing that might help is parse_trans_codegen.erl, which provides a > (IMHO) more readable way to generate code. > The idea is that you simply wrap the code you want to generate in a fun(), > possibly using some simple expansion techniques to import values, etc. If > records are known at generation time, they can simply be included and used > as-is, with normal record syntax. > https://github.com/esl/parse_trans/blob/master/doc/parse_trans_codegen.md > See > also?https://github.com/esl/parse_trans/blob/master/examples/ex_codegen.erl?for > some (admittedly pretty corny) examples. > BR, > Ulf W > On 16 Jun 2011, at 21:48, Dan Kelley wrote: > > Hi, > I'm working on a router of sorts. ?I'd like to have a config file which > specifies how to classify messages, what channels are available, and routing > rules based on classifications and channels. > When my application starts, I'd like to transform the configuration into a > module (let's call it 'router') so I can do someting like > 'router:route(Message)'. ?Message will be a record that encapsulates a bunch > of things about the message, and the returned value will be the channel to > which the message should be delivered. ?The Message record is reasonably > complicated - the top level thing contains several layers of different > records. ?I also have a large number of macros that I use to extract > different fields/subrecords of a top-level Message. > When I started looking at how to do this, I quickly found erl_scan, > erl_parse, and friends. ?I can easily create a module on the fly with these, > but everything falls apart when I need to get at the Message record and > macro definitions. ?Despite the access provided to compiler-ish things by > erl_scan, erl_parse and friends, it looks like the only way to get at the > preprocessor is to use epp on files. ?This is a problem because the code I > want to dynamically create needs to use the record definitions and macros > from the Message header file. > Trying to figure a way around this led me to aleppo, which neatly solved the > problem of including the header file, but aleppo doesn't process records. > ?For a brief and exciting moment, it looked like erl_expand_records might > save the day, but it expects forms, and I can't get anything with records > through erl_parse:parse_form. > Am I missing an easier way to dynamically create code that references macros > and records? > Thanks, > Dan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > Ulf Wiger,?CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From jesper.louis.andersen@REDACTED Fri Jun 17 13:32:02 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 17 Jun 2011 13:32:02 +0200 Subject: [erlang-questions] how to use sockets in erlang In-Reply-To: <4DFB3614.9080406@llaisdy.com> References: <4DFB3614.9080406@llaisdy.com> Message-ID: On Fri, Jun 17, 2011 at 13:10, Ivan Uemlianin wrote: I am just shooting in the dark here, but I have a guess at what is wrong: > Python version --- works: > > ? ?jsondata = json.dumps(data) + "\n" > ? ?s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > ? ?s.connect((appliance_host, appliance_port)) > ? ?s.send(jsondata) > ? ?returndata = s.recv(4096) The Python recv semantics follow that of the unix kernels recv(2) call. I.e., the 4096 is a buffer and you will get at most 4096 bytes, or less than that. > Erlang version --- doesn't work: > > ? ?JsonData = list_to_binary(mochijson2:encode(Data) ++ <<10>>), You usually don't have to call list_to_binary/1 here. The value [mochijson2:encode(Data), <<10>>] is an iolist() and sockets take iolist() types. That way, you won't be creating new binaries but just refer to already existing binaries in lists. It is much faster than building the binary first and then shoving it out the socket. It is a common idiom to make Erlang go fast when rendering data structures to output. Look up iolist()s. They are a key element to make your code go fast as it avoids a large number of binary constructions. > ? ?{ok, Socket} = gen_tcp:connect(?APPLIANCE_HOST, ?APPLIANCE_PORT, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[inet, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? binary, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {active, false}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {packet, 0}]), > ? ?ok = gen_tcp:send(Socket, JsonData), > ? ?Result = gen_tcp:recv(Socket, 4096, 5000), The Erlang semantics here are different. You are blocking until you have exactly 4096 bytes of data. If less is received, you will get {error, closed} back, or a timeout after 5000 ms. You should be calling gen_tcp:recv(Socket, 0, 5000) and then use byte_size(X) on the result yourself. It also means you will have to drain the socket yourself in a separate function. The solution is to figure out what the stream-semantics and syntax is and handle that with a couple of Erlang functions. Another important thing to look out for is that in Erlang the Socket is a port() bound to the connecting process (by default -- the controlling process can change with the controlling_process call). If the bound process dies, the socket closes. As soon as you begin passing sockets between processes this will haunt you unless you are aware of it. The cool thing about it as that you get automatic "resource reclamation" because of this. It also happens with files (they are closed), ETS tables, and so on. -- J. From jameschurchman@REDACTED Fri Jun 17 13:36:24 2011 From: jameschurchman@REDACTED (James Churchman) Date: Fri, 17 Jun 2011 12:36:24 +0100 Subject: [erlang-questions] A PropEr announcement In-Reply-To: <723AE2BF-5F9E-4B27-9265-11B0BC5E41DB@erlang-solutions.com> References: <4DF4BDDA.30809@cs.ntua.gr> <4DF698C1.2060908@catalyst.net.nz> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> <2FED89C1-D44D-4C89-B6F6-1488FD286CBB@gmail.com> <723AE2BF-5F9E-4B27-9265-11B0BC5E41DB@erlang-solutions.com> Message-ID: Dear Ulf I agree almost entirely with the sentiment of your email. I think that it is quite easy to be in a position where you are funded by external bodies, in the case of proper being funded by a large university and also EU grants, and then afford to give the software away for free. I also think it makes it a tad rich to release it under something as restrictive as the gpl given that it's the profits of the commercial companies, by virtue of paying taxes, that have in effect funded the software, tho it is of course in the end entirely the decision of the original authors. I however take a different take on the mater of open source and startups, especially in the case of developer facing tools. Taking the case of QuickCheck as the perfect text book example, it is crazy to turn up with a fantastic product that costs thousands of pounds & is totally closed to any contributions, present it to hundreds of the cleverest programers in the world and then expect them not to copy it when it is that good and also not something beyond the abilities of those people to build themselves. In the world, which is nothing but a realised version of game theory, this is crazy. Also how long was this meant to continue? is it realistic to think in 10 entire years nobody would produce a copy, or 5 , or 3 ? There was an obvious huge demand and the barrier to building one far lower than that huge demand. What it shows is that you want the benefit of these other people contributing to your product, essentially out-sourcing half the costs and innovation, and then sell some ancillary thing. If you create the tool you should know the most about it, and even if you don't people will approach you first. The world is a moving place, especially the world of IT. ( I would also argue that in many regards this is the basis Erlang Solutions operates on, Erlang is open source, and it sells its expertise in erlang whilst contributing somewhat back to erl/opt) Also in the case of QC i would assume that 95 % of people who wanted to use it did not, entirely due to cost, and that 100 % of people who wished to contribute to it could not, and by the pure definition of the tool all 195 % of those people were ALL developers. Therefore if you look at the biz model that by contrast Process One etc. etc. etc. follow they provide the software totally open, and in their case i think they were further clever picking up on an already popular open source project rather than an untested home grow affair, and this allows students that might be building the next Facebook or Google to use the software and pay for what they need in the future if / when they need to pay for things as well as when they have the cash to do so. I doubt ejabberd would have 1% of the users and even that process one would have a profitable biz if it was a closed up shop, tho that is a difficult one to back up. I don't think this applies to all products, but it seems to apply to many developer tools in the software industry. Other products probably less so, as its harder to coordinate say the design of a fantastic GUI between hundreds of dispersed developers, and for many consumers they are not aware of the open alternatives, don't mind paying a few $ and are not comfortable compiling code etc.. but i think this is an ok situation as consumers don't want & would not pay for training, support contracts, custom extensions to be written etc.. for their copy of photoshop elements. And you want to start with that model reasonably early, then hope to make it a successful business, not have the ticking time bomb of when something 0 cost, open source and in many cases far far superior come out & you have to change everything, as well as you becoming the awkward non-industry standard option. In the case of property based testing, i would imaging that there is a far far far larger biz around supporting PropEr, if it becomes the industry standard teasing tool that every single erlang dev uses, than there is in selling the software outright, as writing good test can be tricky, so i hope that Quviq works this out and do prosper greatly from this opportunity :-) James On 17 Jun 2011, at 11:43, Ulf Wiger wrote: > > It has been brought to my attention that some have interpreted my posts in this thread as gunning for Kostis. I am not. I sincerely hope that no one has been on the wrong side of the law here (and even more that there will be no attempt at legal action). Knowing Kostis, I am pretty certain that he had very carefully considered what he was doing. > > I think it is extremely important to have a discussion about not just what is strictly legal, but also how we as a community act to show respect for other people's right to innovate and make money on their innovation. > > OTOH, Kostis definitely could have written a quickcheck-like tool with a different interface, staying absolutely clear of even the slightest risk of being accused of infringement. Safer for Kostis, but actually worse for the community. I believe it is better, ultimately for all parties involved, that the APIs are almost fully compatible. I hope that Quviq will find a way to profit from this too, not least since they have done a tremendous amount of good for the Erlang community. > > Uncritically applauding acts of essentially cloning other people's IPR may have unwanted consequences. > > Let's consider for a while that one of the main driving forces behind the current patent system is the Pharmaceutical industry. Why? Much because it is extremely expensive to develop and approve a new drug, but very cheap to copy it. > > It's clear that OSS is pushing the point at which one can charge for a software component higher and higher. Today, it seems, if you don't have several man-years of effort invested in your software, you might as well not even bother, since as soon as you launch, chances are that someone in the OSS community might decide to build a free version just as a fun exercise. > > I can't help thinking that this sort of activity is mostly self-gratifying, and doesn't take into account the fact that the Open Source community has yet to figure out how to be self-sustaining. Much of the money that funds Open Source (or at least pays the rent for those who contribute to it) comes from largely proprietary companies. > > What OSS has so far demonstrated is that giving things away for free can be extremely beneficial for innovation and quality. On the topic of ensuring steady revenue and managing to pay people's salaries at the end of each month, it has some way to go still. > > BR, > Ulf W > > On 16 Jun 2011, at 09:28, Ulf Wiger wrote: > >> >> On 16 Jun 2011, at 09:01, Joel Reymont wrote: >> >>> >>> On Jun 16, 2011, at 10:43 AM, Ulf Wiger wrote: >>> >>>> As far as I can tell, PropEr is an unauthorised reverse-engineering effort of the later commercial versions (the early version did not have shrinking or statem support, for example); >>> >>> How do you tell it's a reverse-engineering effort as opposed to a clean-room re-implementation? >> >> First of all, my intent was not to pass judgement (IANAL), only to point out that there are other legal subtleties regarding PropEr. But since the point of expressing an opinion is mainly to stimulate discussion, I dug into the legal text in order to try to answer your question. >> >> The relevant EU legal text [1] says, under Article 4 - restricted acts: >> >> "(a) the permanent or temporary reproduction of a computer program by any means and in any form, in part or in whole. Insofar as loading, displaying, running, transmision or storage of the computer program necessitate such reproduction, such acts shall be subject to authorization by the rightholder" >> >> This seems to imply that if it happened as Kostis said - that they wrote PropEr based on available tutorials, articles and documentation, never having been in possession of a copy of the software, this will not constitute a violation of said directive. >> >>> Is QuviQ QuickCheck rocket science that requires decompiling bytecode? >> >> It doesn't have to be. It only has to fall under the laws regarding copyright protection. >> >> "3. A computer program shall be protected if it is original in the sense that it is the author's own intellectual creation. No other criteria shall be applied to determine its eligibility for protection." >> (Article 1) >> >> But the more difficult part is perhaps what parts are protected in the first place: >> >> "2. Protection in accordance with this Directive shall apply to the expression in any form of a computer program. Ideas and principles which underlie any element of a computer program, including those which underlie its interfaces, are not protected by copyright under this Directive." >> >> A possible interpretation of this is that releasing any implementation that almost perfectly emulates the API of a protected program *does* violate the directive, much as a reproduction of a painting would. What is hard to reconcile is whether it is still infringement if the copy was made without ever having been in possession of the original. :) >> >> Presumably, this would also mean that the PropEr authors never used QuickCheck Mini - also a protected program. >> >> BR, >> Ulf >> >> [1] "Council Directive 91/250/EEC of 14 May 1991 on the legal protection of computer programs" >> http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:31991L0250:EN:HTML >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Fri Jun 17 13:39:39 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Fri, 17 Jun 2011 12:39:39 +0100 Subject: [erlang-questions] SOLVED Re: how to use sockets in erlang In-Reply-To: References: <4DFB3614.9080406@llaisdy.com> Message-ID: <4DFB3CFB.5070905@llaisdy.com> Dear Jesper Thank you very much for your help! >> Result = gen_tcp:recv(Socket, 4096, 5000), > > The Erlang semantics here are different. You are blocking until you > have exactly 4096 bytes of data. If less is received, you will get > {error, closed} back, or a timeout after 5000 ms. You should be > calling gen_tcp:recv(Socket, 0, 5000) ... Yes! This works: {ok, Result} = gen_tcp:recv(Socket, 0, 5000), Thank you for your other comments too. I'll shape up the rest of the code. Best wishes Ivan On 17/06/2011 12:32, Jesper Louis Andersen wrote: > On Fri, Jun 17, 2011 at 13:10, Ivan Uemlianin wrote: > > I am just shooting in the dark here, but I have a guess at what is wrong: > >> Python version --- works: >> >> jsondata = json.dumps(data) + "\n" >> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) >> s.connect((appliance_host, appliance_port)) >> s.send(jsondata) >> returndata = s.recv(4096) > > The Python recv semantics follow that of the unix kernels recv(2) > call. I.e., the 4096 is a buffer and you will get at most 4096 bytes, > or less than that. > >> Erlang version --- doesn't work: >> >> JsonData = list_to_binary(mochijson2:encode(Data) ++<<10>>), > > You usually don't have to call list_to_binary/1 here. The value > [mochijson2:encode(Data),<<10>>] is an iolist() and sockets take > iolist() types. That way, you won't be creating new binaries but just > refer to already existing binaries in lists. It is much faster than > building the binary first and then shoving it out the socket. It is a > common idiom to make Erlang go fast when rendering data structures to > output. Look up iolist()s. They are a key element to make your code go > fast as it avoids a large number of binary constructions. > >> {ok, Socket} = gen_tcp:connect(?APPLIANCE_HOST, ?APPLIANCE_PORT, >> [inet, >> binary, >> {active, false}, >> {packet, 0}]), >> ok = gen_tcp:send(Socket, JsonData), >> Result = gen_tcp:recv(Socket, 4096, 5000), > > The Erlang semantics here are different. You are blocking until you > have exactly 4096 bytes of data. If less is received, you will get > {error, closed} back, or a timeout after 5000 ms. You should be > calling gen_tcp:recv(Socket, 0, 5000) and then use byte_size(X) on the > result yourself. It also means you will have to drain the socket > yourself in a separate function. The solution is to figure out what > the stream-semantics and syntax is and handle that with a couple of > Erlang functions. > > Another important thing to look out for is that in Erlang the Socket > is a port() bound to the connecting process (by default -- the > controlling process can change with the controlling_process call). If > the bound process dies, the socket closes. As soon as you begin > passing sockets between processes this will haunt you unless you are > aware of it. The cool thing about it as that you get automatic > "resource reclamation" because of this. It also happens with files > (they are closed), ETS tables, and so on. > > -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From dmitrii@REDACTED Fri Jun 17 14:05:58 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Fri, 17 Jun 2011 15:05:58 +0300 Subject: [erlang-questions] Joe Armstrong's proposal for a test/benchmark implementation Message-ID: Now that a week passed since Erlang Factory, I suddenly remembered one thing we talked during the Birds of a Feather discussion. I think it was originally Joe Armstrong's idea, but I could be wrong. The idea was to build a more-or-less simple site that would force the user to login, do something on the site and then logout. We could build this in various languages (and I assume frameworks) and see how Erlang performs in comparison to other languages. This would not be a purely synthetic test like those on Debian Shootout, but a more or less real-world scenario. I remember myself saying that we could take the dscussion to the list and and arrive at a basic idea how to do this. So here it is ? the idea. Is anyone still interested? From watson.timothy@REDACTED Fri Jun 17 14:14:42 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 17 Jun 2011 13:14:42 +0100 Subject: [erlang-questions] Joe Armstrong's proposal for a test/benchmark implementation In-Reply-To: References: Message-ID: > > I remember myself saying that we could take the dscussion to the list and and arrive at a basic idea how to do this. > If you want it to be "real world" then your application should include talking to a database or other external service (pref over a network). I like the idea of having a kind of log-in-then-do-something approach. What are you going to use for benchmarking? Tsung? Something else? From mikhail.sobolev@REDACTED Fri Jun 17 14:22:17 2011 From: mikhail.sobolev@REDACTED (Mikhail Sobolev) Date: Fri, 17 Jun 2011 15:22:17 +0300 Subject: [erlang-questions] In-memory files Message-ID: Hi, As part of my application I get a binary that needs to be processed by a function that only accepts io_device(). After checking the source code (14B03), I found that it's possible to supply 'ram' as one of the options to file:open/2 and provide the binary as the first parameter. However the aforementioned function use file:read_line/1 and the ram_file module does not implement the necessary function. Can somebody please suggest how to work out this situation? -- Misha From cyplo@REDACTED Fri Jun 17 14:30:29 2011 From: cyplo@REDACTED (=?UTF-8?Q?Cyryl_P=C5=82otnicki=2DChudyk?=) Date: Fri, 17 Jun 2011 14:30:29 +0200 Subject: [erlang-questions] Joe Armstrong's proposal for a test/benchmark implementation In-Reply-To: References: Message-ID: 1up As competitors I would see Ruby on Rails , ASP.NET and some php stuff as these are among the most popular ones. The app might involve validating login via the webvervice/rest/json stuff and later, after the login, getting some data to display and to be edited from the postgres/mysql db The deployment cost should be measured somehow imho, e.g. how much time a admin needs to deploy new machine with each of the solutions. And I mean the whole time, installing all necessary frameworks/gems etc. -- cyryl From fred.hebert@REDACTED Fri Jun 17 14:40:17 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Fri, 17 Jun 2011 08:40:17 -0400 Subject: [erlang-questions] Joe Armstrong's proposal for a test/benchmark implementation In-Reply-To: References: Message-ID: <0CAB59D0-F3E2-4A5A-B4AC-C0BC08F00E5C@erlang-solutions.com> That's an issue with the idea of 'real world'. Real world will vary a lot depending on who you're talking to. Oh yes, my sessions can be stored in memory on memcached, no issue with that, just buy more memory. Hm, you're testing this on this kind of hardware? too bad, this kind of software really runs better on a different kind of it! Did they configure this right? because this is pretty important. Oh sorry, my implementation works better on some other OS... To test real 'real world' solutions, you need to give full freedom to the benchmarkee to build their own environment and an app the way they want it. You might be testing a LAMP stack compared to a LYME (Linux Yaws MySQL Erlang) stack, and there will be plenty of suggestions as to how to make one faster than the other. The rabbit hole only gets deeper at each step. I do support the effort though. I think it would be better than microbenchmarks, although the tests would likely still not be reliable to judge on the performance of an application X. -- Fred H?bert http://www.erlang-solutions.com On 2011-06-17, at 08:14 AM, Tim Watson wrote: >> >> I remember myself saying that we could take the dscussion to the list and and arrive at a basic idea how to do this. >> > > If you want it to be "real world" then your application should include > talking to a database or other external service (pref over a network). > I like the idea of having a kind of log-in-then-do-something approach. > > What are you going to use for benchmarking? Tsung? Something else? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Fri Jun 17 15:08:56 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 17 Jun 2011 14:08:56 +0100 Subject: [erlang-questions] Joe Armstrong's proposal for a test/benchmark implementation In-Reply-To: <0CAB59D0-F3E2-4A5A-B4AC-C0BC08F00E5C@erlang-solutions.com> References: <0CAB59D0-F3E2-4A5A-B4AC-C0BC08F00E5C@erlang-solutions.com> Message-ID: > > To test real 'real world' solutions, you need to give full freedom to the benchmarkee to build their own environment and an app the way they want it. You might be testing a LAMP stack compared to a LYME (Linux Yaws MySQL Erlang) stack, and there will be plenty of suggestions as to how to make one faster than the other. > > The rabbit hole only gets deeper at each step. I do support the effort though. I think it would be better than microbenchmarks, although the tests would likely still not be reliable to judge on the performance of an application X. > Perhaps what is needed is a framework that makes it easier to construct a larger/broader kind of benchmark, that accounts for things like the OS & hw environment, configuration, and does so both for the (test) clients and the application(s) being tested. A lot of effort has gone into automating deployments and infrastructure configuration in recent years - capistrano/fabric, puppet and chef, etc - and various efforts (such as Vagrant) into using virtualisation to make testing against multiple architectures/environments easier. In fact IIRC erlang-solutions have built something that does CI/Build of Erlang/OTP across various configurations and environments automatically, am I right? So I wonder if bringing together some of these things - build + deployment automation, environment/infrastructure configuration, testing + benchmarking tools such as Tsung and basho_bench, etc - into a "something" that make it possible (or even "not-utterly-painful" perhaps) to simplify and/or automate these things for the purposes of testing stuff, might be very useful. I'm pretty sure that the "something" which will make this easier is fairly composite, but it would be nice if you didn't have start from scratch when trying to do something like this (comparing different tools, platforms, architectures and so on). It might also be impossibly complicated to do right. :) From ebegumisa@REDACTED Fri Jun 17 15:23:56 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Fri, 17 Jun 2011 23:23:56 +1000 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: On Fri, 17 Jun 2011 20:43:51 +1000, James Churchman wrote: > making something with a similar-ish api to something else is not reverse > engineering. its a rewrite from scratch. I didn't say that it was, and neither did Ulf. Ulf just raised a hypothetical chain of events which may be interpreted by _some_ people reading _some_ statutes as reverse engineering. In said hypothetical legal dispute -- what *you* and *I* think is reverse engineering doesn't matter. What matters would be the aggrieved parties argument for interpretation of the statute, and any precedents supporting such an interpretation, but most importantly, their evidence for the facts and chain of events that lead up to the dispute, and ultimately how well both sides can convince the court in session. In such a dispute, the defendant/respondent may well present the the points you raise as part of their side of the case. My point was in regards to whom would be liable to whom in such a dispute. - Edmond - > reverse engineering would be disassembling the beam files to discover > how they work, simply reimplementing them it is not. if that was the > case there would be one cd player, one car, one bike, one washing > machine, one fridge, one operating system etc.. in the world and never > ever a second version, and having a copy of the original will make no > difference, if you clean room something, as long as you don't break any > patents in the process. > > On 17 Jun 2011, at 04:30, Edmond Begumisa wrote: > >> On Fri, 17 Jun 2011 12:56:59 +1000, Edmond Begumisa >> wrote: >> >>> Just to clarify/reinforce, what gets a little confusing in these >>> discussions are the rights and obligations between licensor and >>> licensee, which is a contractual arrangement and subject to the >>> doctrine of privity. >>> >> >> CORRECTION: This was inaccurate. A licensee agreement is not strictly >> contractual agreement, but a license may be passed as consideration >> exchanged as part of a contractual agreement. Nevertheless, the privy >> doctrine usually applies to agreements in which licenses are past in >> most jurisdictions. >> >>> [Continuing with the mentioned hypothetical] >>> >>> * If the PropEr developers have copies of QuickCheck Mini, then they >>> bind themselves to a license agreement with Quviq, including the >>> promise not to reverse engineer. >>> * If PropEr is GPL'ed, and Vlad uses it, he binds himself to an >>> agreement with the PropEr developers, and the terms thereunder. >>> >>> Vlad is not privy to the first agreement, and thus can never be held >>> liable for its breach, if there is such a breach. And he cannot be >>> expected to stop using a version he already has. (Though he might have >>> the rug pulled out from under him, should PropEr development cease and >>> future versions stop being made.) >>> >>> However, assuming all parties are in Europe, and although the concept >>> of freedom of contract generally applies, they also bind themselves >>> doctrine that they can never make an agreement that contravenes >>> codified EU law - including said reverse-engineering law in said >>> copyright statute. Then the aggrieved party need not be privy to an >>> agreement to have their rights enforced since these rights are not >>> established by an agreement, but by a statute. In this case, Vlad and >>> all PropEr users in Europe could well be ordered by a court to cease >>> using the program regardless of any agreement they have with it's >>> developers (since such an agreement would be considered invalidated.) >>> Hypothetically speaking. >>> >>> That said: Erlang being a small-friendly community -- I assumed (and >>> like to continue to a assume) that PropEr and QuickCheck developers >>> have not only been aware of each others projects, but I also like to >>> think that they've been in touch with each other and sorted out any >>> such concerns before version 1 of PropEr was announced. Like Erlang, I >>> prefer to think of the happy case! >>> >>> - Edmond - >>> >>> >>> On Thu, 16 Jun 2011 18:23:04 +1000, Ulf Wiger >>> wrote: >>> >>>> >>>> On 16 Jun 2011, at 10:01, Vlad Dumitrescu wrote: >>>> >>>>> On Thu, Jun 16, 2011 at 08:43, Ulf Wiger >>>>> wrote: >>>>> >>>>> On 16 Jun 2011, at 08:30, Vlad Dumitrescu wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> This kind of issues are worse than flame wars! :-) >>>>>> >>>>>> I have yet another point of view that increases the confusion: >>>>>> PropEr is mostly compatible with Triq and QuickCheck (there's a >>>>>> free older version). >>>>> >>>>> Well, there is a version available in jungerl, but to my knowledge, >>>>> it was not put there by the authors, and not actually intended to be >>>>> free. Even so, one might consider it a lot less sensitive to copy >>>>> that version than the later, proprietary, versions of QuickCheck. >>>>> >>>>> Quviq released an official mini-version last year. >>>>> >>>>> http://www.quviq.com/news100621.html >>>> >>>> Yes, but the QuickCheck Mini license agreement, while permitting free >>>> use and re-distribution, specifically forbids reverse-engineering. >>>> Thus, we should hope that the PropEr developers did not obtain a copy >>>> of QuickCheck Mini and ran it in order to learn how it worked, as >>>> this would put them in violation of the EU Directives regarding >>>> software copyright. >>>> >>>> For those who are inspired by PropEr, but are put off by the GPL, >>>> using and re-distributing QuickCheck Mini is of course unproblematic. >>>> ;-) >>>> >>>> BR, >>>> Ulf W >>>> >>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>>> http://erlang-solutions.com >>>> >>>> >>>> >>> >>> >> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From peralta.alejandro@REDACTED Fri Jun 17 15:28:05 2011 From: peralta.alejandro@REDACTED (Ale) Date: Fri, 17 Jun 2011 10:28:05 -0300 Subject: [erlang-questions] MPI and Erlang. (fun experiment) In-Reply-To: <375EEF4D-4315-4FEB-B53F-280DE9965CC8@cs.otago.ac.nz> References: <1308293184.5047.61.camel@seasc1137> <375EEF4D-4315-4FEB-B53F-280DE9965CC8@cs.otago.ac.nz> Message-ID: Hello Richard, Bengt Thanks so much for your comments! I will correct my program as you suggested and look into HiPe. I appreciate taking time to look at this. I'm don't spend as much as time as I would like with Erlang (I "play" with Erlang in my spare time, most of the time I work with Python). I'll try improve my toy benchmark and let you know what happens :-) Regards, -- Ale. From joelr1@REDACTED Fri Jun 17 16:04:26 2011 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jun 2011 15:04:26 +0100 Subject: [erlang-questions] MPI and Erlang. (fun experiment) In-Reply-To: <375EEF4D-4315-4FEB-B53F-280DE9965CC8@cs.otago.ac.nz> References: <1308293184.5047.61.camel@seasc1137> <375EEF4D-4315-4FEB-B53F-280DE9965CC8@cs.otago.ac.nz> Message-ID: <7B779D33-AB0C-4A9F-94AA-960B0BAF3C9A@gmail.com> On Jun 17, 2011, at 9:33 AM, Richard O'Keefe wrote: > The big issue here is that with adequate type information, HiPe can avoid > boxing and unboxing a lot of floating point numbers, so it's quite likely > that you can get a factor of 2 to 4 speedup fairly easily. How can you check what HiPE is doing here? Is there an option to dump some pseudo-code that shows what's being boxed, unboxed, allocated, etc.? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From ericbmerritt@REDACTED Fri Jun 17 16:07:52 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Fri, 17 Jun 2011 09:07:52 -0500 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <1F6ADFE3-528A-4483-BB77-F3B7C670B36B@erlang-solutions.com> <4DF75350.7020208@gmail.com> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: Guys, None of us are lawyers (I think ;) so lets leave the legalities to the lawyers. If you are really worried about things buy a QuickCheck license otherwise Kostis has produced a good open project that finally makes property based testing (an extreme improvement over simple unit testing) accessible to the average joe, from a price perspective yes, but even more importantly from an openness perspective. Kostis is working on the license issue and I suspect will come out with something pretty acceptable. Eric On Fri, Jun 17, 2011 at 8:23 AM, Edmond Begumisa wrote: > On Fri, 17 Jun 2011 20:43:51 +1000, James Churchman > wrote: > >> making something with a similar-ish api to something else is not reverse >> engineering. its a rewrite from scratch. > > I didn't say that it was, and neither did Ulf. > > Ulf just raised a hypothetical chain of events which may be interpreted by > _some_ people reading _some_ statutes as reverse engineering. > > In said hypothetical legal dispute -- what *you* and *I* think is reverse > engineering doesn't matter. What matters would be the aggrieved parties > argument for interpretation of the statute, and any precedents supporting > such an interpretation, but most importantly, their evidence for the facts > and chain of events that lead up to the dispute, and ultimately how well > both sides can convince the court in session. In such a dispute, the > defendant/respondent may well present the the points you raise as part of > their side of the case. > > My point was in regards to whom would be liable to whom in such a dispute. > > - Edmond - > >> reverse engineering would be disassembling the beam files to discover how >> they work, simply reimplementing them it is not. if that was the case there >> would be one cd player, one car, one bike, one washing machine, one fridge, >> one operating system etc.. in the world and never ever a second version, and >> having a copy of the original will make no difference, if you clean room >> something, as long as you don't break any patents in the process. >> >> On 17 Jun 2011, at 04:30, Edmond Begumisa wrote: >> >>> On Fri, 17 Jun 2011 12:56:59 +1000, Edmond Begumisa >>> wrote: >>> >>>> Just to clarify/reinforce, what gets a little confusing in these >>>> discussions are the rights and obligations between licensor and licensee, >>>> which is a contractual arrangement and subject to the doctrine of privity. >>>> >>> >>> CORRECTION: This was inaccurate. A licensee agreement is not strictly >>> contractual agreement, but a license may be passed as consideration >>> exchanged as part of a contractual agreement. Nevertheless, the privy >>> doctrine usually applies to agreements in which licenses are past in most >>> jurisdictions. >>> >>>> [Continuing with the mentioned hypothetical] >>>> >>>> * If the PropEr developers have copies of QuickCheck Mini, then they >>>> bind themselves to a license agreement with Quviq, including the promise not >>>> to reverse engineer. >>>> * If PropEr is GPL'ed, and Vlad uses it, he binds himself to an >>>> agreement with the PropEr developers, and the terms thereunder. >>>> >>>> Vlad is not privy to the first agreement, and thus can never be held >>>> liable for its breach, if there is such a breach. And he cannot be expected >>>> to stop using a version he already has. (Though he might have the rug pulled >>>> out from under him, should PropEr development cease and future versions stop >>>> being made.) >>>> >>>> However, assuming all parties are in Europe, and although the concept of >>>> freedom of contract generally applies, they also bind themselves doctrine >>>> that they can never make an agreement that contravenes codified EU law - >>>> including said reverse-engineering law in said copyright statute. Then the >>>> aggrieved party need not be privy to an agreement to have their rights >>>> enforced since these rights are not established by an agreement, but by a >>>> statute. In this case, Vlad and all PropEr users in Europe could well be >>>> ordered by a court to cease using the program regardless of any agreement >>>> they have with it's developers (since such an agreement would be considered >>>> invalidated.) Hypothetically speaking. >>>> >>>> That said: Erlang being a small-friendly community -- I assumed (and >>>> like to continue to a assume) that PropEr and QuickCheck developers have not >>>> only been aware of each others projects, but I also like to think that >>>> they've been in touch with each other and sorted out any such concerns >>>> before version 1 of PropEr was announced. Like Erlang, I prefer to think of >>>> the happy case! >>>> >>>> - Edmond - >>>> >>>> >>>> On Thu, 16 Jun 2011 18:23:04 +1000, Ulf Wiger >>>> wrote: >>>> >>>>> >>>>> On 16 Jun 2011, at 10:01, Vlad Dumitrescu wrote: >>>>> >>>>>> On Thu, Jun 16, 2011 at 08:43, Ulf Wiger >>>>>> wrote: >>>>>> >>>>>> On 16 Jun 2011, at 08:30, Vlad Dumitrescu wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> This kind of issues are worse than flame wars! :-) >>>>>>> >>>>>>> I have yet another point of view that increases the confusion: PropEr >>>>>>> is mostly compatible with Triq and QuickCheck (there's a free older >>>>>>> version). >>>>>> >>>>>> Well, there is a version available in jungerl, but to my knowledge, it >>>>>> was not put there by the authors, and not actually intended to be free. Even >>>>>> so, one might consider it a lot less sensitive to copy that version than the >>>>>> later, proprietary, versions of QuickCheck. >>>>>> >>>>>> Quviq released an official mini-version last year. >>>>>> >>>>>> http://www.quviq.com/news100621.html >>>>> >>>>> Yes, but the QuickCheck Mini license agreement, while permitting free >>>>> use and re-distribution, specifically forbids reverse-engineering. Thus, we >>>>> should hope that the PropEr developers did not obtain a copy of QuickCheck >>>>> Mini and ran it in order to learn how it worked, as this would put them in >>>>> violation of the EU Directives regarding software copyright. >>>>> >>>>> For those who are inspired by PropEr, but are put off by the GPL, using >>>>> and re-distributing QuickCheck Mini is of course unproblematic. ;-) >>>>> >>>>> BR, >>>>> Ulf W >>>>> >>>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>>>> http://erlang-solutions.com >>>>> >>>>> >>>>> >>>> >>>> >>> >>> >>> -- >>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ebegumisa@REDACTED Fri Jun 17 16:29:52 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sat, 18 Jun 2011 00:29:52 +1000 Subject: [erlang-questions] A PropEr announcement In-Reply-To: References: <4DF4BDDA.30809@cs.ntua.gr> <4D3BB04F-4B38-4404-AF0B-62CC28D0ADEF@gmail.com> <4DF89262.5060500@cs.ntua.gr> <53d61dfa-9f50-4e54-8b50-deb2d9dbd6de@g12g2000yqd.googlegroups.com> Message-ID: I love PropEr! One of it's early adopters (I think). - Edmond - On Sat, 18 Jun 2011 00:07:52 +1000, Eric Merritt wrote: > Guys, > > None of us are lawyers (I think ;) so lets leave the legalities to > the lawyers. If you are really worried about things buy a QuickCheck > license otherwise Kostis has produced a good open project that finally > makes property based testing (an extreme improvement over simple unit > testing) accessible to the average joe, from a price perspective yes, > but even more importantly from an openness perspective. Kostis is > working on the license issue and I suspect will come out with > something pretty acceptable. > > Eric > > On Fri, Jun 17, 2011 at 8:23 AM, Edmond Begumisa > wrote: >> On Fri, 17 Jun 2011 20:43:51 +1000, James Churchman >> wrote: >> >>> making something with a similar-ish api to something else is not >>> reverse >>> engineering. its a rewrite from scratch. >> >> I didn't say that it was, and neither did Ulf. >> >> Ulf just raised a hypothetical chain of events which may be interpreted >> by >> _some_ people reading _some_ statutes as reverse engineering. >> >> In said hypothetical legal dispute -- what *you* and *I* think is >> reverse >> engineering doesn't matter. What matters would be the aggrieved parties >> argument for interpretation of the statute, and any precedents >> supporting >> such an interpretation, but most importantly, their evidence for the >> facts >> and chain of events that lead up to the dispute, and ultimately how well >> both sides can convince the court in session. In such a dispute, the >> defendant/respondent may well present the the points you raise as part >> of >> their side of the case. >> >> My point was in regards to whom would be liable to whom in such a >> dispute. >> >> - Edmond - >> >>> reverse engineering would be disassembling the beam files to discover >>> how >>> they work, simply reimplementing them it is not. if that was the case >>> there >>> would be one cd player, one car, one bike, one washing machine, one >>> fridge, >>> one operating system etc.. in the world and never ever a second >>> version, and >>> having a copy of the original will make no difference, if you clean >>> room >>> something, as long as you don't break any patents in the process. >>> >>> On 17 Jun 2011, at 04:30, Edmond Begumisa wrote: >>> >>>> On Fri, 17 Jun 2011 12:56:59 +1000, Edmond Begumisa >>>> wrote: >>>> >>>>> Just to clarify/reinforce, what gets a little confusing in these >>>>> discussions are the rights and obligations between licensor and >>>>> licensee, >>>>> which is a contractual arrangement and subject to the doctrine of >>>>> privity. >>>>> >>>> >>>> CORRECTION: This was inaccurate. A licensee agreement is not strictly >>>> contractual agreement, but a license may be passed as consideration >>>> exchanged as part of a contractual agreement. Nevertheless, the privy >>>> doctrine usually applies to agreements in which licenses are past in >>>> most >>>> jurisdictions. >>>> >>>>> [Continuing with the mentioned hypothetical] >>>>> >>>>> * If the PropEr developers have copies of QuickCheck Mini, then they >>>>> bind themselves to a license agreement with Quviq, including the >>>>> promise not >>>>> to reverse engineer. >>>>> * If PropEr is GPL'ed, and Vlad uses it, he binds himself to an >>>>> agreement with the PropEr developers, and the terms thereunder. >>>>> >>>>> Vlad is not privy to the first agreement, and thus can never be held >>>>> liable for its breach, if there is such a breach. And he cannot be >>>>> expected >>>>> to stop using a version he already has. (Though he might have the >>>>> rug pulled >>>>> out from under him, should PropEr development cease and future >>>>> versions stop >>>>> being made.) >>>>> >>>>> However, assuming all parties are in Europe, and although the >>>>> concept of >>>>> freedom of contract generally applies, they also bind themselves >>>>> doctrine >>>>> that they can never make an agreement that contravenes codified EU >>>>> law - >>>>> including said reverse-engineering law in said copyright statute. >>>>> Then the >>>>> aggrieved party need not be privy to an agreement to have their >>>>> rights >>>>> enforced since these rights are not established by an agreement, but >>>>> by a >>>>> statute. In this case, Vlad and all PropEr users in Europe could >>>>> well be >>>>> ordered by a court to cease using the program regardless of any >>>>> agreement >>>>> they have with it's developers (since such an agreement would be >>>>> considered >>>>> invalidated.) Hypothetically speaking. >>>>> >>>>> That said: Erlang being a small-friendly community -- I assumed (and >>>>> like to continue to a assume) that PropEr and QuickCheck developers >>>>> have not >>>>> only been aware of each others projects, but I also like to think >>>>> that >>>>> they've been in touch with each other and sorted out any such >>>>> concerns >>>>> before version 1 of PropEr was announced. Like Erlang, I prefer to >>>>> think of >>>>> the happy case! >>>>> >>>>> - Edmond - >>>>> >>>>> >>>>> On Thu, 16 Jun 2011 18:23:04 +1000, Ulf Wiger >>>>> wrote: >>>>> >>>>>> >>>>>> On 16 Jun 2011, at 10:01, Vlad Dumitrescu wrote: >>>>>> >>>>>>> On Thu, Jun 16, 2011 at 08:43, Ulf Wiger >>>>>>> wrote: >>>>>>> >>>>>>> On 16 Jun 2011, at 08:30, Vlad Dumitrescu wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> This kind of issues are worse than flame wars! :-) >>>>>>>> >>>>>>>> I have yet another point of view that increases the confusion: >>>>>>>> PropEr >>>>>>>> is mostly compatible with Triq and QuickCheck (there's a free >>>>>>>> older >>>>>>>> version). >>>>>>> >>>>>>> Well, there is a version available in jungerl, but to my >>>>>>> knowledge, it >>>>>>> was not put there by the authors, and not actually intended to be >>>>>>> free. Even >>>>>>> so, one might consider it a lot less sensitive to copy that >>>>>>> version than the >>>>>>> later, proprietary, versions of QuickCheck. >>>>>>> >>>>>>> Quviq released an official mini-version last year. >>>>>>> >>>>>>> http://www.quviq.com/news100621.html >>>>>> >>>>>> Yes, but the QuickCheck Mini license agreement, while permitting >>>>>> free >>>>>> use and re-distribution, specifically forbids reverse-engineering. >>>>>> Thus, we >>>>>> should hope that the PropEr developers did not obtain a copy of >>>>>> QuickCheck >>>>>> Mini and ran it in order to learn how it worked, as this would put >>>>>> them in >>>>>> violation of the EU Directives regarding software copyright. >>>>>> >>>>>> For those who are inspired by PropEr, but are put off by the GPL, >>>>>> using >>>>>> and re-distributing QuickCheck Mini is of course unproblematic. ;-) >>>>>> >>>>>> BR, >>>>>> Ulf W >>>>>> >>>>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>>>>> http://erlang-solutions.com >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From dmitrii@REDACTED Fri Jun 17 16:34:16 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Fri, 17 Jun 2011 17:34:16 +0300 Subject: [erlang-questions] Joe Armstrong's proposal for a test/benchmark implementation In-Reply-To: References: Message-ID: <158A66EA-02E4-4D82-A3B6-7A0DF46B6714@dmitriid.com> On Jun 17, 2011, at 3:30 PM, Cyryl P?otnicki-Chudyk wrote: > 1up > > As competitors I would see Ruby on Rails , ASP.NET and some php stuff > as these are among the most popular ones. > The app might involve validating login via the webvervice/rest/json > stuff and later, after the login, getting some data to display and to > be edited from the postgres/mysql db > > The deployment cost should be measured somehow imho, e.g. how much > time a admin needs to deploy new machine with each of the solutions. > And I mean the whole time, installing all necessary frameworks/gems > etc. If we take Fred's words into account, then for each system we test we might need to make installation as easy and painless as possible so that ideally anyone could just download the code, deploy it and have a benchmark on their hands relevant to their hardware From dmitrii@REDACTED Fri Jun 17 16:36:23 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Fri, 17 Jun 2011 17:36:23 +0300 Subject: [erlang-questions] Joe Armstrong's proposal for a test/benchmark implementation In-Reply-To: <0CAB59D0-F3E2-4A5A-B4AC-C0BC08F00E5C@erlang-solutions.com> References: <0CAB59D0-F3E2-4A5A-B4AC-C0BC08F00E5C@erlang-solutions.com> Message-ID: > That's an issue with the idea of 'real world'. Real world will vary a lot depending on who you're talking to. Oh yes, my sessions can be stored in memory on memcached, no issue with that, just buy more memory. > > Hm, you're testing this on this kind of hardware? too bad, this kind of software really runs better on a different kind of it! > > Did they configure this right? because this is pretty important. > > Oh sorry, my implementation works better on some other OS... > > To test real 'real world' solutions, you need to give full freedom to the benchmarkee to build their own environment and an app the way they want it. You might be testing a LAMP stack compared to a LYME (Linux Yaws MySQL Erlang) stack, and there will be plenty of suggestions as to how to make one faster than the other. > Definitely. It was mentioned in passing, actually, that it should be easy for people to download, deploy and benchmark the system on their own > The rabbit hole only gets deeper at each step. I do support the effort though. I think it would be better than microbenchmarks, although the tests would likely still not be reliable to judge on the performance of an application X. > Well, there's lies, damned lies and statistics. And only after that there's benchmarks (which are basically statistics in disguise) :) > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-06-17, at 08:14 AM, Tim Watson wrote: > >>> >>> I remember myself saying that we could take the dscussion to the list and and arrive at a basic idea how to do this. >>> >> >> If you want it to be "real world" then your application should include >> talking to a database or other external service (pref over a network). >> I like the idea of having a kind of log-in-then-do-something approach. >> >> What are you going to use for benchmarking? Tsung? Something else? >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From dmitrii@REDACTED Fri Jun 17 16:37:55 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Fri, 17 Jun 2011 17:37:55 +0300 Subject: [erlang-questions] Joe Armstrong's proposal for a test/benchmark implementation In-Reply-To: References: <0CAB59D0-F3E2-4A5A-B4AC-C0BC08F00E5C@erlang-solutions.com> Message-ID: >> >> To test real 'real world' solutions, you need to give full freedom to the benchmarkee to build their own environment and an app the way they want it. You might be testing a LAMP stack compared to a LYME (Linux Yaws MySQL Erlang) stack, and there will be plenty of suggestions as to how to make one faster than the other. >> >> The rabbit hole only gets deeper at each step. I do support the effort though. I think it would be better than microbenchmarks, although the tests would likely still not be reliable to judge on the performance of an application X. >> > > Perhaps what is needed is a framework that makes it easier to > construct a larger/broader kind of benchmark, that accounts for things > like the OS & hw environment, configuration, and does so both for the > (test) clients and the application(s) being tested. A lot of effort > has gone into automating deployments and infrastructure configuration > in recent years - capistrano/fabric, puppet and chef, etc - and > various efforts (such as Vagrant) into using virtualisation to make > testing against multiple architectures/environments easier. In fact > IIRC erlang-solutions have built something that does CI/Build of > Erlang/OTP across various configurations and environments > automatically, am I right? > > So I wonder if bringing together some of these things - build + > deployment automation, environment/infrastructure configuration, > testing + benchmarking tools such as Tsung and basho_bench, etc - into > a "something" that make it possible (or even "not-utterly-painful" > perhaps) to simplify and/or automate these things for the purposes of > testing stuff, might be very useful. > > I'm pretty sure that the "something" which will make this easier is > fairly composite, but it would be nice if you didn't have start from > scratch when trying to do something like this (comparing different > tools, platforms, architectures and so on). It might also be > impossibly complicated to do right. :) Hmm... I never thought of it this way :) It sounds like a lot of work, but it would definitely be nice to arrive to something like that in the end. From daniel@REDACTED Fri Jun 17 16:32:49 2011 From: daniel@REDACTED (Daniel Luna) Date: Fri, 17 Jun 2011 16:32:49 +0200 Subject: [erlang-questions] MPI and Erlang. (fun experiment) In-Reply-To: <7B779D33-AB0C-4A9F-94AA-960B0BAF3C9A@gmail.com> References: <1308293184.5047.61.camel@seasc1137> <375EEF4D-4315-4FEB-B53F-280DE9965CC8@cs.otago.ac.nz> <7B779D33-AB0C-4A9F-94AA-960B0BAF3C9A@gmail.com> Message-ID: On 17 June 2011 16:04, Joel Reymont wrote: > How can you check what HiPE is doing here? > > Is there an option to dump some pseudo-code that shows what's being boxed, unboxed, allocated, etc.? hipe:c(MODULE, [pp_asm]). will probably do what you want. There is also pp_beam and pp_icode (and pp_all). pp_icode could be the level you want. Cheers, Daniel -- http://havebackpack.com - Exploring the world with a 30L backpack. Available for fun challenging assignments anywhere in the world. From robert.virding@REDACTED Fri Jun 17 17:21:40 2011 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 17 Jun 2011 15:21:40 +0000 (GMT) Subject: [erlang-questions] Everything Everywhere receives marketing award In-Reply-To: <431896814.311621308323736416.JavaMail.root@zimbra> Message-ID: <2001156527.311761308324100781.JavaMail.root@zimbra> Until you realise that "Everything Everywhere" is actually the name of a company the message's subject seems very disparaging about awards: big deal, everything everywhere receives marketing awards. :-) Robert ----- "Chandru" wrote: > Thank you Ivan. > We certainly use a lot of Erlang and OTP behaviours in Everything Everywhere, and it is great fun working here! So if anyone is looking for a job hacking Erlang in a telecoms environment, please drop me a mail offline. I currently have one permanent and two contracting positions open. > cheers Chandru > > > On 17 June 2011 08:34, Ivan Uemlianin < ivan@REDACTED > wrote: > Dear All > > I hope this isn't too off-topic. > > I just saw Everything Everywhere received an award from the Institute of Promotional Marketing: Gold for Employee Recognition. > > http://www.theipm.org.uk/The_ Institute_of_Promotional_ Marketing_Awards/list_of_IPM_ Awards_2011_winners.aspx > > Their campaign was called "The Bee's Knees" and the blurb says: > > A range of bee assets, a bee lexicon and bee-haviours > were developed ... > > OTP bee-haviours perhaps? > > Best wishes (and congrats to Everything Everywhere people if any are reading) > > Ivan > > > -- > ============================== ============================== > Ivan A. Uemlianin > Speech Technology Research and Development > > ivan@REDACTED > www.llaisdy.com > llaisdy.wordpress.com > www.linkedin.com/in/ ivanuemlianin > > "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" > (Schiller, Beethoven) > ============================== ============================== > ______________________________ _________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/ listinfo/erlang-questions > > > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Jun 17 17:35:18 2011 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Fri, 17 Jun 2011 17:35:18 +0200 Subject: [erlang-questions] Joe Armstrong's proposal for a test/benchmark implementation In-Reply-To: <158A66EA-02E4-4D82-A3B6-7A0DF46B6714@dmitriid.com> References: <158A66EA-02E4-4D82-A3B6-7A0DF46B6714@dmitriid.com> Message-ID: <4DFB7436.5050903@dev-extend.eu> On 06/17/2011 04:34 PM, Dmitrii Dimandt wrote: > > On Jun 17, 2011, at 3:30 PM, Cyryl P?otnicki-Chudyk wrote: > >> 1up >> >> As competitors I would see Ruby on Rails , ASP.NET and some php stuff >> as these are among the most popular ones. >> The app might involve validating login via the webvervice/rest/json >> stuff and later, after the login, getting some data to display and to >> be edited from the postgres/mysql db >> >> The deployment cost should be measured somehow imho, e.g. how much >> time a admin needs to deploy new machine with each of the solutions. >> And I mean the whole time, installing all necessary frameworks/gems >> etc. > > If we take Fred's words into account, then for each system we test we might need to make installation as easy and painless as possible so that ideally anyone could just download the code, deploy it and have a benchmark on their hands relevant to their hardware While not a bad idea per se, this is unfortunately not really representative as only small applications require no or few system changes to make them perform well. If you benchmark scaling to 1k simultaneous connections then it's all fine and well, but Erlang can scale to millions of simultaneous connections on the same server and that's not something easily tested. General benchmarks like you propose are more useful as a testing method than actual performance comparison. For example you could detect performance issues on one or another platform that you don't have at your disposal. They're also a very good marketing tool as most people trust them without question. That's why you get many meaningless benchmarks in blogs for example. So sure, that's not necessarily a bad idea, but if you are going to tell people to install your app + benchmark to get an idea of the performance of different products, you should also sensibilize them about this and give a few pointers on how they might improve the performance of all products on their system through system tweaks and more. This is pretty much a full-time job. Regardless I'm still looking forward to see php and ruby getting their ass kicked even though the benchmarks would probably be biased. -- Lo?c Hoguin Dev:Extend From skruger@REDACTED Fri Jun 17 18:27:58 2011 From: skruger@REDACTED (Shaun Kruger) Date: Fri, 17 Jun 2011 10:27:58 -0600 Subject: [erlang-questions] Joe Armstrong's proposal for a test/benchmark implementation In-Reply-To: <0CAB59D0-F3E2-4A5A-B4AC-C0BC08F00E5C@erlang-solutions.com> References: <0CAB59D0-F3E2-4A5A-B4AC-C0BC08F00E5C@erlang-solutions.com> Message-ID: <1308328078.2092.71.camel@quadratic> On Fri, 2011-06-17 at 08:40 -0400, Fr?d?ric Trottier-H?bert wrote: > To test real 'real world' solutions, you need to give full freedom to the benchmarkee to build their own environment and an app the way they want it. You might be testing a LAMP stack compared to a LYME (Linux Yaws MySQL Erlang) stack, and there will be plenty of suggestions as to how to make one faster than the other. > > The rabbit hole only gets deeper at each step. I do support the effort though. I think it would be better than microbenchmarks, although the tests would likely still not be reliable to judge on the performance of an application X. > The one test that every benchmark misses is the "How well does this language help me ship software" test. Selecting for performance is great when you're trying to avoid selecting a language that is going to bog you down. However, once you determine that a language is going to work "well enough" you should select based on what you're trying to do and accept that developer time is more expensive than hardware. Develop a benchmark that takes into account the cost of the time spent by development and admin teams to accomplish the benchmark and you will be the smartest man alive. :) That being said, I started my HTTP proxy project 9 months ago. Erlang has enabled me to deliver it faster and with more features than any other language I could have selected. When someone suggests that I shouldn't use language X because it doesn't have good benchmarks I instantly find myself wondering how long it has been since that person did anything useful. Shaun https://github.com/skruger/Surrogate From williamstw@REDACTED Fri Jun 17 18:41:41 2011 From: williamstw@REDACTED (Tim Williams) Date: Fri, 17 Jun 2011 12:41:41 -0400 Subject: [erlang-questions] Joe Armstrong's proposal for a test/benchmark implementation In-Reply-To: References: Message-ID: On Fri, Jun 17, 2011 at 8:05 AM, Dmitrii Dimandt wrote: > Now that a week passed since Erlang Factory, I suddenly remembered one thing we talked during the Birds of a Feather discussion. > > I think it was originally Joe Armstrong's idea, but I could be wrong. > > The idea was to build a more-or-less simple site that would force the user to login, do something on the site and then logout. We could build this in various languages (and I assume frameworks) and see how Erlang performs in comparison to other languages. This would not be a purely synthetic test like those on Debian Shootout, but a more or less real-world scenario. > > I remember myself saying that we could take the dscussion to the list and and arrive at a basic idea how to do this. > > So here it is ? the idea. Is anyone still interested? It seems it's not the "real" goal in what you suggest, but a nice side effect is that it would be incredibly helpful for introducing Erlang for those coming from other web stuff. A simple bootstrapped web app that a new person could poke around and learn how things work would be incredibly useful IMHO. Analogous to "fallback[1]" for a heavy Java stack... anyway, just suggesting it could be a nice side effect... --tim [1] - https://github.com/lookfirst/fallback From joelr1@REDACTED Fri Jun 17 22:39:59 2011 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 17 Jun 2011 21:39:59 +0100 Subject: [erlang-questions] erlang thrit bindings and TJSONProtocol Message-ID: <2B9E17AF-1B25-4902-B0B8-779C43CB688A@gmail.com> Do Erlang bindings for Thrift support TJSONProtocol? http://sundresh.org/docs/thrift-0.2.0-javadoc/org/apache/thrift/protocol/TJSONProtocol.html -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From anthonym@REDACTED Fri Jun 17 23:09:22 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Fri, 17 Jun 2011 14:09:22 -0700 Subject: [erlang-questions] erlang thrit bindings and TJSONProtocol In-Reply-To: <2B9E17AF-1B25-4902-B0B8-779C43CB688A@gmail.com> References: <2B9E17AF-1B25-4902-B0B8-779C43CB688A@gmail.com> Message-ID: <20110617210922.GA50776@alumni.caltech.edu> I don't believe so, currently only base64 and binary protocols. However, if you feel like taking on creation of this protocol, I'm willing to look it over and possibly merge it (I'm a maintainer/committer). -Anthony On Fri, Jun 17, 2011 at 09:39:59PM +0100, Joel Reymont wrote: > Do Erlang bindings for Thrift support TJSONProtocol? > > http://sundresh.org/docs/thrift-0.2.0-javadoc/org/apache/thrift/protocol/TJSONProtocol.html > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ------------------------------------------------------------------------ Anthony Molinaro From peralta.alejandro@REDACTED Sat Jun 18 04:44:04 2011 From: peralta.alejandro@REDACTED (Ale) Date: Fri, 17 Jun 2011 23:44:04 -0300 Subject: [erlang-questions] MPI and Erlang. (fun experiment) In-Reply-To: <375EEF4D-4315-4FEB-B53F-280DE9965CC8@cs.otago.ac.nz> References: <1308293184.5047.61.camel@seasc1137> <375EEF4D-4315-4FEB-B53F-280DE9965CC8@cs.otago.ac.nz> Message-ID: Richard, Bengt: I changed the code with you suggestions and here are my results: $ ./run.sh Hostname: dell-desktop Proc[0]: iterating from 0 to 50000000 Proc[1]: iterating from 50000000 to 100000000 Result: 1.8390715290761144 real 0m10.988s user 0m9.405s sys 0m0.044s Thank you guys. Regards, -- Ale. From avinash@REDACTED Sat Jun 18 04:48:23 2011 From: avinash@REDACTED (Avinash Dhumane) Date: Sat, 18 Jun 2011 08:18:23 +0530 (IST) Subject: [erlang-questions] Hierarchical FSMs Message-ID: <615850.88810.qm@web5602.biz.mail.in.yahoo.com> I'm new entrant in Erlang's world, and have read only Joe's thesis so far, and I'm not a programmer :-( How to realise (implement) FSMs organised as state (superstate, substate) and event (superevent, subevent) hierarchies using gen_fsm? I see these hierarchies in dynamic models of Rumbaugh OMT. Thoughts, or a quick example (code)? Thanks Avinash -------------- next part -------------- An HTML attachment was scrubbed... URL: From dang@REDACTED Sat Jun 18 04:59:45 2011 From: dang@REDACTED (Daniel Goertzen) Date: Fri, 17 Jun 2011 21:59:45 -0500 Subject: [erlang-questions] Hierarchical FSMs In-Reply-To: <615850.88810.qm@web5602.biz.mail.in.yahoo.com> References: <615850.88810.qm@web5602.biz.mail.in.yahoo.com> Message-ID: Take a look at plain_fsm (not part of OTP). Dan. On Fri, Jun 17, 2011 at 9:48 PM, Avinash Dhumane wrote: > I'm new entrant in Erlang's world, and have read only Joe's thesis so far, > and I'm not a programmer :-( > > How to realise (implement) FSMs organised as state (superstate, substate) > and event (superevent, subevent) hierarchies using gen_fsm? I see these > hierarchies in dynamic models of Rumbaugh OMT. > > Thoughts, or a quick example (code)? > > Thanks > Avinash > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Daniel Goertzen ----------------- dang@REDACTED (work) daniel.goertzen@REDACTED (home) ----------------- 1 204 272 6149 (home/office) 1 204 470 8360 (mobile) ----------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From icarus.alive@REDACTED Sat Jun 18 09:55:53 2011 From: icarus.alive@REDACTED (Icarus Alive) Date: Sat, 18 Jun 2011 13:25:53 +0530 Subject: [erlang-questions] problem setting up multi-node comm on Windows 7 Message-ID: What I would like to do is to set up communication between these 2 instances -- C:> erl -name abc@REDACTED -setcookie erlide C:> erl -name xyz@REDACTED -setcookie erlide however, when I try -- (abc@REDACTED)6> net_adm:ping('xyz@REDACTED'). pang (abc@REDACTED)7> nodes(). [] I can successfully do this:- C:> ping -4 FELIX.somedomainsomewhere.com C:> tracert FELIX.somedomainsomewhere.com which shows that the domain-name resolves fine. Instead of "-name" if I do the same experiment with "-sname" and drop the usage of host.FQDN, the result is same, i.e. a "pang" ! How do I get it to ping successfully ? My whole investigation started from the "freeze"/"hang" issues with usage of latest "erlide", and investigating, I found the eclipse to erlide-backend communication to be the issue. cheers, Icarus From dmitrii@REDACTED Sat Jun 18 10:56:27 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Sat, 18 Jun 2011 11:56:27 +0300 Subject: [erlang-questions] Joe Armstrong's proposal for a test/benchmark implementation In-Reply-To: References: Message-ID: On Jun 17, 2011, at 7:41 PM, Tim Williams wrote: > On Fri, Jun 17, 2011 at 8:05 AM, Dmitrii Dimandt wrote: >> Now that a week passed since Erlang Factory, I suddenly remembered one thing we talked during the Birds of a Feather discussion. >> >> I think it was originally Joe Armstrong's idea, but I could be wrong. >> >> The idea was to build a more-or-less simple site that would force the user to login, do something on the site and then logout. We could build this in various languages (and I assume frameworks) and see how Erlang performs in comparison to other languages. This would not be a purely synthetic test like those on Debian Shootout, but a more or less real-world scenario. >> >> I remember myself saying that we could take the dscussion to the list and and arrive at a basic idea how to do this. >> >> So here it is ? the idea. Is anyone still interested? > > It seems it's not the "real" goal in what you suggest, but a nice side > effect is that it would be incredibly helpful for introducing Erlang > for those coming from other web stuff. A simple bootstrapped web app > that a new person could poke around and learn how things work would be > incredibly useful IMHO. Analogous to "fallback[1]" for a heavy Java > stack... anyway, just suggesting it could be a nice side effect... > > --tim > > [1] - https://github.com/lookfirst/fallback Agreed! :) From erlang@REDACTED Sat Jun 18 13:57:12 2011 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 18 Jun 2011 13:57:12 +0200 Subject: [erlang-questions] Any tiddlywiki/tiddlespace fans out there? Message-ID: I've been playing with shared tiddlers at tiddlyspace.com. It's brilliant. Try playing with it for a few hours - it's great stuff. I started a shared space at espace.tiddlyspace.com - If you want volunteer for a project I have in mind take a look there Shared tiddlyspaces provide an amazing way to organize data and collaborate. A tiddly space feels like a smalltalk image - but the way this is done feels much more "web friendly" than smalltalk. Now I want to make a tiddlyspace backend server in Erlang. This (shared tiddlyspaces) seems to be crying out for a highly scalable Key-Value backend store. Has anybody implemented the tiddlywiki REST protocol in Erlang? I would also be interested in collaborating with anybody who has first hand knowledge of hacking the tiddlywiki front-ends. The front-end and back-ends are cleanly separated by a REST interface which allows a very clean separation of interests. Javascript in the frontend <=> Erlang in the backend Cheers /Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Sat Jun 18 14:00:57 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 18 Jun 2011 13:00:57 +0100 Subject: [erlang-questions] Any tiddlywiki/tiddlespace fans out there? In-Reply-To: References: Message-ID: Tiddlywiki is pretty cool. I know some of the Osmosoft guys from work (they're part of BT) so I'll ask about this on the internal developers mailing list and let you know what they say. Cheers, Tim On 18 June 2011 12:57, Joe Armstrong wrote: > > I've been playing with shared tiddlers at tiddlyspace.com. It's brilliant. > Try playing with it for a few hours - it's great stuff. > > I started a shared space at espace.tiddlyspace.com - If you want > volunteer for a project I have in mind take a look there > > Shared tiddlyspaces provide an amazing way to organize data and > collaborate. > > A tiddly space feels like a smalltalk image - but the way this is > done feels much more "web friendly" than smalltalk. > > Now I want to make a tiddlyspace backend server in Erlang. > > > This (shared tiddlyspaces) seems to be crying out for a > highly scalable Key-Value backend store. > > Has anybody implemented the tiddlywiki REST protocol in Erlang? > > I would also be interested in collaborating with anybody who has first > hand knowledge of hacking the tiddlywiki front-ends. > > The front-end and back-ends are cleanly separated by a REST interface > which allows a very clean separation of interests. > > Javascript in the frontend <=> Erlang in the backend > > Cheers > > /Joe > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From avinash@REDACTED Sat Jun 18 15:55:20 2011 From: avinash@REDACTED (Avinash Dhumane) Date: Sat, 18 Jun 2011 19:25:20 +0530 (IST) Subject: [erlang-questions] Hierarchical FSMs In-Reply-To: References: <615850.88810.qm@web5602.biz.mail.in.yahoo.com> Message-ID: <285134.16104.qm@web5607.biz.mail.in.yahoo.com> Both gen_fsm and plain_fsm appear to support only "flat", and not hierarchical, FSMs. Maybe, the mental model of hierarchical FSMs has basic impedance mismatch with Erlang's view of the world - "everything is a process". I need to see "processes" rather than events and states of "objects" in the problem domain. Thanks by all means! ________________________________ From: Daniel Goertzen To: Avinash Dhumane Cc: erlang-questions@REDACTED Sent: Sat, 18 June, 2011 8:29:45 AM Subject: Re: [erlang-questions] Hierarchical FSMs Take a look at plain_fsm (not part of OTP). Dan. On Fri, Jun 17, 2011 at 9:48 PM, Avinash Dhumane wrote: I'm new entrant in Erlang's world, and have read only Joe's thesis so far, and I'm not a programmer :-( > >How to realise (implement) FSMs organised as state (superstate, substate) and >event (superevent, subevent) hierarchies using gen_fsm? I see these hierarchies >in dynamic models of Rumbaugh OMT. > >Thoughts, or a quick example (code)? > >Thanks >Avinash >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions > > -- Daniel Goertzen ----------------- dang@REDACTED (work) daniel.goertzen@REDACTED (home) ----------------- 1 204 272 6149 (home/office) 1 204 470 8360 (mobile) ----------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth.lundin@REDACTED Sat Jun 18 16:10:10 2011 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Sat, 18 Jun 2011 16:10:10 +0200 Subject: [erlang-questions] (no subject) Message-ID: -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Sat Jun 18 19:21:03 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 18 Jun 2011 19:21:03 +0200 Subject: [erlang-questions] Hierarchical FSMs Message-ID: It is true that hierarchical state machines are not that common in Erlang. One reason is that erlang's lightweight processes make it easier to split fsms into multiple, smaller fsms, and another is that erlang's selective semantics help to "hide" much of the state-event matrix, simplifying the fsm. That said, it can be useful with hierarchical state machines sometimes. What one can do is model the subordinate fsms after gen_fsm, but keeping the state as a parameter in the enclosing fsm. If you keep the mapping faithful enough, you can test the subordinate fsm as a real gen_fsm. Only a little care is needed to ensure that the code can work well in both modes. BR, Ulf W Ulf Wiger CTO, Erlang Solutions, Ltd http://www.erlang-solutions.com Avinash Dhumane wrote: >Both gen_fsm and plain_fsm appear to support only "flat", and not hierarchical, >FSMs. > >Maybe, the mental model of hierarchical FSMs has basic impedance mismatch with >Erlang's view of the world - "everything is a process". > >I need to see "processes" rather than events and states of "objects" in the >problem domain. > >Thanks by all means! > > > >________________________________ >From: Daniel Goertzen >To: Avinash Dhumane >Cc: erlang-questions@REDACTED >Sent: Sat, 18 June, 2011 8:29:45 AM >Subject: Re: [erlang-questions] Hierarchical FSMs > >Take a look at plain_fsm (not part of OTP). > >Dan. > > >On Fri, Jun 17, 2011 at 9:48 PM, Avinash Dhumane wrote: > >I'm new entrant in Erlang's world, and have read only Joe's thesis so far, and >I'm not a programmer :-( >> >>How to realise (implement) FSMs organised as state (superstate, substate) and >>event (superevent, subevent) hierarchies using gen_fsm? I see these hierarchies >>in dynamic models of Rumbaugh OMT. >> >>Thoughts, or a quick example (code)? >> >>Thanks >>Avinash >>_______________________________________________ >>erlang-questions mailing list >>erlang-questions@REDACTED >>http://erlang.org/mailman/listinfo/erlang-questions >> >> > > >-- >Daniel Goertzen >----------------- >dang@REDACTED (work) >daniel.goertzen@REDACTED (home) >----------------- >1 204 272 6149 (home/office) >1 204 470 8360 (mobile) >----------------- >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions From mevans@REDACTED Sat Jun 18 19:54:58 2011 From: mevans@REDACTED (Evans, Matthew) Date: Sat, 18 Jun 2011 13:54:58 -0400 Subject: [erlang-questions] Freedom Box and Erlang Message-ID: Hi, I've recently been reading about Freedom box (see: http://wiki.debian.org/FreedomBox and http://freedomboxfoundation.org/). In a nutshell, directly from their website: "Smart devices whose engineered purpose is to work together to facilitate free communication among people, safely and securely, beyond the ambition of the strongest power to penetrate, they can make freedom of thought and information a permanent, ineradicable feature of the net that holds our souls." "We live in a world where our use of the network is mediated by organizations that often do not have our best interests at heart. By building software that does not rely on a central service, we can regain control and privacy. By keeping our data in our homes, we gain useful legal protections over it. By giving back power to the users over their networks and machines, we are returning the Internet to its intended peer-to-peer architecture. " I'm not a member of the organization, but it seems to me that the distributed nature of Erlang could offer awesome possibilities to work around certain technical difficulties. I could, for example, envisage a mesh of Erlang nodes connected together providing various distributed services where no individual is in overall control. Or you could provide services (Erlang processes/application) owned and managed by someone from a country with poor human rights that is in fact running on an Erlang node on a country with better human rights (a chat service for example). I'm not talking about "cloud" services here (that some organization ultimately owns), but running applications on a mesh of privately owned "Freedom boxes" that host Erlang VMs. The beauty of Erlang is that it's innate ability to distribute code, processes and applications in such a simple and seamless manner could provide very powerful tools for the organization. Any comments and thoughts on this? Matt From mjtruog@REDACTED Sat Jun 18 21:14:33 2011 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 18 Jun 2011 12:14:33 -0700 Subject: [erlang-questions] Freedom Box and Erlang In-Reply-To: References: Message-ID: <4DFCF919.5070106@gmail.com> To have an Erlang cloud that doesn't depend on an organization, there is CloudI (http://cloudi.org) . So it could be used for running applications on a distributed mesh. There is ZeroMQ integration that could provide communication between the FreedomBoxes. CloudI currently lacks a pretty UI for managing the configuration and will be going through more testing, so those are current barriers. On 06/18/2011 10:54 AM, Evans, Matthew wrote: > Hi, > > I've recently been reading about Freedom box (see: http://wiki.debian.org/FreedomBox and http://freedomboxfoundation.org/). > > In a nutshell, directly from their website: > > "Smart devices whose engineered purpose is to work together to facilitate free communication among people, safely and securely, beyond the ambition of the strongest power to penetrate, they can make freedom of thought and information a permanent, ineradicable feature of the net that holds our souls." > > "We live in a world where our use of the network is mediated by organizations that often do not have our best interests at heart. By building software that does not rely on a central service, we can regain control and privacy. By keeping our data in our homes, we gain useful legal protections over it. By giving back power to the users over their networks and machines, we are returning the Internet to its intended peer-to-peer architecture. " > > I'm not a member of the organization, but it seems to me that the distributed nature of Erlang could offer awesome possibilities to work around certain technical difficulties. I could, for example, envisage a mesh of Erlang nodes connected together providing various distributed services where no individual is in overall control. Or you could provide services (Erlang processes/application) owned and managed by someone from a country with poor human rights that is in fact running on an Erlang node on a country with better human rights (a chat service for example). I'm not talking about "cloud" services here (that some organization ultimately owns), but running applications on a mesh of privately owned "Freedom boxes" that host Erlang VMs. The beauty of Erlang is that it's innate ability to distribute code, processes and applications in such a simple and seamless manner could provide very powerful tools for the organization. > > Any comments and thoughts on this? > > Matt > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From overminddl1@REDACTED Sat Jun 18 21:16:59 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Sat, 18 Jun 2011 13:16:59 -0600 Subject: [erlang-questions] problem setting up multi-node comm on Windows 7 In-Reply-To: References: Message-ID: You sure that there is no firewall? On Jun 18, 2011 1:56 AM, "Icarus Alive" wrote: > What I would like to do is to set up communication between these 2 instances -- > > C:> erl -name abc@REDACTED -setcookie erlide > C:> erl -name xyz@REDACTED -setcookie erlide > > however, when I try -- > > (abc@REDACTED)6> > net_adm:ping('xyz@REDACTED'). > pang > (abc@REDACTED)7> nodes(). > [] > > I can successfully do this:- > C:> ping -4 FELIX.somedomainsomewhere.com > C:> tracert FELIX.somedomainsomewhere.com > which shows that the domain-name resolves fine. > > Instead of "-name" if I do the same experiment with "-sname" and drop > the usage of host.FQDN, the result is same, i.e. a "pang" ! > > How do I get it to ping successfully ? > > My whole investigation started from the "freeze"/"hang" issues with > usage of latest "erlide", and investigating, I found the eclipse to > erlide-backend communication to be the issue. > > cheers, > Icarus > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Sat Jun 18 21:26:47 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Sat, 18 Jun 2011 21:26:47 +0200 Subject: [erlang-questions] problem setting up multi-node comm on Windows 7 In-Reply-To: References: Message-ID: Hi, On Sat, Jun 18, 2011 at 09:55, Icarus Alive wrote: > What I would like to do is to set up communication between these 2 > instances -- > > C:> erl -name abc@REDACTED -setcookie erlide > C:> erl -name xyz@REDACTED -setcookie erlide > > however, when I try -- > > (abc@REDACTED)6> > net_adm:ping('xyz@REDACTED'). > pang > (abc@REDACTED)7> nodes(). > [] > > I can successfully do this:- > C:> ping -4 FELIX.somedomainsomewhere.com > C:> tracert FELIX.somedomainsomewhere.com > which shows that the domain-name resolves fine. > > Instead of "-name" if I do the same experiment with "-sname" and drop > the usage of host.FQDN, the result is same, i.e. a "pang" ! > > How do I get it to ping successfully ? I checked on my machine and I got the same problem between two Erlang nodes. The Java node connects fine because it doesn't care about a long name being a FQDN or not. In my case, the problem was that I have the Home edition and I'm not connected to any domain. I fixed it by editing c:\windows\system32\drivers\etc\hosts and adding an entry like 192.168.1.2 mypc mypc.localdomain This lets me connect Erlang nodes that use mypc.localdomain as -name. My guess is that it is something similar in your case, please check this file and see what it contains. regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From krab@REDACTED Sat Jun 18 23:15:23 2011 From: krab@REDACTED (Kresten Krab Thorup) Date: Sat, 18 Jun 2011 23:15:23 +0200 Subject: [erlang-questions] problem setting up multi-node comm on Windows 7 In-Reply-To: References: Message-ID: Erlang will do a dns lookup and it must match the reverse; so if you don't control dns then it usually works to use name@REDACTED. On 18/06/2011, at 21.27, "Vlad Dumitrescu" > wrote: Hi, On Sat, Jun 18, 2011 at 09:55, Icarus Alive <icarus.alive@REDACTED> wrote: What I would like to do is to set up communication between these 2 instances -- C:> erl -name abc@REDACTED -setcookie erlide C:> erl -name xyz@REDACTED -setcookie erlide however, when I try -- (abc@REDACTED)6> net_adm:ping('xyz@REDACTED'). pang (abc@REDACTED)7> nodes(). [] I can successfully do this:- C:> ping -4 FELIX.somedomainsomewhere.com C:> tracert FELIX.somedomainsomewhere.com which shows that the domain-name resolves fine. Instead of "-name" if I do the same experiment with "-sname" and drop the usage of host.FQDN, the result is same, i.e. a "pang" ! How do I get it to ping successfully ? I checked on my machine and I got the same problem between two Erlang nodes. The Java node connects fine because it doesn't care about a long name being a FQDN or not. In my case, the problem was that I have the Home edition and I'm not connected to any domain. I fixed it by editing c:\windows\system32\drivers\etc\hosts and adding an entry like 192.168.1.2 mypc mypc.localdomain This lets me connect Erlang nodes that use mypc.localdomain as -name. My guess is that it is something similar in your case, please check this file and see what it contains. regards, Vlad _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From pfisher@REDACTED Sat Jun 18 23:25:57 2011 From: pfisher@REDACTED (Paul Fisher) Date: Sat, 18 Jun 2011 16:25:57 -0500 Subject: [erlang-questions] Photos from the Erlang Factory In-Reply-To: <860089743.306471308222346212.JavaMail.root@zimbra> References: <860089743.306471308222346212.JavaMail.root@zimbra> Message-ID: <3C1E545A-65D9-49D1-A516-A47D38FF1F5E@alertlogic.com> It was really great to put names to faces. Thank you Roberto! On Jun 16, 2011, at 6:05 AM, Roberto Aloi wrote: > Hello everyone, > > sorry if I misuse this mailinglist but I know that most (if not all of them) of the delegates and speakers at the latest Erlang Factory conference are listening here. > > The photos from the conference are finally online: > > http://www.flickr.com/photos/erlang-consulting/sets/72157626849482931/ > > I promise I won't blackmail Kostis, featured in a PropEr hug... > > Regards, > > Roberto Aloi > -- > University of Kent - Erlang Solutions Ltd. > Blog: http://aloiroberto.wordpress.com > Twitter: @robertoaloi > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- paul director, platform services alertlogic, inc. 713-484-8383 x2314 From pfisher@REDACTED Sat Jun 18 23:43:59 2011 From: pfisher@REDACTED (Paul Fisher) Date: Sat, 18 Jun 2011 16:43:59 -0500 Subject: [erlang-questions] Exciting opportunities for Erlang developers @ Alert Logic Message-ID: <58EEAB46-6EDB-4541-9A3C-862582620C16@alertlogic.com> I just wanted to let everyone know that we are expanding our platform technologies group at Alert Logic,. We are looking for Lead & Senior Software Engineers, specifically three (3) people in Houston, Texas, and six (6) and St. Petersburg, Russia through our partner Reksoft. A bit about the positions: "The Lead/Senior Platform Software Engineer works to deliver ground-breaking technology for Alert Logic, an established and rapidly growing Software-as-a-Service company located in the Galleria area of Houston, TX. In this role, you will lead or participate in one or more of the following technology areas: ? Extend our scalable map-reduce computing platform ? Apply statistical methods and machine learning techniques to real-time data analysis ? Scale our storage technology from managing petabytes, to 10s or even 100s of petabytes of data ? Research and develop solutions around emerging hardware technologies (e.g. GPUs)" The direct link to full details and the job description is available at: http://newton.newtonsoftware.com/career/JobIntroduction.action?clientId=4028f88c2d4c007b012d6385331315b4&id=8aa005063062881c01309b863adb368a&specialization=4028f88c2d4c007b012d6385331315be&gnewtonResize=http://newton.newtonsoftware.com/career/GnewtonResize.htm&source= All available positions can be found at: http://www.alertlogic.com/careers - Lead/Senior Platform Engineers Please apply online, or feel free to contact me directly about these positions. -- paul director, platform services alertlogic, inc. 713-484-8383 x2314 pfisher@REDACTED From zsolt.erl@REDACTED Sun Jun 19 04:43:18 2011 From: zsolt.erl@REDACTED (Zsolt Keszthelyi) Date: Sat, 18 Jun 2011 22:43:18 -0400 (EDT) Subject: [erlang-questions] bug in timer module in R14B02? Message-ID: Dear All, I ran into something in R14B02. It looks like if I start a timer with a long interval then all previously started timers stop. The timer server is still running and the timers are in the ETS table. No error returned. Nothing crashes. This was working on R13B03 and is not working on R14B02 The long interval is exactly 2^31/1000 ms. Is it a bug or an intentional change? Maybe the timer module was never meant to be used with long intervals but shouldn't it give at least an error msg.? Code: %% works on R13B03 %% doesn't work on R14B02 -module(timerbug). -compile(export_all). start()-> timer:apply_interval(1000, io, format, ["Hello!\n"]), timer:sleep(5000), timer:apply_interval(40*60*1000, io, format, ["No more hello.\n"]). From dan@REDACTED Sun Jun 19 06:23:23 2011 From: dan@REDACTED (Daniel Dormont) Date: Sun, 19 Jun 2011 00:23:23 -0400 Subject: [erlang-questions] BEAM backward compatibility Message-ID: Since I work mostly in Java, when I started learning Erlang I recognized .beam files as being at least somewhat like .class files and assumed they would be backward compatible - that is, beams compiled in R12 would run in R13 and R14 and there shouldn't be any problems unless I were calling experimental APIs or deprecated APIs that were later removed. But a discussion on the ejabberd mailing list suggested that this might not be the case. I didn't see anything offhand in the official documentation. What's the word on this? thanks, Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth.lundin@REDACTED Sun Jun 19 10:55:26 2011 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Sun, 19 Jun 2011 10:55:26 +0200 Subject: [erlang-questions] BEAM backward compatibility In-Reply-To: References: Message-ID: .beam files are definately backwards compatible meaning that files created with an older compiler can be loaded and run in a later release. We have as a strategy to atleast support backwards compatibility 2 major releases back in time. IThis means that .beam files created with OTP R12B something can be used in R13 and R14 as well. The same strategy applies to the distribution protocol between Erlang nodes. I.e a R12B based node can connect and talk to a R14 node. This does not apply to files containing native code, i.e produced with the HiPE compiler. /Kenneth Erlang/OTP, Ericsson On Sun, Jun 19, 2011 at 6:23 AM, Daniel Dormont wrote: > Since I work mostly in Java, when I started learning Erlang I recognized > .beam files as being? at least somewhat like .class files and assumed they > would be backward compatible - that is, beams compiled in R12 would run in > R13 and R14 and there shouldn't be any problems unless I were calling > experimental APIs or deprecated APIs that were later removed. But a > discussion on the ejabberd mailing list suggested that this might not be the > case. I didn't see anything offhand in the official documentation. What's > the word on this? > > thanks, > Dan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From ivan@REDACTED Sun Jun 19 11:06:03 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Sun, 19 Jun 2011 10:06:03 +0100 Subject: [erlang-questions] Erlang for Speech Recognition Message-ID: <4DFDBBFB.6090909@llaisdy.com> Dear All I am developing a suite of software tools for use in language technology (primarily speech recognition). A lot of the procedures in speech technology have a very "map-reduce" feel to them and I think Erlang would be a good fit. Below I list and briefly describe the tools I'm developing. Does anyone know if there are similar current Erlang projects? (I have looked). As the whole lot is needed for a speech rec system, I don't quite know how I should proceed: should I write the easiest component first (probably the language model builder/server), the hardest (probably the audio preprocessor), the most useful outside of speech recognition (probably the hidden Markov model builder/server), ...? ** Audio Preprocessor Automatic Speech Recognition (ASR) is essentially mapping a sequence of integers (i.e., acoustic signals) onto a sequence of linguistic symbols (i.e., phonemes (units of linguistic sound) or words). The raw audio data (e.g. from a wav file or a microphone) is not terribly useful for this and the first step is to convert this data into a more useful abstract representation. Each 100ms of sound is transformed into a feature vector of 39 features, known as Mel-Frequency Cepstral Co-efficients (MFCCs). The first step in recognition, or in training a recogniser, is always to convert the audio like this. A while ago I wrote a little script to read and write wav files [1]. I also have a 'dummy' make_feats.erl which sets out the imagined tasks for converting audio data into MFCCs. However, there are two possible ways ahead: 1. Write the whole lot from scratch in Erlang. 2. There is a mature, respected open-source speech recognition toolkit, written in C called Sphinx [2]. Sphinx has a make_feats function which could be called as a port or a NIF. The Sphinx make_feats doesn't work quite how I'd like, so some changes would have to be made (Sphinx is released under a BSD-style license). (2) would avoid working out how to implement the maths, but (1) would avoid fiddling around inside someone else's C. Any advice? ** Hidden Markov Model Builder/Server A hidden markov Model (HMM) is essentially a finite state machine with probabilities given to the edges connecting states. We train up a HMM for each phoneme of a language (HMMs are trained using MFCC sequences). The foundational recognition task is recognising a single phoneme (this is then conditioned by probabilities of different phoneme sequences). So, we take an MFCC sequence, match it up against each HMM in turn and ask, "What is the probability that this HMM could have produced this MFCC sequence?" Although used mainly in ASR, HMMs can be used in speech synthesis (aka text-to-speech), and they are used outside speech tech of course (e.g. in finance [3]). I have written a toy HMM trainer and recogniser, for simple symbol sequences. I think the sensible next step would be to tone this up and test it again simple real world data (I could compare performance and results with the R HMM package [4]). Once that seems stable, enhance the code to work with sequences of real number vectors, build a phoneme recogniser and compare with Sphinx. The set of HMMs is referred to as the Acoustic Model (AM) of the language. Other mathematical models can be used but, since at least the mid-80s, HMMs dominate. There is some interesting recent work using dynamic Bayesian networks, and using conditional random fields. ** Language Model Builder/Server The Language Model (LM) furnishes probabilities of various linguistic structures, and sits on top off, or collaborates with, the AM. Sequences of phonemes are dealt with by a simple pronunciation dictionary, which is just a list mapping sequences of phonemes to words. Just as HMMs dominate AMs, the dominant model for syntactic structures is the ngram grammar, which assigns probabilities to sequences of words (the most common 'n' is 3, often called a trigram). As well as their use in ASR, LMs are an essential component in statistical machine translation. I have written a toy LM builder, which assigns probabilities to trigrams based on a given corpus. I think the sensible next step would be to tone this up to work with large corpora, and compare performance and results against a standard open-source LM builder [5]. ** references [1] http://llaisdy.wordpress.com/2010/06/01/wave-erl-an-erlang-script-to-read-and-write-wav-files/ [2] http://cmusphinx.sourceforge.net/ [3] http://www.optirisk-systems.com/events/application-of-hidden-markov-models-and- filters-to-financial-time-series-data.asp [4] http://r-forge.r-project.org/projects/rhmm/ [5] i.e. irstlm (http://sourceforge.net/projects/irstlm/). There are several semi-open-source LM builders available for research use only, but afaik irstlm is the only one that is bona fide open-source. -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From bob.paddock@REDACTED Sun Jun 19 16:07:53 2011 From: bob.paddock@REDACTED (Bob Paddock) Date: Sun, 19 Jun 2011 10:07:53 -0400 Subject: [erlang-questions] Erlang for Speech Recognition In-Reply-To: <4DFDBBFB.6090909@llaisdy.com> References: <4DFDBBFB.6090909@llaisdy.com> Message-ID: > As the whole lot is needed for a speech rec system, I don't quite know how I > should proceed: should I write the easiest component first (probably the > language model builder/server), the hardest (probably the audio > preprocessor), the most useful outside of speech recognition (probably the > hidden Markov model builder/server), ...? You start with the audio processor, and the rest of the front end. If that doesn't work then any down stream work you do is wasted time. > ** Audio Preprocessor > >?The raw audio data > (e.g. from a wav file or a microphone) is not terribly useful for this and > the first step is to convert this data into a more useful abstract > representation. Consider a different approach such as Extrema Processing. EP removes the identity of the speaker from the intelligence of what the speaker said, which makes down stream matching easier. If you are trying to do a voice identification security application, where you need the identity of the speaker this approach would be useless. EP takes a signal input and converts it to the time domain with a differentiator. For example a pure sine wave input would give a transition when at the peak, and at the valley, of the sine where the direction of the slope changes direction. Now your template matcher matches these time domain signals. Adding out of band 'noise' may also help, similarly to the concept of dithering in a A/D converter. I've been meaning to build such a system Real Soon Now for far to long... > The foundational recognition task is recognizing a single phoneme If I remember my theory correctly, Phoneme's can be further broken down into Allophones. Important in the Identity vs Intelligence of speech. -- http://blog.softwaresafety.net/ http://www.designer-iii.com/ http://www.wearablesmartsensors.com/ From banibrata.dutta@REDACTED Sun Jun 19 16:45:40 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Sun, 19 Jun 2011 20:15:40 +0530 Subject: [erlang-questions] Erlang for Speech Recognition In-Reply-To: <4DFDBBFB.6090909@llaisdy.com> References: <4DFDBBFB.6090909@llaisdy.com> Message-ID: On Sun, Jun 19, 2011 at 2:36 PM, Ivan Uemlianin wrote: > ** Audio Preprocessor > > Automatic Speech Recognition (ASR) is essentially mapping a sequence of > integers (i.e., acoustic signals) onto a sequence of linguistic symbols > (i.e., phonemes (units of linguistic sound) or words). The raw audio data > (e.g. from a wav file or a microphone) is not terribly useful for this and > the first step is to convert this data into a more useful abstract > representation. Each 100ms of sound is transformed into a feature vector of > 39 features, known as Mel-Frequency Cepstral Co-efficients (MFCCs). > Doesn't that remember where I'd read this, but the topic was something like "what Erlang as a language is probably not well suited for...", and significant amount of mathematical calculation, was probably one of those. Has that stand changed ? Could it be a factor to consider ? -- regards, BDutta -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Sun Jun 19 16:58:54 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Sun, 19 Jun 2011 15:58:54 +0100 Subject: [erlang-questions] Erlang for Speech Recognition In-Reply-To: <20110619122243.GA54638@k2r.org> References: <4DFDBBFB.6090909@llaisdy.com> <20110619122243.GA54638@k2r.org> Message-ID: <4DFE0EAE.1070305@llaisdy.com> Dear Kenji Thanks for the tip-off. Husky looks interesting. My Japanese is not what it should be, but it looks like Husky uses HMMs produced by ATT and HTK, is that right? http://www.furui.cs.titech.ac.jp/~shinot/husky/runhusky.html Where it says: ?????????HMM???? ????????????????????????????????HTK?ATT?????????????????. But whatever part Husky does, it'll be very interesting to see how it's done in Haskell. I'll check out Takahiro Shinozaki's papers and see what he's written about Husky. Best wishes Ivan On 19/06/11 13:22, Kenji Rikitake wrote: > Takahiro Shinozaki has published a Haskell speech recognition program > called Husky. > http://www.furui.cs.titech.ac.jp/~shinot/husky/husky.hs > > FYI. > Kenji Rikitake > Note: I know very very little about speech recognition. -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Sun Jun 19 17:04:23 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Sun, 19 Jun 2011 16:04:23 +0100 Subject: [erlang-questions] Erlang for Speech Recognition In-Reply-To: References: <4DFDBBFB.6090909@llaisdy.com> Message-ID: <4DFE0FF7.7070203@llaisdy.com> Dear Bob Thanks for your comments. On 19/06/11 15:07, Bob Paddock wrote: > > You start with the audio processor, and the rest of the front end. > If that doesn't work then any down stream work you do is wasted time. Sensible. The audio processor is in the lead so far. > Consider a different approach such as Extrema Processing... I'm afraid I'm not familiar with Extrema Processing. Can you give me some pointers? Do you knoe if it's used in speech recognition? Using Mel-Frequency Cepstral Coefficients removes many speaker-dependent properties of the signal (like voice pitch). I don't know much about voice identification, but I imagine you'd abstract out a different set of features (e.g., you'd probably want to keep voice pitch). Best wishes Ivan -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From ivan@REDACTED Sun Jun 19 17:07:20 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Sun, 19 Jun 2011 16:07:20 +0100 Subject: [erlang-questions] Erlang for Speech Recognition In-Reply-To: References: <4DFDBBFB.6090909@llaisdy.com> Message-ID: <4DFE10A8.5050500@llaisdy.com> Dear Banibrata Thanks for your comment. On 19/06/11 15:45, Banibrata Dutta wrote: > > Doesn't that remember where I'd read this, but the topic was something > like "what Erlang as a language is probably not well suited for...", and > significant amount of mathematical calculation, was probably one of > those. Has that stand changed ? Could it be a factor to consider ? Yes, that's another factor leaning me towards using the Sphinx code (written in C) as a basis for a make_feats function and calling it as a port or a NIF. Best wishes Ivan -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From neumark.peter@REDACTED Sun Jun 19 20:48:35 2011 From: neumark.peter@REDACTED (Peter Neumark) Date: Sun, 19 Jun 2011 20:48:35 +0200 Subject: [erlang-questions] erlang thrit bindings and TJSONProtocol In-Reply-To: <20110617210922.GA50776@alumni.caltech.edu> References: <2B9E17AF-1B25-4902-B0B8-779C43CB688A@gmail.com> <20110617210922.GA50776@alumni.caltech.edu> Message-ID: I'm in the process of writing one now, but I'm not sure when it will be finished. If all goes well maybe sometime this week. Peter On Fri, Jun 17, 2011 at 11:09 PM, Anthony Molinaro wrote: > I don't believe so, currently only base64 and binary protocols. ?However, > if you feel like taking on creation of this protocol, I'm willing to look > it over and possibly merge it (I'm a maintainer/committer). > > -Anthony > > On Fri, Jun 17, 2011 at 09:39:59PM +0100, Joel Reymont wrote: >> Do Erlang bindings for Thrift support TJSONProtocol? >> >> http://sundresh.org/docs/thrift-0.2.0-javadoc/org/apache/thrift/protocol/TJSONProtocol.html >> >> -------------------------------------------------------------------------- >> - for hire: mac osx device driver ninja, kernel extensions and usb drivers >> ---------------------+------------+--------------------------------------- >> http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont >> ---------------------+------------+--------------------------------------- >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > ------------------------------------------------------------------------ > Anthony Molinaro ? ? ? ? ? ? ? ? ? ? ? ? ? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From amindfv@REDACTED Mon Jun 20 00:11:49 2011 From: amindfv@REDACTED (Tom Murphy) Date: Sun, 19 Jun 2011 18:11:49 -0400 Subject: [erlang-questions] Best Linux Distro for Erlang Devel Message-ID: Hi *, (I posted a very similar question to a Haskell list yesterday) If my choice of Lunix distro depended 100% on its solidness as an Erlang devel platform (completeness of base, packages, stability, support, robustness), what would you all recommend? Thanks for your time! Tom From mojitotech@REDACTED Mon Jun 20 00:21:12 2011 From: mojitotech@REDACTED (Mojito Sorbet) Date: Sun, 19 Jun 2011 18:21:12 -0400 Subject: [erlang-questions] code_change in the large In-Reply-To: <1050086879.302371308141341437.JavaMail.root@zimbra> References: <1050086879.302371308141341437.JavaMail.root@zimbra> Message-ID: <1308522072.10640.4.camel@sapphire> As I understand how code_change works, all processes using a module must be suspended before the switch can happen, the code is purged and reloaded, then they are all resumed. The application upgrade support takes care of the mechanics for this, but I wonder about the timing implications when I have possibly tens of thousands of processes using the same module. Are there any numbers on this? From octopusfluff@REDACTED Mon Jun 20 00:53:50 2011 From: octopusfluff@REDACTED (Amy Lear) Date: Sun, 19 Jun 2011 15:53:50 -0700 Subject: [erlang-questions] Status of Windows platform support in Erlang/OTP Message-ID: I set aside my fun with clock granularity once I knew what the problem and (hypothetical) solution are, since instrumentation was only a hypothetical concern for me. While taking a break from some other tasks, I started looking into what it would take to actually implement things like windows-specific high granularity clocks, and tripped across some interesting items. Right now, what I see seems to be the following: Erlang itself is not able to be fully compiled on Windows using open source tools; it requires VC++ as a command line compiler for much of it. Erlang doesn't seem to support 64-bit under Windows. HiPE is not compiled into the official Windows binaries. Research indicates that at one point in time, Erlang could be compiled using MinGW. If I understand correctly, things went somewhat awry around the time SMP support was added. The 64-bit problem seems to be a long =/= void sizing issue, and I wasn't able t find anything on HiPE. My questions are as follows: Has anything happened since then as far as efforts to be able to fully compile Erlang using mingw? The mingw/msys environment does provide a working shell and the ability to run configure scripts, etc, without bringing the whole cygwin cavalcade to the party. Has any effort gone into the cleanup necessary for being able to produce a 64-bit Erlang for Windows? The implication from what I found suggested it's mostly rote work that the core team doesn't have time to do. What exactly is the status of HiPE on the Windows platform? If work yet remains on these, is there a good starting point on how we might be able to push forward on these items? From dale@REDACTED Mon Jun 20 01:46:37 2011 From: dale@REDACTED (Dale Harvey) Date: Mon, 20 Jun 2011 00:46:37 +0100 Subject: [erlang-questions] Automated Stripping of otp libraries / modules Message-ID: I have seen this mentioned a few times but no answers as yet Has anyone written any code that takes an application and strips the otp libraries to the absolute minimum amount of modules* needed to run that application? I am assuming it cant be entirely automated by dialyzer (evalled / runtime generated calls etc) but it should get fairly close right? If noone has already done this then I guess any pointers for how to implement it would be cool, I figured I would just need generate a plt with otp + myapp + deps, find my applications entry point in the plt and traverse the callgraph marking modules off one at a time, that easy? Cheers Dale Harvey * I do mean modules and not just libraries -------------- next part -------------- An HTML attachment was scrubbed... URL: From dale@REDACTED Mon Jun 20 02:00:59 2011 From: dale@REDACTED (Dale Harvey) Date: Mon, 20 Jun 2011 01:00:59 +0100 Subject: [erlang-questions] bug in timer module in R14B02? In-Reply-To: References: Message-ID: I believe there was an integer overflow bug on 64 bit systems that affect ordered ets tables (and therefore caused timers not to trigger) in R14B02 http://erlang.org/pipermail/erlang-bugs/2011-April/002294.html That may be your problem? On 19 June 2011 03:43, Zsolt Keszthelyi wrote: > Dear All, > > I ran into something in R14B02. > It looks like if I start a timer with a long interval then all previously > started timers stop. > The timer server is still running and the timers are in the ETS table. > No error returned. Nothing crashes. > This was working on R13B03 and is not working on R14B02 > The long interval is exactly 2^31/1000 ms. > Is it a bug or an intentional change? > Maybe the timer module was never meant to be used with long intervals but > shouldn't it give at least an error msg.? > > Code: > > %% works on R13B03 > %% doesn't work on R14B02 > > -module(timerbug). > > -compile(export_all). > > start()-> > timer:apply_interval(1000, io, format, ["Hello!\n"]), > timer:sleep(5000), > timer:apply_interval(40*60***1000, io, format, ["No more hello.\n"]). > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Mon Jun 20 03:33:52 2011 From: mihai@REDACTED (Mihai Balea) Date: Sun, 19 Jun 2011 21:33:52 -0400 Subject: [erlang-questions] Best Linux Distro for Erlang Devel In-Reply-To: References: Message-ID: On Jun 19, 2011, at 6:11 PM, Tom Murphy wrote: > Hi *, > (I posted a very similar question to a Haskell list yesterday) > If my choice of Lunix distro depended 100% on its solidness as an > Erlang devel platform (completeness of base, packages, stability, > support, robustness), what would you all recommend? Packages that come with most popular distros tend to be outdated, and in some cases they even package unstable versions of the OTP environment (would really like to know what happened in the mind of the person who packaged R14A for Debian Squeeze) End result is that you will probably end up building from source anyway. So if you'll do that anyway, you might as well use the same platform you will use for deployment, which is usually either Debian or Redhat (or CentOS) Mihai From wmacgyver@REDACTED Mon Jun 20 03:53:23 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Sun, 19 Jun 2011 21:53:23 -0400 Subject: [erlang-questions] Best Linux Distro for Erlang Devel In-Reply-To: References: Message-ID: I use Fedora for both haskell and erlang. Fedora is fairly good about keeping up to date with both haskell platform and erlang providing you keep Fedora up to date. For example, the current Fedora 15 has Erlang R14B02. and haskell platform 2011.2.0.0. On Sun, Jun 19, 2011 at 6:11 PM, Tom Murphy wrote: > Hi *, > ? ? (I posted a very similar question to a Haskell list yesterday) > ?If my choice of Lunix distro depended 100% on its solidness as an > Erlang devel platform (completeness of base, packages, stability, > support, robustness), what would you all recommend? > > Thanks for your time! > Tom > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Omnem crede diem tibi diluxisse supremum. From ryanhuffman@REDACTED Mon Jun 20 04:02:18 2011 From: ryanhuffman@REDACTED (Ryan Huffman) Date: Sun, 19 Jun 2011 19:02:18 -0700 Subject: [erlang-questions] bug in timer module in R14B02? In-Reply-To: References: Message-ID: I ran into a lot of problems with the timer module on R14B02 and eventually switched to using erlang:send_after. This particular section in the erlang doc also convinced me to switch over to erlang:send_after and maintain my own timer: http://www.erlang.org/doc/efficiency_guide/commoncaveats.html#id58959 I was running on a 64-bit system (CentOS 5.5), so, as Dale mentioned, the problem may be fixed. Ryan On Sun, Jun 19, 2011 at 5:00 PM, Dale Harvey wrote: > I believe there was an integer overflow bug on 64 bit systems that affect > ordered ets tables (and therefore caused timers not to trigger) in R14B02 > > http://erlang.org/pipermail/erlang-bugs/2011-April/002294.html > > That may be your problem? > > > On 19 June 2011 03:43, Zsolt Keszthelyi wrote: > >> Dear All, >> >> I ran into something in R14B02. >> It looks like if I start a timer with a long interval then all previously >> started timers stop. >> The timer server is still running and the timers are in the ETS table. >> No error returned. Nothing crashes. >> This was working on R13B03 and is not working on R14B02 >> The long interval is exactly 2^31/1000 ms. >> Is it a bug or an intentional change? >> Maybe the timer module was never meant to be used with long intervals but >> shouldn't it give at least an error msg.? >> >> Code: >> >> %% works on R13B03 >> %% doesn't work on R14B02 >> >> -module(timerbug). >> >> -compile(export_all). >> >> start()-> >> timer:apply_interval(1000, io, format, ["Hello!\n"]), >> timer:sleep(5000), >> timer:apply_interval(40*60***1000, io, format, ["No more hello.\n"]). >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Mon Jun 20 06:35:58 2011 From: jwatte@REDACTED (Jon Watte) Date: Sun, 19 Jun 2011 21:35:58 -0700 Subject: [erlang-questions] Kill process if message mailbox reaches a certain size (was discarding signals) In-Reply-To: <006A2298-6FBA-4F87-8AFF-010CE4DDABA7@gmail.com> References: <3717CEF51B134A4F8073627839A9DF4120F405A85A@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A85B@ESESSCMS0353.eemea.ericsson.se> <3717CEF51B134A4F8073627839A9DF4120F405A9F8@ESESSCMS0353.eemea.ericsson.se> <006A2298-6FBA-4F87-8AFF-010CE4DDABA7@gmail.com> Message-ID: By "local networking" I mean "single-hop networking" -- you really want your Erlang cluster to be "local" in the sense that all the nodes share a single, non-blocking switch, for as far as you can push that (I know you can get 48-port 10-gig switches reasonably affordably; I imagine density is getting even higher). When you lease hardware on a cloud, you don't know how far away each instance will be, and it's pretty clear that you will share network resources with other virtual instances, so it's not a great fit for applications that are highly communications driven or require low latency. The latency is even more problematic, because virtualization will cause random scheduling jitter to the order of 30 milliseconds at best, and > 1000 milliseconds at worst, last I measured. I've never measured worse than 3 ms jitter on "bare metal" Linux-based installations. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Thu, Jun 16, 2011 at 12:55 AM, Joel Reymont wrote: > Jon, > > On Jun 16, 2011, at 9:09 AM, Jon Watte wrote: > > > I would advise against single-core nodes or cloud-based nodes that don't > have local networking, > > What do you mean by "don't have local networking"? > > I thought the loopback network interface is a given. > > > I also recommend modeling the traffic across the backplane of the Erlang > nodes. How much data do you send per user "event" to other users, and how > many other users? Broadcast or point-to-point? Sum it all up, double it, and > see if you can still swing that on your current network backplane. If not, > buy a bigger network, or start working on ways to compress/reduce the data > stream :-) > > I would also keep in mind that messages between Erlang processes and > inter-node "kernel pings" go over the same socket. This means that pings > start to lag as the message traffic gets heavy. Erlang nodes split when > pings are significantly delayed and then in your in deep doodoo. > > This used to be a case a couple of years ago, I don't think anything > changed since. > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Mon Jun 20 06:41:53 2011 From: jwatte@REDACTED (Jon Watte) Date: Sun, 19 Jun 2011 21:41:53 -0700 Subject: [erlang-questions] erl_scan, erl_parse, macros and records In-Reply-To: References: Message-ID: First, why is it not sufficient to provide your router rules as configuration information for a static module? If you do that, then you will have no problems with in-place code update etc. It will also be easier to debug! Second, if you need to parameterize routing, then you can use parameterized modules. There are some gotchas, such as the "real" arity of functions is actually higher than the "apparent" arity, but they work well for certain cases. If you really need to go code generation, then I suggest you consider the case of changing rules carefully, and plan out how your system will perform in that case. (Also, HiPE vs interpreter, etc) Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Thu, Jun 16, 2011 at 12:48 PM, Dan Kelley wrote: > > Hi, > > I'm working on a router of sorts. I'd like to have a config file which > specifies how to classify messages, what channels are available, and routing > rules based on classifications and channels. > > When my application starts, I'd like to transform the configuration into a > module (let's call it 'router') so I can do someting like > 'router:route(Message)'. Message will be a record that encapsulates a bunch > of things about the message, and the returned value will be the channel to > which the message should be delivered. The Message record is reasonably > complicated - the top level thing contains several layers of different > records. I also have a large number of macros that I use to extract > different fields/subrecords of a top-level Message. > > When I started looking at how to do this, I quickly found erl_scan, > erl_parse, and friends. I can easily create a module on the fly with these, > but everything falls apart when I need to get at the Message record and > macro definitions. Despite the access provided to compiler-ish things by > erl_scan, erl_parse and friends, it looks like the only way to get at the > preprocessor is to use epp on files. This is a problem because the code I > want to dynamically create needs to use the record definitions and macros > from the Message header file. > > Trying to figure a way around this led me to aleppo, which neatly solved > the problem of including the header file, but aleppo doesn't process > records. For a brief and exciting moment, it looked like erl_expand_records > might save the day, but it expects forms, and I can't get anything with > records through erl_parse:parse_form. > > Am I missing an easier way to dynamically create code that references > macros and records? > > Thanks, > > Dan > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Mon Jun 20 07:36:58 2011 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 20 Jun 2011 06:36:58 +0100 Subject: [erlang-questions] Integration with IBM websphere Message-ID: Hi, Has anyone built a bridge between Erlang and IBM WebSphere MQ using its Java MQ client API? I'm just looking for something which will fetch messages out a queue and pass them on to Erlang transparently. For the curious, the docs are here: http://publib.boulder.ibm.com/infocenter/wmqv6/v6r0/topic/com.ibm.mq.csqzaw.doc/uj41012_.htm cheers Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From z_axis@REDACTED Mon Jun 20 08:08:18 2011 From: z_axis@REDACTED (z_axis) Date: Sun, 19 Jun 2011 23:08:18 -0700 (PDT) Subject: [erlang-questions] How to split such a string elegantly ? Message-ID: <1308550098043-3610585.post@n4.nabble.com> The string contains two parts separated by "-", such as "red - blue" . However, sometimes there may be no red or blue part. For example, the following are all right string. "1 2 3 - 33 23", "- 33 23", "1 2 3 -", etc. >[NoRed,NoBlue] = string:tokens("1 2 3 - 33 23","-"). ["1 2 3 "," 33 23"] >[NoRed,NoBlue] = string:tokens("- 33 23","-"). % i hope it returns [""," 33 23"] ** exception error: no match of right hand side value [" 33 23"] >[NoRed,NoBlue] = string:tokens("1 2 3 -","-"). % i hope it returns [1 2 3 ", ""] ** exception error: no match of right hand side value ["1 2 3 "] Sincerely! ----- e^(?.i) + 1 = 0 -- View this message in context: http://erlang.2086793.n4.nabble.com/How-to-split-such-a-string-elegantly-tp3610585p3610585.html Sent from the Erlang Questions mailing list archive at Nabble.com. From qwertymaniac@REDACTED Mon Jun 20 08:23:15 2011 From: qwertymaniac@REDACTED (Harsh J) Date: Mon, 20 Jun 2011 11:53:15 +0530 Subject: [erlang-questions] How to split such a string elegantly ? In-Reply-To: <1308550098043-3610585.post@n4.nabble.com> References: <1308550098043-3610585.post@n4.nabble.com> Message-ID: Hey, You may use re:split(...) for this purpose instead of string:tokens(...). 6> re:split("red - blue", "-", [{return, list}]). ["red "," blue"] 7> re:split("- blue", "-", [{return, list}]). [[]," blue"] 8> re:split("red -", "-", [{return, list}]). ["red ",[]] Have a look at re:split(...) in erldocs.com for more info: http://erldocs.com/R14B02/stdlib/re.html?i=1&search=re:spl#split/3 On Mon, Jun 20, 2011 at 11:38 AM, z_axis wrote: > The string contains two parts separated by "-", such as "red - blue" . > However, sometimes there may be no red or blue part. For example, the > following are all right string. > "1 2 3 - 33 23", "- 33 23", "1 2 3 -", etc. > >>[NoRed,NoBlue] = string:tokens("1 2 3 - 33 23","-"). > ["1 2 3 "," 33 23"] > >>[NoRed,NoBlue] = string:tokens("- 33 23","-"). % i hope it returns [""," > 33 23"] > ** exception error: no match of right hand side value [" 33 23"] > >>[NoRed,NoBlue] = string:tokens("1 2 3 -","-"). % i hope it returns [1 2 3 > ", ""] > ** exception error: no match of right hand side value ["1 2 3 "] > > > Sincerely! > > ----- > e^(?.i) + 1 = 0 > -- > View this message in context: http://erlang.2086793.n4.nabble.com/How-to-split-such-a-string-elegantly-tp3610585p3610585.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Harsh J http://harshj.com From ulf.wiger@REDACTED Mon Jun 20 09:46:42 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 20 Jun 2011 09:46:42 +0200 Subject: [erlang-questions] code_change in the large In-Reply-To: <1308522072.10640.4.camel@sapphire> References: <1050086879.302371308141341437.JavaMail.root@zimbra> <1308522072.10640.4.camel@sapphire> Message-ID: <7D8113BE-19BA-47F3-8190-C143B958BDB1@erlang-solutions.com> When you have tens of thousand processes doing time-critical work, you tend to have to think hard about how to manage code change. There are some alternatives, and your balance between speed and complexity will determine your mileage. 1. Simply stop new jobs from being spawned before upgrade, wait until all running jobs are finished, upgrade, then allow new jobs 2. Bootstrap state changes on-demand, e.g. by checking the data version at some strategic point, and calling conversion routines when needed. This obviously requires some foresight. 3. A particularly dirty trick, if the upgrade involves new message formats, is to use hibernation; when waking from hibernation, a version tag could be checked and the appropriate code change functions called. This has the advantage that messages can be converted before they are "received" by the normal application code. This type of code change is implemented in plain_fsm. 4. Redundancy upgrade. This is tricky to do if you're using mnesia, but there are some ways around that too. Redundancy upgrade works best if you have version control in the communication between nodes. An old library of mine, http://svn.ulf.wiger.net/vcc, has a plugin system for message transform functions, chaining them together to form a transform path between two versions. A particular problem with soft upgrade in this regard is that you tend to be forced to take shortcuts, relaxing the tests ensuring that the upgrade in fact works. This is one reason why I tend to favour redundancy upgrade, at least for very complex systems. It gives you time to carry out thorough consistency tests. BR, Ulf W On 20 Jun 2011, at 00:21, Mojito Sorbet wrote: > As I understand how code_change works, all processes using a module must > be suspended before the switch can happen, the code is purged and > reloaded, then they are all resumed. The application upgrade support > takes care of the mechanics for this, but I wonder about the timing > implications when I have possibly tens of thousands of processes using > the same module. Are there any numbers on this? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From watson.timothy@REDACTED Mon Jun 20 10:11:36 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 20 Jun 2011 09:11:36 +0100 Subject: [erlang-questions] Automated Stripping of otp libraries / modules In-Reply-To: References: Message-ID: On 20 June 2011 00:46, Dale Harvey wrote: > I have seen this mentioned a few times but no answers as yet > > Has anyone written any code that takes an application and strips the otp > libraries to the absolute minimum amount of modules* needed to run that > application? I am assuming it cant be entirely automated by dialyzer > (evalled / runtime generated calls etc) but it should get fairly close > right? > That sounds plausible to me, although I'd suggest that you might want to look at xref as a potential data source for this sort of thing. > If noone has already done this then I guess any pointers for how to > implement it would be cool, I figured I would just need generate a plt with > otp + myapp + deps, find my applications entry point in the plt and traverse > the callgraph marking modules off one at a time, that easy? So again, I suspect that xref is your friend here, probably more so than dialyzer. Also it might be worth looking at the eco-system around reltool, where the OTP tools may (or may not) be trying to do this a bit more carefully in the latest release. From mikpe@REDACTED Mon Jun 20 11:31:10 2011 From: mikpe@REDACTED (Mikael Pettersson) Date: Mon, 20 Jun 2011 11:31:10 +0200 Subject: [erlang-questions] Status of Windows platform support in Erlang/OTP In-Reply-To: References: Message-ID: <19967.4958.254498.696620@pilspetsen.it.uu.se> Amy Lear writes: > I set aside my fun with clock granularity once I knew what the problem > and (hypothetical) solution are, since instrumentation was only a > hypothetical concern for me. > > While taking a break from some other tasks, I started looking into > what it would take to actually implement things like windows-specific > high granularity clocks, and tripped across some interesting items. > > Right now, what I see seems to be the following: > > Erlang itself is not able to be fully compiled on Windows using open > source tools; it requires VC++ as a command line compiler for much of > it. > Erlang doesn't seem to support 64-bit under Windows. > HiPE is not compiled into the official Windows binaries. > > Research indicates that at one point in time, Erlang could be compiled > using MinGW. If I understand correctly, things went somewhat awry > around the time SMP support was added. The 64-bit problem seems to be > a long =/= void sizing issue, and I wasn't able t find anything on > HiPE. > > My questions are as follows: > > Has anything happened since then as far as efforts to be able to fully > compile Erlang using mingw? The mingw/msys environment does provide a > working shell and the ability to run configure scripts, etc, without > bringing the whole cygwin cavalcade to the party. IMO the preferred solution would be to do a pseudo-cross build in a Cygwin environment using a cygwin-to-{i686,x86_64}-w64-mingw32 cross compiler. This gives you all the needed Unix:y build tools, but the resulting binaries don't need Cygwin. That's how we build the Windows versions of our products. > Has any effort gone into the cleanup necessary for being able to > produce a 64-bit Erlang for Windows? The implication from what I found > suggested it's mostly rote work that the core team doesn't have time > to do. No updates that I know of. > What exactly is the status of HiPE on the Windows platform? Same as before, namely that it's not supported but it's probably possible to support it if you accept a 4KB per-native-thread memory consumption penalty. FP exn handling must also be implemented, but I did a prototype of that some time ago. > If work yet remains on these, is there a good starting point on how we > might be able to push forward on these items? Well, as I wrote recently in another thread on erlang-questions, there are no compiler or VM developers left in the HiPE team, we've all moved on to other jobs. I run a small compiler/VM consultancy business on the side, and in that capacity I'd be willing to port HiPE to Windows and the mingw-w64 toolchain, assuming a Cygwin build environment. Fixes for the Windows IL32P64 model should ideally be done by the OTP group, but as they haven't done those already, and they a prerequisite for a 64-bit Erlang VM on Windows, I could do those as well. Contact me offline to work out the details if you're interested. /Mikael From octopusfluff@REDACTED Mon Jun 20 12:00:54 2011 From: octopusfluff@REDACTED (Amy Lear) Date: Mon, 20 Jun 2011 03:00:54 -0700 Subject: [erlang-questions] Status of Windows platform support in Erlang/OTP In-Reply-To: <19967.4958.254498.696620@pilspetsen.it.uu.se> References: <19967.4958.254498.696620@pilspetsen.it.uu.se> Message-ID: On Mon, Jun 20, 2011 at 2:31 AM, Mikael Pettersson wrote: > IMO the preferred solution would be to do a pseudo-cross build in a > Cygwin environment using a cygwin-to-{i686,x86_64}-w64-mingw32 cross > compiler. ?This gives you all the needed Unix:y build tools, but the > resulting binaries don't need Cygwin. ?That's how we build the Windows > versions of our products. My experience with porting some other projects to Windows suggests cygwin is not actually needed if you're making native binaries instead of linking against the cygwin compatibility layer; the mingw/msys toolchain provides many (all?) of the typically necessary unix tools and seems to be significantly faster for some builds. Technically the msys portion is a fork of cygwin, but their branch seems to have less overhead for many tasks somehow. > ?> What exactly is the status of HiPE on the Windows platform? > > Same as before, namely that it's not supported but it's probably possible > to support it if you accept a 4KB per-native-thread memory consumption > penalty. ?FP exn handling must also be implemented, but I did a > prototype of that some time ago. I was afraid 'unsupported' would be the answer. I couldn't find any indication anywhere whether HiPE was actually supposed to be at all available for Windows; if the release notes or other supporting documentation say I managed to miss it. As far as potentially enabling it, for the workloads I would personally consider a penalty of 4KB per native thread would almost certainly be noise. > Well, as I wrote recently in another thread on erlang-questions, > there are no compiler or VM developers left in the HiPE team, we've > all moved on to other jobs. Oh right, I did see that during my research, but managed to forget while reviewing my notes for my post. :/ > I run a small compiler/VM consultancy business on the side, and in > that capacity I'd be willing to port HiPE to Windows and the mingw-w64 > toolchain, assuming a Cygwin build environment. ?Fixes for the Windows > IL32P64 model should ideally be done by the OTP group, but as they > haven't done those already, and they a prerequisite for a 64-bit Erlang > VM on Windows, I could do those as well. ?Contact me offline to work out > the details if you're interested. I appreciate the offer, but I'm afraid for the near term my operating budget is currently $0 plus whatever ephemeral value I might assign to my time. :) Thank you for your responses; mostly I just wanted to know where things currently stood, since I couldn't find recent answers for these questions. From matthias@REDACTED Mon Jun 20 12:01:28 2011 From: matthias@REDACTED (Matthias Lang) Date: Mon, 20 Jun 2011 12:01:28 +0200 Subject: [erlang-questions] Automated Stripping of otp libraries / modules In-Reply-To: References: Message-ID: <20110620100128.GA3232@corelatus.se> On Monday, June 20, Dale Harvey wrote: > Has anyone written any code that takes an application and strips the otp > libraries to the absolute minimum amount of modules* needed to run that > application? We do this in an ugly, pragmatic way. Start the system with all modules available, run the test suite, then code:all_loaded() Now you have a list of exactly what's needed. I then basically paste that into a Makefile. Works well for us because our system is small and test suite is large. I realise that it won't work in general. apply(), M:F, spawn() and hot loading make the problem impossible for "sufficiently nasty" code. Your post has made me wonder how good the information from xref and dialyzer (which came out after I started doing this) would be for a real-world system. I'll try it out on ours and report back. Matt From bob.paddock@REDACTED Mon Jun 20 12:05:38 2011 From: bob.paddock@REDACTED (Bob Paddock) Date: Mon, 20 Jun 2011 06:05:38 -0400 Subject: [erlang-questions] Status of Windows platform support in Erlang/OTP In-Reply-To: References: Message-ID: > Has anything happened since then as far as efforts to be able to fully > compile Erlang using mingw? The mingw/msys environment does provide a > working shell and the ability to run configure scripts, etc, without > bringing the whole cygwin cavalcade to the party. If your not aware of it, take a look at the TDM-GCC project. http://tdm-gcc.tdragon.net/ Sometimes it is a head of MinGW development especially in 64-bit support. -- http://blog.softwaresafety.net/ http://www.designer-iii.com/ http://www.wearablesmartsensors.com/ From hm@REDACTED Mon Jun 20 12:30:35 2011 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Mon, 20 Jun 2011 12:30:35 +0200 Subject: [erlang-questions] Automated Stripping of otp libraries / modules In-Reply-To: References: Message-ID: On Mon, Jun 20, 2011 at 1:46 AM, Dale Harvey wrote: > I have seen this mentioned a few times but no answers as yet > > Has anyone written any code that takes an application and strips the otp > libraries to the absolute minimum amount of modules* needed to run that > application? Use the {mod_cond, derived} setting in reltool. /H?kan From watson.timothy@REDACTED Mon Jun 20 15:11:46 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 20 Jun 2011 14:11:46 +0100 Subject: [erlang-questions] BEAM backward compatibility In-Reply-To: References: Message-ID: On 19 June 2011 05:23, Daniel Dormont wrote: > Since I work mostly in Java, when I started learning Erlang I recognized > .beam files as being? at least somewhat like .class files and assumed they > would be backward compatible - that is, beams compiled in R12 would run in > R13 and R14 and there shouldn't be any problems unless I were calling > experimental APIs or deprecated APIs that were later removed. But a > discussion on the ejabberd mailing list suggested that this might not be the > case. I didn't see anything offhand in the official documentation. What's > the word on this? Java byte code is forwards compatible, rather than backwards, which is what you're describing. From dan@REDACTED Mon Jun 20 15:36:44 2011 From: dan@REDACTED (Daniel Dormont) Date: Mon, 20 Jun 2011 09:36:44 -0400 Subject: [erlang-questions] BEAM backward compatibility In-Reply-To: References: Message-ID: <9CE6E70A-9B12-4664-B78F-F303509A4162@greywallsoftware.com> On Jun 20, 2011, at 9:11 AM, Tim Watson wrote: > On 19 June 2011 05:23, Daniel Dormont wrote: >> Since I work mostly in Java, when I started learning Erlang I recognized >> .beam files as being at least somewhat like .class files and assumed they >> would be backward compatible - that is, beams compiled in R12 would run in >> R13 and R14 and there shouldn't be any problems unless I were calling >> experimental APIs or deprecated APIs that were later removed. But a >> discussion on the ejabberd mailing list suggested that this might not be the >> case. I didn't see anything offhand in the official documentation. What's >> the word on this? > > Java byte code is forwards compatible, rather than backwards, which is > what you're describing. Yeah, that's one of those things where the terminology gets confusing, but yes that's what I meant. thanks, Dan From pablo.platt@REDACTED Mon Jun 20 16:22:40 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 20 Jun 2011 07:22:40 -0700 (PDT) Subject: [erlang-questions] error when trying to use erl +B c Message-ID: <156240.69663.qm@web112611.mail.gq1.yahoo.com> Hi I'm trying to start a node with the "+B c" option so Ctrl-C will stop the shell but won't stop the node. The node fails to run and I'm getting an error: Missing argument(s) for '-B' "+B" and "+B i" works as expected. I'm using R14B03 built from source on ubuntu 11.04 The node is created using rebar release handleing. I'm setting the "+B c" option in the vm.args file which is included with -args_file: CMD="$BINDIR/erlexec -boot $RUNNER_BASE_DIR/releases/$APP_VSN/$BOOTFILE -embedded -config $RUNNER_ETC_DIR/app.config -args_file $RUNNER_ETC_DIR/vm.args -- ${1+"$@"}" Am I using "+B c" in the wrong way? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From mevans@REDACTED Mon Jun 20 16:55:55 2011 From: mevans@REDACTED (Evans, Matthew) Date: Mon, 20 Jun 2011 10:55:55 -0400 Subject: [erlang-questions] Freedom Box and Erlang In-Reply-To: <4DFCF919.5070106@gmail.com> References: <4DFCF919.5070106@gmail.com> Message-ID: Seems like you have an interesting project with cloudi. Matt -----Original Message----- From: Michael Truog [mailto:mjtruog@REDACTED] Sent: Saturday, June 18, 2011 3:15 PM To: erlang-questions@REDACTED Cc: Evans, Matthew Subject: Re: [erlang-questions] Freedom Box and Erlang To have an Erlang cloud that doesn't depend on an organization, there is CloudI (http://cloudi.org) . So it could be used for running applications on a distributed mesh. There is ZeroMQ integration that could provide communication between the FreedomBoxes. CloudI currently lacks a pretty UI for managing the configuration and will be going through more testing, so those are current barriers. On 06/18/2011 10:54 AM, Evans, Matthew wrote: > Hi, > > I've recently been reading about Freedom box (see: http://wiki.debian.org/FreedomBox and http://freedomboxfoundation.org/). > > In a nutshell, directly from their website: > > "Smart devices whose engineered purpose is to work together to facilitate free communication among people, safely and securely, beyond the ambition of the strongest power to penetrate, they can make freedom of thought and information a permanent, ineradicable feature of the net that holds our souls." > > "We live in a world where our use of the network is mediated by organizations that often do not have our best interests at heart. By building software that does not rely on a central service, we can regain control and privacy. By keeping our data in our homes, we gain useful legal protections over it. By giving back power to the users over their networks and machines, we are returning the Internet to its intended peer-to-peer architecture. " > > I'm not a member of the organization, but it seems to me that the distributed nature of Erlang could offer awesome possibilities to work around certain technical difficulties. I could, for example, envisage a mesh of Erlang nodes connected together providing various distributed services where no individual is in overall control. Or you could provide services (Erlang processes/application) owned and managed by someone from a country with poor human rights that is in fact running on an Erlang node on a country with better human rights (a chat service for example). I'm not talking about "cloud" services here (that some organization ultimately owns), but running applications on a mesh of privately owned "Freedom boxes" that host Erlang VMs. The beauty of Erlang is that it's innate ability to distribute code, processes and applications in such a simple and seamless manner could provide very powerful tools for the organization. > > Any comments and thoughts on this? > > Matt > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jonas.falkevik@REDACTED Mon Jun 20 17:18:15 2011 From: jonas.falkevik@REDACTED (Jonas Falkevik) Date: Mon, 20 Jun 2011 17:18:15 +0200 Subject: [erlang-questions] error when trying to use erl +B c In-Reply-To: <156240.69663.qm@web112611.mail.gq1.yahoo.com> References: <156240.69663.qm@web112611.mail.gq1.yahoo.com> Message-ID: Have you tried +Bc ? /Jonas On Jun 20, 2011, at 16:22 , Pablo Platt wrote: > Hi > > I'm trying to start a node with the "+B c" option so Ctrl-C will stop the shell but won't stop the node. > The node fails to run and I'm getting an error: > Missing argument(s) for '-B' > > "+B" and "+B i" works as expected. > > I'm using R14B03 built from source on ubuntu 11.04 > The node is created using rebar release handleing. > I'm setting the "+B c" option in the vm.args file which is included with -args_file: > CMD="$BINDIR/erlexec -boot $RUNNER_BASE_DIR/releases/$APP_VSN/$BOOTFILE -embedded -config $RUNNER_ETC_DIR/app.config -args_file $RUNNER_ETC_DIR/vm.args -- ${1+"$@"}" > > Am I using "+B c" in the wrong way? > > Thanks > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From hynek@REDACTED Mon Jun 20 17:21:02 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Mon, 20 Jun 2011 17:21:02 +0200 Subject: [erlang-questions] Best Linux Distro for Erlang Devel In-Reply-To: References: Message-ID: On Mon, Jun 20, 2011 at 3:53 AM, Wilson MacGyver wrote: > I use Fedora for both haskell and erlang. Fedora is fairly good about > keeping up to > date with both haskell platform and erlang providing you keep Fedora up to date. > > For example, the current Fedora 15 has Erlang R14B02. and haskell platform > 2011.2.0.0. Current stable Erlang version is 14B03 and even this version is provided in Debian. > > On Sun, Jun 19, 2011 at 6:11 PM, Tom Murphy wrote: >> Hi *, >> ? ? (I posted a very similar question to a Haskell list yesterday) >> ?If my choice of Lunix distro depended 100% on its solidness as an >> Erlang devel platform (completeness of base, packages, stability, >> support, robustness), what would you all recommend? >> >> Thanks for your time! >> Tom >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Omnem crede diem tibi diluxisse supremum. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From francis.joanis@REDACTED Mon Jun 20 17:30:36 2011 From: francis.joanis@REDACTED (Francis Joanis) Date: Mon, 20 Jun 2011 11:30:36 -0400 Subject: [erlang-questions] Version of C compiler for Erlang on Windows Message-ID: Hi, Is Visual Studio 2005 still the preferred version of the Microsoft compiler for Erlang on Windows and is it still used for the pre-compiled binaries? I think some work had been done to support VS 2008 and I don't know about VS 2010. Thanks, Francis From pablo.platt@REDACTED Mon Jun 20 17:40:04 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 20 Jun 2011 08:40:04 -0700 (PDT) Subject: [erlang-questions] error when trying to use erl +B c In-Reply-To: References: <156240.69663.qm@web112611.mail.gq1.yahoo.com> Message-ID: <930104.74360.qm@web112607.mail.gq1.yahoo.com> +Bc doesn't give me error but when I'm attached to the node, Ctrl-C Ctrl-C stops the node ________________________________ From: Jonas Falkevik To: Pablo Platt Cc: "erlang-questions@REDACTED" Sent: Monday, June 20, 2011 6:18 PM Subject: Re: [erlang-questions] error when trying to use erl +B c Have you tried +Bc ? /Jonas On Jun 20, 2011, at 16:22 , Pablo Platt wrote: > Hi > > I'm trying to start a node with the "+B c" option so Ctrl-C will stop the shell but won't stop the node. > The node fails to run and I'm getting an error: > Missing argument(s) for '-B' > > "+B" and "+B i" works as expected. > > I'm using R14B03 built from source on ubuntu 11.04 > The node is created using rebar release handleing. > I'm setting the "+B c" option in the vm.args file which is included with -args_file: > CMD="$BINDIR/erlexec -boot $RUNNER_BASE_DIR/releases/$APP_VSN/$BOOTFILE -embedded -config $RUNNER_ETC_DIR/app.config -args_file $RUNNER_ETC_DIR/vm.args -- ${1+"$@"}" > > Am I using "+B c" in the wrong way? > > Thanks > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Mon Jun 20 18:29:03 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Mon, 20 Jun 2011 17:29:03 +0100 Subject: [erlang-questions] Thanks! Re: Erlang for Speech Recognition In-Reply-To: <4DFDBBFB.6090909@llaisdy.com> References: <4DFDBBFB.6090909@llaisdy.com> Message-ID: <4DFF754F.1060703@llaisdy.com> Dear All Thank you for your comments. I'm going to tackle the audio preprocessor first, converting pcm audio into MFCCs. I'll use Sphinx's make_feats as a basis. This will mean, on the C side, writing a function to call what I need from Sphinx, and on the Erlang side, writing the necessary Port/LinkedIn Driver/NIF. With thanks and best wishes Ivan On 19/06/2011 10:06, Ivan Uemlianin wrote: > Dear All > > I am developing a suite of software tools for use in language technology > (primarily speech recognition). A lot of the procedures in speech > technology have a very "map-reduce" feel to them and I think Erlang > would be a good fit. > > Below I list and briefly describe the tools I'm developing. Does anyone > know if there are similar current Erlang projects? (I have looked). > > As the whole lot is needed for a speech rec system, I don't quite know > how I should proceed: should I write the easiest component first > (probably the language model builder/server), the hardest (probably the > audio preprocessor), the most useful outside of speech recognition > (probably the hidden Markov model builder/server), ...? > > > ** Audio Preprocessor > > Automatic Speech Recognition (ASR) is essentially mapping a sequence of > integers (i.e., acoustic signals) onto a sequence of linguistic symbols > (i.e., phonemes (units of linguistic sound) or words). The raw audio > data (e.g. from a wav file or a microphone) is not terribly useful for > this and the first step is to convert this data into a more useful > abstract representation. Each 100ms of sound is transformed into a > feature vector of 39 features, known as Mel-Frequency Cepstral > Co-efficients (MFCCs). > > The first step in recognition, or in training a recogniser, is always to > convert the audio like this. > > A while ago I wrote a little script to read and write wav files [1]. I > also have a 'dummy' make_feats.erl which sets out the imagined tasks for > converting audio data into MFCCs. However, there are two possible ways > ahead: > > 1. Write the whole lot from scratch in Erlang. > > 2. There is a mature, respected open-source speech recognition toolkit, > written in C called Sphinx [2]. Sphinx has a make_feats function which > could be called as a port or a NIF. The Sphinx make_feats doesn't work > quite how I'd like, so some changes would have to be made (Sphinx is > released under a BSD-style license). > > (2) would avoid working out how to implement the maths, but (1) would > avoid fiddling around inside someone else's C. Any advice? > > > ** Hidden Markov Model Builder/Server > > A hidden markov Model (HMM) is essentially a finite state machine with > probabilities given to the edges connecting states. We train up a HMM > for each phoneme of a language (HMMs are trained using MFCC sequences). > > The foundational recognition task is recognising a single phoneme (this > is then conditioned by probabilities of different phoneme sequences). > So, we take an MFCC sequence, match it up against each HMM in turn and > ask, "What is the probability that this HMM could have produced this > MFCC sequence?" > > Although used mainly in ASR, HMMs can be used in speech synthesis (aka > text-to-speech), and they are used outside speech tech of course (e.g. > in finance [3]). > > I have written a toy HMM trainer and recogniser, for simple symbol > sequences. I think the sensible next step would be to tone this up and > test it again simple real world data (I could compare performance and > results with the R HMM package [4]). Once that seems stable, enhance the > code to work with sequences of real number vectors, build a phoneme > recogniser and compare with Sphinx. > > The set of HMMs is referred to as the Acoustic Model (AM) of the > language. Other mathematical models can be used but, since at least the > mid-80s, HMMs dominate. There is some interesting recent work using > dynamic Bayesian networks, and using conditional random fields. > > > ** Language Model Builder/Server > > The Language Model (LM) furnishes probabilities of various linguistic > structures, and sits on top off, or collaborates with, the AM. > > Sequences of phonemes are dealt with by a simple pronunciation > dictionary, which is just a list mapping sequences of phonemes to words. > > Just as HMMs dominate AMs, the dominant model for syntactic structures > is the ngram grammar, which assigns probabilities to sequences of words > (the most common 'n' is 3, often called a trigram). > > As well as their use in ASR, LMs are an essential component in > statistical machine translation. > > I have written a toy LM builder, which assigns probabilities to trigrams > based on a given corpus. I think the sensible next step would be to tone > this up to work with large corpora, and compare performance and results > against a standard open-source LM builder [5]. > > > ** references > > [1] > http://llaisdy.wordpress.com/2010/06/01/wave-erl-an-erlang-script-to-read-and-write-wav-files/ > > > [2] http://cmusphinx.sourceforge.net/ > > [3] > http://www.optirisk-systems.com/events/application-of-hidden-markov-models-and- > > filters-to-financial-time-series-data.asp > > [4] http://r-forge.r-project.org/projects/rhmm/ > > [5] i.e. irstlm (http://sourceforge.net/projects/irstlm/). There are > several semi-open-source LM builders available for research use only, > but afaik irstlm is the only one that is bona fide open-source. > > -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From overminddl1@REDACTED Mon Jun 20 18:40:20 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Mon, 20 Jun 2011 10:40:20 -0600 Subject: [erlang-questions] Status of Windows platform support in Erlang/OTP In-Reply-To: References: Message-ID: clang might also work. On Jun 20, 2011 4:05 AM, "Bob Paddock" wrote: >> Has anything happened since then as far as efforts to be able to fully >> compile Erlang using mingw? The mingw/msys environment does provide a >> working shell and the ability to run configure scripts, etc, without >> bringing the whole cygwin cavalcade to the party. > > If your not aware of it, take a look at the TDM-GCC project. > > http://tdm-gcc.tdragon.net/ > > Sometimes it is a head of MinGW development especially in 64-bit support. > > > > -- > http://blog.softwaresafety.net/ > http://www.designer-iii.com/ > http://www.wearablesmartsensors.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon Jun 20 20:34:53 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 20 Jun 2011 19:34:53 +0100 Subject: [erlang-questions] Freedom Box and Erlang In-Reply-To: References: <4DFCF919.5070106@gmail.com> Message-ID: There is also http://www.cloudfoundry.org, which is sponsored by vmware and has a commercial offering too (at cloudfoundry.com) but is open source - https://github.com/cloudfoundry. On 20 June 2011 15:55, Evans, Matthew wrote: > Seems like you have an interesting project with cloudi. > > Matt > > -----Original Message----- > From: Michael Truog [mailto:mjtruog@REDACTED] > Sent: Saturday, June 18, 2011 3:15 PM > To: erlang-questions@REDACTED > Cc: Evans, Matthew > Subject: Re: [erlang-questions] Freedom Box and Erlang > > To have an Erlang cloud that doesn't depend on an organization, there is CloudI (http://cloudi.org) . ?So it could be used for running applications on a distributed mesh. ?There is ZeroMQ integration that could provide communication between the FreedomBoxes. ?CloudI currently lacks a pretty UI for managing the configuration and will be going through more testing, so those are current barriers. > > On 06/18/2011 10:54 AM, Evans, Matthew wrote: >> Hi, >> >> I've recently been reading about Freedom box (see: http://wiki.debian.org/FreedomBox and http://freedomboxfoundation.org/). >> >> In a nutshell, directly from their website: >> >> "Smart devices whose engineered purpose is to work together to facilitate free communication among people, safely and securely, beyond the ambition of the strongest power to penetrate, they can make freedom of thought and information a permanent, ineradicable feature of the net that holds our souls." >> >> "We live in a world where our use of the network is mediated by organizations that often do not have our best interests at heart. By building software that does not rely on a central service, we can regain control and privacy. By keeping our data in our homes, we gain useful legal protections over it. By giving back power to the users over their networks and machines, we are returning the Internet to its intended peer-to-peer architecture. " >> >> I'm not a member of the organization, but it seems to me that the distributed nature of Erlang could offer awesome possibilities to work around certain technical difficulties. I could, for example, envisage a mesh of Erlang nodes connected together providing various distributed services where no individual is in overall control. Or you could provide services (Erlang processes/application) owned and managed by someone from a country with poor human rights that is in fact running on an Erlang node on a country with better human rights (a chat service for example). I'm not talking about "cloud" services here (that some organization ultimately owns), but running applications on a mesh of privately owned "Freedom boxes" that host Erlang VMs. The beauty of Erlang is that it's innate ability to distribute code, processes and applications in such a simple and seamless manner could provide very powerful tools for the organization. >> >> Any comments and thoughts on this? >> >> Matt >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mjtruog@REDACTED Mon Jun 20 21:04:59 2011 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 20 Jun 2011 12:04:59 -0700 Subject: [erlang-questions] Freedom Box and Erlang In-Reply-To: References: <4DFCF919.5070106@gmail.com> Message-ID: <4DFF99DB.60401@gmail.com> True. CloudFoundry exists and is open source. However, it is Ruby based so it is unable to scale like an Erlang system (like CloudI). CloudFoundry is control by VMware, which is an organization, and they are using the Apache License 2.0. CloudI is independent and uses a BSD license, so it can be reused anywhere. On 06/20/2011 11:34 AM, Tim Watson wrote: > There is also http://www.cloudfoundry.org, which is sponsored by > vmware and has a commercial offering too (at cloudfoundry.com) but is > open source - https://github.com/cloudfoundry. > > On 20 June 2011 15:55, Evans, Matthew wrote: >> Seems like you have an interesting project with cloudi. >> >> Matt >> >> -----Original Message----- >> From: Michael Truog [mailto:mjtruog@REDACTED] >> Sent: Saturday, June 18, 2011 3:15 PM >> To: erlang-questions@REDACTED >> Cc: Evans, Matthew >> Subject: Re: [erlang-questions] Freedom Box and Erlang >> >> To have an Erlang cloud that doesn't depend on an organization, there is CloudI (http://cloudi.org) . So it could be used for running applications on a distributed mesh. There is ZeroMQ integration that could provide communication between the FreedomBoxes. CloudI currently lacks a pretty UI for managing the configuration and will be going through more testing, so those are current barriers. >> >> On 06/18/2011 10:54 AM, Evans, Matthew wrote: >>> Hi, >>> >>> I've recently been reading about Freedom box (see: http://wiki.debian.org/FreedomBox and http://freedomboxfoundation.org/). >>> >>> In a nutshell, directly from their website: >>> >>> "Smart devices whose engineered purpose is to work together to facilitate free communication among people, safely and securely, beyond the ambition of the strongest power to penetrate, they can make freedom of thought and information a permanent, ineradicable feature of the net that holds our souls." >>> >>> "We live in a world where our use of the network is mediated by organizations that often do not have our best interests at heart. By building software that does not rely on a central service, we can regain control and privacy. By keeping our data in our homes, we gain useful legal protections over it. By giving back power to the users over their networks and machines, we are returning the Internet to its intended peer-to-peer architecture. " >>> >>> I'm not a member of the organization, but it seems to me that the distributed nature of Erlang could offer awesome possibilities to work around certain technical difficulties. I could, for example, envisage a mesh of Erlang nodes connected together providing various distributed services where no individual is in overall control. Or you could provide services (Erlang processes/application) owned and managed by someone from a country with poor human rights that is in fact running on an Erlang node on a country with better human rights (a chat service for example). I'm not talking about "cloud" services here (that some organization ultimately owns), but running applications on a mesh of privately owned "Freedom boxes" that host Erlang VMs. The beauty of Erlang is that it's innate ability to distribute code, processes and applications in such a simple and seamless manner could provide very powerful tools for the organization. >>> >>> Any comments and thoughts on this? >>> >>> Matt >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From andrew@REDACTED Mon Jun 20 23:19:07 2011 From: andrew@REDACTED (Andrew Thompson) Date: Mon, 20 Jun 2011 17:19:07 -0400 Subject: [erlang-questions] Version of C compiler for Erlang on Windows In-Reply-To: References: Message-ID: <20110620211907.GS2051@hijacked.us> On Mon, Jun 20, 2011 at 11:30:36AM -0400, Francis Joanis wrote: > Hi, > > Is Visual Studio 2005 still the preferred version of the Microsoft > compiler for Erlang on Windows and is it still used for the > pre-compiled binaries? > > I think some work had been done to support VS 2008 and I don't know > about VS 2010. I submitted some patches to make it build with VS2008, and they were accepted. No idea about 2010. Andrew From watson.timothy@REDACTED Mon Jun 20 23:42:19 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 20 Jun 2011 22:42:19 +0100 Subject: [erlang-questions] Freedom Box and Erlang In-Reply-To: <4DFF99DB.60401@gmail.com> References: <4DFCF919.5070106@gmail.com> <4DFF99DB.60401@gmail.com> Message-ID: On 20 June 2011 20:04, Michael Truog wrote: > True. ?CloudFoundry exists and is open source. ?However, it is Ruby based so it is unable to scale like an Erlang system (like CloudI). ?CloudFoundry is control by VMware, which is an organization, and they are using the Apache License 2.0. ?CloudI is independent and uses a BSD license, so it can be reused anywhere. > First up, I'm definitely not knocking CloudI here - I think it looks awesome. CloudFoundry is owned and developed by an organisation, but so are plenty of other things (riak, rabbitmq, etc) so that doesn't automatically mean lock-in. I mean Erlang/OTP is controlled by Ericsson right, and by the same measure CloudI is controlled by Michael Truog. The Apache License 2.0 is more restrictive than the BSD type license you've chosen for CloudI - and good on you for making it as open as possible - but I can still use and adapt stuff based on the Apache license for commercial or similarly licensed open source projects. Secondly, I agree that Erlang/OTP is a gift from the Old Gods of the North when it comes to fault tolerance and so on, but just because app-1 is written in Erlang and app-2 is written in Ruby, doesn't automatically mean that one will scale better than another. Both these cloud frameworks are based (at least in part) on services running as native OS processes - admittedly you support native Erlang services/jobs - and the other comparative technologies are really around routing and distribution. Here both frameworks are relying on the same kind of underlying TCP stack that takes advantage of non-blocking socket I/O - distributed Erlang using the inet infrastructure running on a beam emulator compiled with --enable-poll or whatever, versus CloudFoundry which runs on event-machine, a framework a la libevent. Distributed Erlang is known to have limitations of its own, that wouldn't necessarily apply to a very large grid built on some other distribution protocol - riak for example uses a custom protocol, as does Scalaris I think and probably quite a few others. The fault tolerance that is built in for you (because you're running on top of OTP) though, is admittedly a big selling point. Don't get me wrong - I'm not asserting any amazing properties about ruby, nor am I trying to criticise your work. It's just that you're clearly very smart (because you wrote a complex beast like CloudI) so your peers (and even underlings like me) need to hold you to account for intellectual honesty. Saying "it is Ruby based so it is unable to scale..." is FUD unless you qualify it in more detail. Most people on this list would be inclined to think you're on to something, but this is not a black and white area. I'm pretty sure that services such as Github and Heroku are very scalable, and they use both Ruby and Erlang (and probably only Odin knows what else) for various tasks. Cheers, Tim > On 06/20/2011 11:34 AM, Tim Watson wrote: >> There is also http://www.cloudfoundry.org, which is sponsored by >> vmware and has a commercial offering too (at cloudfoundry.com) but is >> open source - https://github.com/cloudfoundry. >> >> On 20 June 2011 15:55, Evans, Matthew wrote: >>> Seems like you have an interesting project with cloudi. >>> >>> Matt >>> >>> -----Original Message----- >>> From: Michael Truog [mailto:mjtruog@REDACTED] >>> Sent: Saturday, June 18, 2011 3:15 PM >>> To: erlang-questions@REDACTED >>> Cc: Evans, Matthew >>> Subject: Re: [erlang-questions] Freedom Box and Erlang >>> >>> To have an Erlang cloud that doesn't depend on an organization, there is CloudI (http://cloudi.org) . ?So it could be used for running applications on a distributed mesh. ?There is ZeroMQ integration that could provide communication between the FreedomBoxes. ?CloudI currently lacks a pretty UI for managing the configuration and will be going through more testing, so those are current barriers. >>> >>> On 06/18/2011 10:54 AM, Evans, Matthew wrote: >>>> Hi, >>>> >>>> I've recently been reading about Freedom box (see: http://wiki.debian.org/FreedomBox and http://freedomboxfoundation.org/). >>>> >>>> In a nutshell, directly from their website: >>>> >>>> "Smart devices whose engineered purpose is to work together to facilitate free communication among people, safely and securely, beyond the ambition of the strongest power to penetrate, they can make freedom of thought and information a permanent, ineradicable feature of the net that holds our souls." >>>> >>>> "We live in a world where our use of the network is mediated by organizations that often do not have our best interests at heart. By building software that does not rely on a central service, we can regain control and privacy. By keeping our data in our homes, we gain useful legal protections over it. By giving back power to the users over their networks and machines, we are returning the Internet to its intended peer-to-peer architecture. " >>>> >>>> I'm not a member of the organization, but it seems to me that the distributed nature of Erlang could offer awesome possibilities to work around certain technical difficulties. I could, for example, envisage a mesh of Erlang nodes connected together providing various distributed services where no individual is in overall control. Or you could provide services (Erlang processes/application) owned and managed by someone from a country with poor human rights that is in fact running on an Erlang node on a country with better human rights (a chat service for example). I'm not talking about "cloud" services here (that some organization ultimately owns), but running applications on a mesh of privately owned "Freedom boxes" that host Erlang VMs. The beauty of Erlang is that it's innate ability to distribute code, processes and applications in such a simple and seamless manner could provide very powerful tools for the organization. >>>> >>>> Any comments and thoughts on this? >>>> >>>> Matt >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> > > > From dave@REDACTED Tue Jun 21 01:13:05 2011 From: dave@REDACTED (Dave Cottlehuber) Date: Tue, 21 Jun 2011 11:13:05 +1200 Subject: [erlang-questions] Version of C compiler for Erlang on Windows In-Reply-To: <20110620211907.GS2051@hijacked.us> References: <20110620211907.GS2051@hijacked.us> Message-ID: On 21 June 2011 09:19, Andrew Thompson wrote: > On Mon, Jun 20, 2011 at 11:30:36AM -0400, Francis Joanis wrote: >> Hi, >> >> Is Visual Studio 2005 still the preferred version of the Microsoft >> compiler for Erlang on Windows and is it still used for the >> pre-compiled binaries? >> >> I think some work had been done to support VS 2008 and I don't know >> about VS 2010. > > I submitted some patches to make it build with VS2008, and they were > accepted. No idea about 2010. > > Andrew 90% of the hassle is getting the compiler/linker/manifest paths sorted out; these vary for every combo of VS IDE/SDK out there. VS 2008 Express compiles fine since R13B so long as you install the Windows SDK v7.0 as well. At the time wxWidgets didn't compile using VS2010 express so I stayed with VS2008. I use the following script / notes to set up a CouchDB build chain for VS2008 https://github.com/dch/glazier the README is still out of date vs scripts while I clean up some changes needed for CouchDB but the erlang stuff should be pretty stable. The patches & notes for doing this are now all included in the Erlang release so if it doesn't line up please let me know & I'll work through fixing them. Finally, I now compile OpenSSL from source https://github.com/dch/glazier/blob/master/bin/build_openssl.cmd & wxWidgets via https://github.com/dch/glazier/blob/master/bin/build_wx.cmd after applying patch equivalent to the manual steps in the README. IIRC wxWidgets wasn't able to be built with VS2010 last time I checked; but there appears to be a workaround now since http://trac.wxwidgets.org/changeset/66836/ ref http://wiki.wxwidgets.org/Microsoft_Visual_C%2B%2B_Guide#Visual_Studio_C.2B.2B_2010 please post back how you get on with this. A+ Dave From z_axis@REDACTED Tue Jun 21 01:54:03 2011 From: z_axis@REDACTED (z_axis) Date: Mon, 20 Jun 2011 16:54:03 -0700 (PDT) Subject: [erlang-questions] How to split such a string elegantly ? In-Reply-To: References: <1308550098043-3610585.post@n4.nabble.com> Message-ID: <1308614043461-3612796.post@n4.nabble.com> It works great! thanks ----- e^(?.i) + 1 = 0 -- View this message in context: http://erlang.2086793.n4.nabble.com/How-to-split-such-a-string-elegantly-tp3610585p3612796.html Sent from the Erlang Questions mailing list archive at Nabble.com. From corticalcomputer@REDACTED Tue Jun 21 02:09:10 2011 From: corticalcomputer@REDACTED (G.S.) Date: Mon, 20 Jun 2011 17:09:10 -0700 Subject: [erlang-questions] Regarding PteraCUDA. Message-ID: Hello, Has anyone experimented with PteraCUDA and could share their experience with it? Regards, -Gene -------------- next part -------------- An HTML attachment was scrubbed... URL: From dale@REDACTED Tue Jun 21 03:16:52 2011 From: dale@REDACTED (Dale Harvey) Date: Tue, 21 Jun 2011 02:16:52 +0100 Subject: [erlang-questions] Automated Stripping of otp libraries / modules In-Reply-To: References: Message-ID: > We do this in an ugly, pragmatic way. > Start the system with all modules available, run the test suite, then > code:all_loaded() Hah I love that way, its pretty hacky but makes sense, thanks. 2011/6/20 H?kan Mattsson > On Mon, Jun 20, 2011 at 1:46 AM, Dale Harvey wrote: > > I have seen this mentioned a few times but no answers as yet > > > > Has anyone written any code that takes an application and strips the otp > > libraries to the absolute minimum amount of modules* needed to run that > > application? > > Use the {mod_cond, derived} setting in reltool. > > /H?kan > Ah this is perfect thanks, unfortunately the codebase isnt setup to be packaged by reltool, but thats the codebases problem since this is obviously the most sensible option. Cheers Dale -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Tue Jun 21 03:21:54 2011 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 21 Jun 2011 01:21:54 +0000 (GMT) Subject: [erlang-questions] erl_scan, erl_parse, macros and records In-Reply-To: <196488475.317701308619248819.JavaMail.root@zimbra> Message-ID: <1596968578.317721308619314207.JavaMail.root@zimbra> Sorry, I have not really been able to understand exactly what you want to do and what you are having problems with. A short review of what is done where: erl_scan - generates tokens from characters epp (the preprocessor) - handles macros, both definition and expansion, and include files. That's all it does not, no more. It works on tokens and generates tokens. erl_parse - parses everything except macros which have been done in epp. So it parses records, both the definition and use of. compile (the compiler) - takes the abstract syntax from erl_parse and generates code. All record processing is done here (in a very early pass). Erl_scan, erl_parse and the compiler are easy to run directly but unfortunately epp, the preprocessor, only works on files. There have been numerous suggestions to modify it to work directly on tokens but this has never been done. So if you want to generate code from within a program without writing to temporary files you can basically do everything, including records, except for macros. Include files can easily handled explicitly by parsing them and including the abstract forms directly in with your own forms. If you are generating your own code it should not be too difficult to get around the lack of macros. An alternative to using the standard libraries to generate code is to use the syntax_tools application with the erl_syntax module to build an erlang abstract syntax . These tools can handle macros. It does not internally use the same abstract syntax as the standard tools but it has functions for generating the standard abstract syntax from its representation, which can then be used by the compiler. An example of generating a record definition from a string would be: {ok,Tokens,Line} = erl_scan:string("-record(foo, {a,b=42,c}). "), {ok,FooDef} = erl_parse:parse_form(Tokens) where FooDef is the abstract syntax form of the record definition. The abstract syntax of a module is just a list of the abstract syntax of al the forms (attributes and functions) in the module. This is why include files are basically simple to handle. At what level do you want to start generating input? From strings, or what? Robert ----- "Dan Kelley" wrote: > > Hi, > I'm working on a router of sorts. I'd like to have a config file which specifies how to classify messages, what channels are available, and routing rules based on classifications and channels. > When my application starts, I'd like to transform the configuration into a module (let's call it 'router') so I can do someting like 'router:route(Message)'. Message will be a record that encapsulates a bunch of things about the message, and the returned value will be the channel to which the message should be delivered. The Message record is reasonably complicated - the top level thing contains several layers of different records. I also have a large number of macros that I use to extract different fields/subrecords of a top-level Message. > When I started looking at how to do this, I quickly found erl_scan, erl_parse, and friends. I can easily create a module on the fly with these, but everything falls apart when I need to get at the Message record and macro definitions. Despite the access provided to compiler-ish things by erl_scan, erl_parse and friends, it looks like the only way to get at the preprocessor is to use epp on files. This is a problem because the code I want to dynamically create needs to use the record definitions and macros from the Message header file. > Trying to figure a way around this led me to aleppo, which neatly solved the problem of including the header file, but aleppo doesn't process records. For a brief and exciting moment, it looked like erl_expand_records might save the day, but it expects forms, and I can't get anything with records through erl_parse:parse_form. > Am I missing an easier way to dynamically create code that references macros and records? > Thanks, > Dan > > > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin@REDACTED Tue Jun 21 03:55:30 2011 From: kevin@REDACTED (Kevin Smith) Date: Mon, 20 Jun 2011 18:55:30 -0700 Subject: [erlang-questions] Regarding PteraCUDA. In-Reply-To: References: Message-ID: <14781FCA-64CA-4569-BB33-5F11B6B39FF2@hypotheticalabs.com> Hi -- I'm the author of pteracuda and I'd love to hear what, if anything, people are doing with pteracuda. I've hit a temporary stall in development as my primary dev machine was rendered slightly inoperable when I shipped it back from Erlang Factory. I expect to have the broken hardware replaced soonish and then I can resume development. --Kevin On Jun 20, 2011, at 5:09 PM, G.S. wrote: > Hello, > > Has anyone experimented with PteraCUDA and could share their experience with it? > > Regards, > -Gene > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mjtruog@REDACTED Tue Jun 21 04:36:06 2011 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 20 Jun 2011 19:36:06 -0700 Subject: [erlang-questions] Freedom Box and Erlang In-Reply-To: References: <4DFCF919.5070106@gmail.com> <4DFF99DB.60401@gmail.com> Message-ID: <4E000396.30507@gmail.com> On 06/20/2011 02:42 PM, Tim Watson wrote: > On 20 June 2011 20:04, Michael Truog wrote: >> True. CloudFoundry exists and is open source. However, it is Ruby based so it is unable to scale like an Erlang system (like CloudI). CloudFoundry is control by VMware, which is an organization, and they are using the Apache License 2.0. CloudI is independent and uses a BSD license, so it can be reused anywhere. > First up, I'm definitely not knocking CloudI here - I think it looks > awesome. CloudFoundry is owned and developed by an organisation, but > so are plenty of other things (riak, rabbitmq, etc) so that doesn't > automatically mean lock-in. I mean Erlang/OTP is controlled by > Ericsson right, and by the same measure CloudI is controlled by > Michael Truog. The Apache License 2.0 is more restrictive than the BSD > type license you've chosen for CloudI - and good on you for making it > as open as possible - but I can still use and adapt stuff based on the > Apache license for commercial or similarly licensed open source > projects. I agree. Often times organizations make strategic decisions to include and exclude support for various types of software, which either benefits their partners/divisions or harms their competition. Having an open source project that is independent means that it is able to pursue a path to the best combination of technology based on what the community thinks (though limited by time/money). That way there should be no major influence based on "competitive advantage" decisions that are commonly made by any commercial organization. I am not a lawyer and I am not giving legal advise. However, I think it is more likely that BSD licensed software is reused and then sold in a commercial product, when compared with Apache licensed software. The main difference between the licenses is over patent ownership (BSD is assumed, Apache is explicit), so the differences might be very negligible when you consider the content in each license. Part of the bias for support of the BSD license may be because of its simplicity. > Secondly, I agree that Erlang/OTP is a gift from the Old Gods of the > North when it comes to fault tolerance and so on, but just because > app-1 is written in Erlang and app-2 is written in Ruby, doesn't > automatically mean that one will scale better than another. Both these > cloud frameworks are based (at least in part) on services running as > native OS processes - admittedly you support native Erlang > services/jobs - and the other comparative technologies are really > around routing and distribution. Here both frameworks are relying on > the same kind of underlying TCP stack that takes advantage of > non-blocking socket I/O - distributed Erlang using the inet > infrastructure running on a beam emulator compiled with --enable-poll > or whatever, versus CloudFoundry which runs on event-machine, a > framework a la libevent. Distributed Erlang is known to have > limitations of its own, that wouldn't necessarily apply to a very > large grid built on some other distribution protocol - riak for > example uses a custom protocol, as does Scalaris I think and probably > quite a few others. The fault tolerance that is built in for you > (because you're running on top of OTP) though, is admittedly a big > selling point. I am judging based on the VMs involved and their known scalability limitations. Having libevent in a project does not make it scalable. There are many projects that use libevent that do not scale judging based on loadtest results. Also, libevent usage, when done incorrectly, makes the system fail prematurely, so it should not be seen as a panacea for scalability. The approach to scalability is achievable in many ways, but the complexity of the result is a concern since that controls how testable the system is. Looking at the Erlang VM scale (with epoll/kqueue, the "+K true" (kernel-poll) erl command line argument) using the inet infrastructure and light-weight processes (one Erlang process per socket), 10K+ scalability is easily achievable with simple code that is usable as beginner examples. When you compare that to other solutions where loadtests rarely hit 10K (usually HTTP servers and different configurations), with code that is actively developed for any length of time, I think Erlang is clearly the winner. When you consider scalability across separate machines, that is a separate concern. The distributed Erlang fully connected network topology is an advantage for creating Erlang systems. However, when the node count needs to be increased beyond 100 (roughly) it is necessary to have a tree network topology to keep things fault-tolerant. ZeroMQ integration configuration could be used within CloudI to avoid the distributed Erlang node limitation. > Don't get me wrong - I'm not asserting any amazing properties about > ruby, nor am I trying to criticise your work. It's just that you're > clearly very smart (because you wrote a complex beast like CloudI) so > your peers (and even underlings like me) need to hold you to account > for intellectual honesty. Saying "it is Ruby based so it is unable to > scale..." is FUD unless you qualify it in more detail. Most people on > this list would be inclined to think you're on to something, but this > is not a black and white area. I'm pretty sure that services such as > Github and Heroku are very scalable, and they use both Ruby and Erlang > (and probably only Odin knows what else) for various tasks. Sorry, not trying to start a language flame-war. I like Ruby and enjoy how succinct it is. I just think that it is important to use the right tool for the right job. Github and Heroku have both talked publicly about how Erlang has helped them scale their back-end systems. So, I think most companies are becoming aware that a Ruby front-end can scale with an Erlang back-end. However, I don't see anything within CloudFoundry that can provide scalability like Erlang. Michael >> On 06/20/2011 11:34 AM, Tim Watson wrote: >>> There is also http://www.cloudfoundry.org, which is sponsored by >>> vmware and has a commercial offering too (at cloudfoundry.com) but is >>> open source - https://github.com/cloudfoundry. >>> >>> On 20 June 2011 15:55, Evans, Matthew wrote: >>>> Seems like you have an interesting project with cloudi. >>>> >>>> Matt >>>> >>>> -----Original Message----- >>>> From: Michael Truog [mailto:mjtruog@REDACTED] >>>> Sent: Saturday, June 18, 2011 3:15 PM >>>> To: erlang-questions@REDACTED >>>> Cc: Evans, Matthew >>>> Subject: Re: [erlang-questions] Freedom Box and Erlang >>>> >>>> To have an Erlang cloud that doesn't depend on an organization, there is CloudI (http://cloudi.org) . So it could be used for running applications on a distributed mesh. There is ZeroMQ integration that could provide communication between the FreedomBoxes. CloudI currently lacks a pretty UI for managing the configuration and will be going through more testing, so those are current barriers. >>>> >>>> On 06/18/2011 10:54 AM, Evans, Matthew wrote: >>>>> Hi, >>>>> >>>>> I've recently been reading about Freedom box (see: http://wiki.debian.org/FreedomBox and http://freedomboxfoundation.org/). >>>>> >>>>> In a nutshell, directly from their website: >>>>> >>>>> "Smart devices whose engineered purpose is to work together to facilitate free communication among people, safely and securely, beyond the ambition of the strongest power to penetrate, they can make freedom of thought and information a permanent, ineradicable feature of the net that holds our souls." >>>>> >>>>> "We live in a world where our use of the network is mediated by organizations that often do not have our best interests at heart. By building software that does not rely on a central service, we can regain control and privacy. By keeping our data in our homes, we gain useful legal protections over it. By giving back power to the users over their networks and machines, we are returning the Internet to its intended peer-to-peer architecture. " >>>>> >>>>> I'm not a member of the organization, but it seems to me that the distributed nature of Erlang could offer awesome possibilities to work around certain technical difficulties. I could, for example, envisage a mesh of Erlang nodes connected together providing various distributed services where no individual is in overall control. Or you could provide services (Erlang processes/application) owned and managed by someone from a country with poor human rights that is in fact running on an Erlang node on a country with better human rights (a chat service for example). I'm not talking about "cloud" services here (that some organization ultimately owns), but running applications on a mesh of privately owned "Freedom boxes" that host Erlang VMs. The beauty of Erlang is that it's innate ability to distribute code, processes and applications in such a simple and seamless manner could provide very powerful tools for the organization. >>>>> >>>>> Any comments and thoughts on this? >>>>> >>>>> Matt >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >> >> From dangud@REDACTED Tue Jun 21 08:31:58 2011 From: dangud@REDACTED (Dan Gudmundsson) Date: Tue, 21 Jun 2011 08:31:58 +0200 Subject: [erlang-questions] Version of C compiler for Erlang on Windows In-Reply-To: References: <20110620211907.GS2051@hijacked.us> Message-ID: On Tue, Jun 21, 2011 at 1:13 AM, Dave Cottlehuber wrote: > On 21 June 2011 09:19, Andrew Thompson wrote: > > On Mon, Jun 20, 2011 at 11:30:36AM -0400, Francis Joanis wrote: > >> Hi, > >> > >> Is Visual Studio 2005 still the preferred version of the Microsoft > >> compiler for Erlang on Windows and is it still used for the > >> pre-compiled binaries? > >> > >> I think some work had been done to support VS 2008 and I don't know > >> about VS 2010. > > > > I submitted some patches to make it build with VS2008, and they were > > accepted. No idea about 2010. > > > > Andrew > > 90% of the hassle is getting the compiler/linker/manifest paths sorted > out; these vary for every combo of VS IDE/SDK out there. > > VS 2008 Express compiles fine since R13B so long as you install the > Windows SDK v7.0 as well. At the time wxWidgets didn't compile using > VS2010 express so I stayed with VS2008. > > I use the following script / notes to set up a CouchDB build chain for > VS2008 https://github.com/dch/glazier the README is still out of date > vs scripts while I clean up some changes needed for CouchDB but the > erlang stuff should be pretty stable. The patches & notes for doing > this are now all included in the Erlang release so if it doesn't line > up please let me know & I'll work through fixing them. Finally, I now > compile OpenSSL from source > https://github.com/dch/glazier/blob/master/bin/build_openssl.cmd > & wxWidgets via > https://github.com/dch/glazier/blob/master/bin/build_wx.cmd after > applying patch equivalent to the manual steps in the README. > > IIRC wxWidgets wasn't able to be built with VS2010 last time I > checked; but there appears to be a workaround now since > http://trac.wxwidgets.org/changeset/66836/ > ref > http://wiki.wxwidgets.org/Microsoft_Visual_C%2B%2B_Guide#Visual_Studio_C.2B.2B_2010 > please post back how you get on with this. > > A+ > Dave > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Building with VS2010 and nmake works fine as well. Even better than fiddling around in the gui. /Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From qoocku@REDACTED Tue Jun 21 11:10:28 2011 From: qoocku@REDACTED (=?UTF-8?B?RGFtaWFuIERvYnJvY3p5xYRza2k=?=) Date: Tue, 21 Jun 2011 11:10:28 +0200 Subject: [erlang-questions] reltool in R14B03 strange behavior Message-ID: <4E006004.9050707@gmail.com> Hi list! I'm trying to generate a release with such reltool.config contents: {sys, [ {lib_dirs, ["../..", "../deps"]}, {rel, "mfinder", "1", [ kernel, stdlib, sasl, mfinder ]}, {rel, "start_clean", "", [ kernel, stdlib ]}, {boot_rel, "mfinder"}, {profile, embedded}, {excl_sys_filters, ["^bin/.*", "^erts.*/bin/(dialyzer|typer)"]}, {mod_cond, all}, {app_file, all}, {excl_archive_filters, [".*"]}, {debug_info, strip}, {app, stdlib, [{incl_cond, include}]}, {app, sasl, [{incl_cond, include}]}, {app, kernel, [{incl_cond, include}]} ]}. and the result is that the release does not have "inet*" modules beams. So, any attempt to run a node of this release ends up with error rreporting undefined inet_db:start_link/1 function. What's happening? -- D. -------------- next part -------------- A non-text attachment was scrubbed... Name: 0x0CAE3AEB.asc Type: application/pgp-keys Size: 5351 bytes Desc: not available URL: From qoocku@REDACTED Tue Jun 21 11:30:31 2011 From: qoocku@REDACTED (=?UTF-8?B?RGFtaWFuIERvYnJvY3p5xYRza2k=?=) Date: Tue, 21 Jun 2011 11:30:31 +0200 Subject: [erlang-questions] reltool in R14B03 strange behavior In-Reply-To: <4E006004.9050707@gmail.com> References: <4E006004.9050707@gmail.com> Message-ID: <4E0064B7.50708@gmail.com> Ok. There's no problem. My fault. I've had "excl_app_filter" patterns which did the job and in fact excluded a bunch of applications from the release. -- D. W dniu 21.06.2011 11:10, Damian Dobroczy?ski pisze: > Hi list! > > I'm trying to generate a release with such reltool.config contents: > > {sys, [ > {lib_dirs, ["../..", "../deps"]}, > {rel, "mfinder", "1", > [ > kernel, > stdlib, > sasl, > mfinder > ]}, > {rel, "start_clean", "", > [ > kernel, > stdlib > ]}, > {boot_rel, "mfinder"}, > {profile, embedded}, > {excl_sys_filters, ["^bin/.*", > "^erts.*/bin/(dialyzer|typer)"]}, > {mod_cond, all}, > {app_file, all}, > {excl_archive_filters, [".*"]}, > {debug_info, strip}, > {app, stdlib, [{incl_cond, include}]}, > {app, sasl, [{incl_cond, include}]}, > {app, kernel, [{incl_cond, include}]} > ]}. > > and the result is that the release does not have "inet*" modules beams. > So, any attempt to run a node of this release ends up with error > rreporting undefined inet_db:start_link/1 function. What's happening? > > -- D. > > -------------- next part -------------- A non-text attachment was scrubbed... Name: 0x0CAE3AEB.asc Type: application/pgp-keys Size: 5351 bytes Desc: not available URL: From watson.timothy@REDACTED Tue Jun 21 12:47:46 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 21 Jun 2011 11:47:46 +0100 Subject: [erlang-questions] Freedom Box and Erlang In-Reply-To: <4E000396.30507@gmail.com> References: <4DFCF919.5070106@gmail.com> <4DFF99DB.60401@gmail.com> <4E000396.30507@gmail.com> Message-ID: On 21 June 2011 03:36, Michael Truog wrote: > On 06/20/2011 02:42 PM, Tim Watson wrote: >> On 20 June 2011 20:04, Michael Truog wrote: >>> True. ?CloudFoundry exists and is open source. ?However, it is Ruby based so it is unable to scale like an Erlang system (like CloudI). ?CloudFoundry is control by VMware, which is an organization, and they are using the Apache License 2.0. ?CloudI is independent and uses a BSD license, so it can be reused anywhere. >> First up, I'm definitely not knocking CloudI here - I think it looks >> awesome. CloudFoundry is owned and developed by an organisation, but >> so are plenty of other things (riak, rabbitmq, etc) so that doesn't >> automatically mean lock-in. I mean Erlang/OTP is controlled by >> Ericsson right, and by the same measure CloudI is controlled by >> Michael Truog. The Apache License 2.0 is more restrictive than the BSD >> type license you've chosen for CloudI - and good on you for making it >> as open as possible - but I can still use and adapt stuff based on the >> Apache license for commercial or similarly licensed open source >> projects. > I agree. ?Often times organizations make strategic decisions to include and exclude support for various types of software, which either benefits their partners/divisions or harms their competition. ?Having an open source project that is independent means that it is able to pursue a path to the best combination of technology based on what the community thinks (though limited by time/money). ?That way there should be no major influence based on "competitive advantage" decisions that are commonly made by any commercial organization. > > I am not a lawyer and I am not giving legal advise. ?However, I think it is more likely that BSD licensed software is reused and then sold in a commercial product, when compared with Apache licensed software. ?The main difference between the licenses is over patent ownership (BSD is assumed, Apache is explicit), so the differences might be very negligible when you consider the content in each license. ?Part of the bias for support of the BSD license may be because of its simplicity. I'd agree with that - I think the simplicity is for many of us, a compelling reason for adopting it. >> Secondly, I agree that Erlang/OTP is a gift from the Old Gods of the >> North when it comes to fault tolerance and so on, but just because >> app-1 is written in Erlang and app-2 is written in Ruby, doesn't >> automatically mean that one will scale better than another. Both these >> cloud frameworks are based (at least in part) on services running as >> native OS processes - admittedly you support native Erlang >> services/jobs - and the other comparative technologies are really >> around routing and distribution. Here both frameworks are relying on >> the same kind of underlying TCP stack that takes advantage of >> non-blocking socket I/O - distributed Erlang using the inet >> infrastructure running on a beam emulator compiled with --enable-poll >> or whatever, versus CloudFoundry which runs on event-machine, a >> framework a la libevent. Distributed Erlang is known to have >> limitations of its own, that wouldn't necessarily apply to a very >> large grid built on some other distribution protocol - riak for >> example uses a custom protocol, as does Scalaris I think and probably >> quite a few others. The fault tolerance that is built in for you >> (because you're running on top of OTP) though, is admittedly a big >> selling point. > I am judging based on the VMs involved and their known scalability limitations. ?Having libevent in a project does not make it scalable. ?There are many projects that use libevent that do not scale judging based on loadtest results. ?Also, libevent usage, when done incorrectly, makes the system fail prematurely, so it should not be seen as a panacea for scalability. ?The approach to scalability is achievable in many ways, but the complexity of the result is a concern since that controls how testable the system is. ?Looking at the Erlang VM scale (with epoll/kqueue, the "+K true" (kernel-poll) erl command line argument) using the inet infrastructure and light-weight processes (one Erlang process per socket), 10K+ scalability is easily achievable with simple code that is usable as beginner examples. ?When you compare that to other solutions where loadtests rarely hit 10K (usually HTTP servers and different configurations), with code that is actively developed for any length of > time, I think Erlang is clearly the winner. Yes I agree, based on my own experience migrating SIP/H.323 applications from Java/J2EE/JAIN-SIP to Erlang/OTP - even supposedly very stable platforms (e.g., the JVM) don't necessarily compete with Erlang and when using OTP the resulting system is actually far simpler and easier to understand and test. My thoughts here were that it is the (soft) routing that is really under consideration when we compare the two. Looking at both code bases, I don't see the CloudFoundry code actually doing that much work with requests - it mainly just forwards them to the right HTTP endpoint. I wanted to point out that whilst Ruby VMs are likely inferior to BEAM when it comes to building HTTP servers, multiplexing socket I/O *can* shave away that advantage if it is done right. I've seen python network servers out perform Java applications, which surprised the hell out of a lot of people. > > When you consider scalability across separate machines, that is a separate concern. ?The distributed Erlang fully connected network topology is an advantage for creating Erlang systems. ?However, when the node count needs to be increased beyond 100 (roughly) it is necessary to have a tree network topology to keep things fault-tolerant. ?ZeroMQ integration configuration could be used within CloudI to avoid the distributed Erlang node limitation. > Yes, and this is definitely an area where Erlang/OTP clearly shines. I think not depending on distributed Erlang - using some other integration such as 0MQ or whatever - would be very sensible for CloudI. Just as I suspect that having a routing engine built in Erlang would be much better for CloudFoundry IMO. :) >> Don't get me wrong - I'm not asserting any amazing properties about >> ruby, nor am I trying to criticise your work. It's just that you're >> clearly very smart (because you wrote a complex beast like CloudI) so >> your peers (and even underlings like me) need to hold you to account >> for intellectual honesty. Saying "it is Ruby based so it is unable to >> scale..." is FUD unless you qualify it in more detail. Most people on >> this list would be inclined to think you're on to something, but this >> is not a black and white area. I'm pretty sure that services such as >> Github and Heroku are very scalable, and they use both Ruby and Erlang >> (and probably only Odin knows what else) for various tasks. > Sorry, not trying to start a language flame-war. ?I like Ruby and enjoy how succinct it is. ?I just think that it is important to use the right tool for the right job. ?Github and Heroku have both talked publicly about how Erlang has helped them scale their back-end systems. ?So, I think most companies are becoming aware that a Ruby front-end can scale with an Erlang back-end. ?However, I don't see anything within CloudFoundry that can provide scalability like Erlang. That's better - I was mainly concerned with getting you to state your reasoning. ;) What you say makes good sense and I agree that Erlang's benefits are exemplified by the Github and Heroku architectures (any many others) which rely on it. I would point out that some backend processes (on both platform IIRC) that are not front-end are actually implemented in Ruby (or other tech), but Erlang is right there in the core nonetheless. I've found that "the right tool for the right job" is absolutely the best way to go about building software, so I wholeheartedly support that sentiment. Thanks for your constructive responses Michael, and very best of luck with CloudI! Cheers, Tim From fernando.benavides@REDACTED Tue Jun 21 15:00:22 2011 From: fernando.benavides@REDACTED (Fernando "Brujo" Benavides) Date: Tue, 21 Jun 2011 10:00:22 -0300 Subject: [erlang-questions] bug in timer module in R14B02? In-Reply-To: References: Message-ID: +1. We have the same issues here, and also moved to erlang:send_after. We still use timer:send_interval but just for small intervals. On 19/06/2011, at 23:02, Ryan Huffman wrote: > I ran into a lot of problems with the timer module on R14B02 and eventually switched to using erlang:send_after. This particular section in the erlang doc also convinced me to switch over to erlang:send_after and maintain my own timer: http://www.erlang.org/doc/efficiency_guide/commoncaveats.html#id58959 > > I was running on a 64-bit system (CentOS 5.5), so, as Dale mentioned, the problem may be fixed. > > Ryan > > On Sun, Jun 19, 2011 at 5:00 PM, Dale Harvey wrote: > I believe there was an integer overflow bug on 64 bit systems that affect ordered ets tables (and therefore caused timers not to trigger) in R14B02 > > http://erlang.org/pipermail/erlang-bugs/2011-April/002294.html > > That may be your problem? > > > On 19 June 2011 03:43, Zsolt Keszthelyi wrote: > Dear All, > > I ran into something in R14B02. > It looks like if I start a timer with a long interval then all previously started timers stop. > The timer server is still running and the timers are in the ETS table. > No error returned. Nothing crashes. > This was working on R13B03 and is not working on R14B02 > The long interval is exactly 2^31/1000 ms. > Is it a bug or an intentional change? > Maybe the timer module was never meant to be used with long intervals but shouldn't it give at least an error msg.? > > Code: > > %% works on R13B03 > %% doesn't work on R14B02 > > -module(timerbug). > > -compile(export_all). > > start()-> > timer:apply_interval(1000, io, format, ["Hello!\n"]), > timer:sleep(5000), > timer:apply_interval(40*60*1000, io, format, ["No more hello.\n"]). > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethstone@REDACTED Tue Jun 21 17:26:53 2011 From: kennethstone@REDACTED (Kenny Stone) Date: Tue, 21 Jun 2011 10:26:53 -0500 Subject: [erlang-questions] strftime for formatting dates and times Message-ID: https://github.com/kennystone/strftimerl I literally copied ruby's strftime docs into the readme and implemented (almost) each type of formatting. I'm considering patching the calendar module with this functionality. Thoughts? strftime:f(now(), "Printed on %m/%d/%Y"). %=> "Printed on 11/19/2007" strftime:f(now(), "at %I:%M%p"). %=> "at 08:37AM" strftime:f(now(), "at %I:%M%p", universal). %=> "at 02:37PM" strftime:f(now(), "%D-%T.%N"). %=> "11/19/2007-08:38:02.445443" -Kenny -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam@REDACTED Tue Jun 21 17:28:41 2011 From: adam@REDACTED (Adam Lindberg) Date: Tue, 21 Jun 2011 17:28:41 +0200 Subject: [erlang-questions] strftime for formatting dates and times In-Reply-To: References: Message-ID: <4E00B8A9.5040708@erlang-solutions.com> Would love this being integrated into calendar! (Or appropriate module, although I think calendar should be the one) Cheers, Adam Kenny Stone wrote: > https://github.com/kennystone/strftimerl > > I literally copied ruby's strftime docs into the readme and implemented > (almost) each type of formatting. > > I'm considering patching the calendar module with this functionality. > Thoughts? > > strftime:f(now(), "Printed on %m/%d/%Y"). %=> "Printed on 11/19/2007" > strftime:f(now(), "at %I:%M%p"). %=> "at 08:37AM" > strftime:f(now(), "at %I:%M%p", universal). %=> "at 02:37PM" > strftime:f(now(), "%D-%T.%N"). %=> > "11/19/2007-08:38:02.445443" > > -Kenny > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vincent.dephily@REDACTED Tue Jun 21 18:41:57 2011 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Tue, 21 Jun 2011 09:41:57 -0700 (PDT) Subject: [erlang-questions] bug in timer module in R14B02? In-Reply-To: References: Message-ID: <40909869.boHhXl9cKr@moltowork> On Tuesday 21 June 2011 10:00:22 Fernando Brujo Benavides wrote: > +1. We have the same issues here, and also moved to erlang:send_after. > We still use timer:send_interval but just for small intervals. FWIW, we moved to processes that do "receive ... after" in a loop, because we need to "update the timers" (aka "dont do this now, do it a little later instead") very frequently. We just send the new deadline to that process whenever the timer is to be updated. -- Vincent de Phily Mobile Devices +33 (0) 142 119 325 +353 (0) 85 710 6320 Warning This message (and any associated files) is intended only for the use of its intended recipient and may contain information that is confidential, subject to copyright or constitutes a trade secret. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this message, or files associated with this message, is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. Any views or opinions presented are solely those of the author vincent.dephily@REDACTED and do not necessarily represent those of the company. Although the company has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. From zsolt.erl@REDACTED Tue Jun 21 19:20:47 2011 From: zsolt.erl@REDACTED (Zsolt Keszthelyi) Date: Tue, 21 Jun 2011 13:20:47 -0400 Subject: [erlang-questions] strftime for formatting dates and times In-Reply-To: <4E00B8A9.5040708@erlang-solutions.com> References: <4E00B8A9.5040708@erlang-solutions.com> Message-ID: +1 for adding it to calendar (I keep having to use some partial solution) Are you planning to do a parser also (parse date string to datetime tuple)? That would really make it complete. Zsolt On Tue, Jun 21, 2011 at 11:28 AM, Adam Lindberg wrote: > Would love this being integrated into calendar! (Or appropriate module, > although I think calendar should be the one) > > Cheers, > Adam > > > > Kenny Stone wrote: > >> https://github.com/kennystone/**strftimerl >> >> I literally copied ruby's strftime docs into the readme and implemented >> (almost) each type of formatting. >> >> I'm considering patching the calendar module with this functionality. >> Thoughts? >> >> strftime:f(now(), "Printed on %m/%d/%Y"). %=> "Printed on 11/19/2007" >> strftime:f(now(), "at %I:%M%p"). %=> "at 08:37AM" >> strftime:f(now(), "at %I:%M%p", universal). %=> "at 02:37PM" >> strftime:f(now(), "%D-%T.%N"). %=> >> "11/19/2007-08:38:02.445443" >> >> -Kenny >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dale@REDACTED Tue Jun 21 19:26:19 2011 From: dale@REDACTED (Dale Harvey) Date: Tue, 21 Jun 2011 18:26:19 +0100 Subject: [erlang-questions] strftime for formatting dates and times In-Reply-To: References: <4E00B8A9.5040708@erlang-solutions.com> Message-ID: Jinx :) https://github.com/daleharvey/dh_date/blob/master/src/dh_date.erl (date formatter + parser) Cheers Dale On 21 June 2011 18:20, Zsolt Keszthelyi wrote: > +1 for adding it to calendar (I keep having to use some partial solution) > Are you planning to do a parser also (parse date string to datetime tuple)? > That would really make it complete. > > Zsolt > > > On Tue, Jun 21, 2011 at 11:28 AM, Adam Lindberg > wrote: > >> Would love this being integrated into calendar! (Or appropriate module, >> although I think calendar should be the one) >> >> Cheers, >> Adam >> >> >> >> Kenny Stone wrote: >> >>> https://github.com/kennystone/**strftimerl >>> >>> I literally copied ruby's strftime docs into the readme and implemented >>> (almost) each type of formatting. >>> >>> I'm considering patching the calendar module with this functionality. >>> Thoughts? >>> >>> strftime:f(now(), "Printed on %m/%d/%Y"). %=> "Printed on 11/19/2007" >>> strftime:f(now(), "at %I:%M%p"). %=> "at 08:37AM" >>> strftime:f(now(), "at %I:%M%p", universal). %=> "at 02:37PM" >>> strftime:f(now(), "%D-%T.%N"). %=> >>> "11/19/2007-08:38:02.445443" >>> >>> -Kenny >>> >>> ______________________________**_________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/**listinfo/erlang-questions >>> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From egil@REDACTED Tue Jun 21 19:38:38 2011 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Tue, 21 Jun 2011 19:38:38 +0200 Subject: [erlang-questions] strftime for formatting dates and times In-Reply-To: <4E00B8A9.5040708@erlang-solutions.com> References: <4E00B8A9.5040708@erlang-solutions.com> Message-ID: <4E00D71E.30003@erix.ericsson.se> On 2011-06-21 17:28, Adam Lindberg wrote: > Would love this being integrated into calendar! (Or appropriate module, > although I think calendar should be the one) I like this also and I probably would want something similar folded into calendar or another module. * Abit of topic * This weekend a wrote something similar looking at python, ruby and perl api:s for datetime, time and calendar functionality. Trying to get a feel for what would be ok. However, most of those api:s are emulations of standard C time conversions. gmtime, mktime, localtime etc. Horrible horrible syntax in my opinion. On the other hand it is rather familiar. Why would you look on something like this? Well conversion between timezones, utc, epochs, dst and such would really be great to have. And i do not think calendar will suffice for this. For instance, should we have a #time{} record instead of {{Y,Mon,D},{H,Min,S}} :: datetime()? -record(time, { year :: integer(), month :: 1..12, mday :: 1..31, hour :: 0..23, min :: 0..59, sec :: 0..60, wday :: 1..7, yday :: 1..366, is_dst :: -1 | 0 | 1 }). Or what would suffice? and timezones? Just some thoughts. // Bj?rn-Egil > > Cheers, > Adam > > > > Kenny Stone wrote: >> https://github.com/kennystone/strftimerl >> >> I literally copied ruby's strftime docs into the readme and implemented >> (almost) each type of formatting. >> >> I'm considering patching the calendar module with this functionality. >> Thoughts? >> >> strftime:f(now(), "Printed on %m/%d/%Y"). %=> "Printed on 11/19/2007" >> strftime:f(now(), "at %I:%M%p"). %=> "at 08:37AM" >> strftime:f(now(), "at %I:%M%p", universal). %=> "at 02:37PM" >> strftime:f(now(), "%D-%T.%N"). %=> >> "11/19/2007-08:38:02.445443" >> >> -Kenny >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From max.lapshin@REDACTED Tue Jun 21 19:54:36 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 21 Jun 2011 21:54:36 +0400 Subject: [erlang-questions] Trace/BPT trap on OS X when loading linked-in driver Message-ID: I've written linked-in driver for capturing sound with portaudio library. When I load it, erlang exits with Trace/BPT trap message. The following stacktrace in gdb: #0 0x00007fff88d1f570 in __CFInitialize () #1 0x00007fff5fc0d5ce in __dyld__ZN16ImageLoaderMachO11doImageInitERKN11ImageLoader11LinkContextE () #2 0x00007fff5fc0d607 in __dyld__ZN16ImageLoaderMachO16doInitializationERKN11ImageLoader11LinkContextE () #3 0x00007fff5fc0bcec in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEj () ... This happens because it is loading in non-main thread and CFInitialize _must_ be called in main thread. http://openradar.appspot.com/7209349 Is there any way to order erlang to load driver in main thread? Or maybe there is some way to do workaround with some DYLD environment variables? From darach@REDACTED Tue Jun 21 23:53:36 2011 From: darach@REDACTED (Darach Ennis) Date: Tue, 21 Jun 2011 22:53:36 +0100 Subject: [erlang-questions] Regarding PteraCUDA. In-Reply-To: <14781FCA-64CA-4569-BB33-5F11B6B39FF2@hypotheticalabs.com> References: <14781FCA-64CA-4569-BB33-5F11B6B39FF2@hypotheticalabs.com> Message-ID: <3A4B690C-7D46-4EC4-94CC-5EECDAD6EEAF@gmail.com> Hi Gene, I'm happy to assist while Kevin's machine is out of comish Cheers, Darach. On 21 Jun 2011, at 02:55, Kevin Smith wrote: > Hi -- > > I'm the author of pteracuda and I'd love to hear what, if anything, people are doing with pteracuda. I've hit a temporary stall in development as my primary dev machine was rendered slightly inoperable when I shipped it back from Erlang Factory. I expect to have the broken hardware replaced soonish and then I can resume development. > > --Kevin > On Jun 20, 2011, at 5:09 PM, G.S. wrote: > >> Hello, >> >> Has anyone experimented with PteraCUDA and could share their experience with it? >> >> Regards, >> -Gene >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From freza@REDACTED Tue Jun 21 23:42:44 2011 From: freza@REDACTED (Jachym Holecek) Date: Tue, 21 Jun 2011 22:42:44 +0100 Subject: [erlang-questions] strftime for formatting dates and times In-Reply-To: References: Message-ID: <20110621214244.GA1936@hanele> # Kenny Stone 2011-06-21: > https://github.com/kennystone/strftimerl > > I literally copied ruby's strftime docs into the readme and implemented (almost) each type of > formatting. > > I'm considering patching the calendar module with this functionality. Thoughts? > > strftime:f(now(), "Printed on %m/%d/%Y"). %=> "Printed on 11/19/2007" > strftime:f(now(), "at %I:%M%p"). %=> "at 08:37AM" > strftime:f(now(), "at %I:%M%p", universal). %=> "at 02:37PM" > strftime:f(now(), "%D-%T.%N"). %=> "11/19/2007-08:38:02.445443" Yet another weird formatting DSL for something that is more conveniently done as iolists using calendar and perhaps a few helper functions. Oh boy, am I a killjoy... The timezone/dst/etc conversion API mentioned in another post would be lovely though, preferrably using now() and/or datetime() as primary time representations, like everything else does (just provide functions that calculate the interesting bits when you need them, no?). BR, -- Jachym From vances@REDACTED Wed Jun 22 03:51:24 2011 From: vances@REDACTED (Vance Shipley) Date: Tue, 21 Jun 2011 21:51:24 -0400 Subject: [erlang-questions] Books Message-ID: <20110622015124.GD49732@h216-235-12-167.host.egate.net> Please excuse the noise ... I'm divesting myself of my paper based library and have at least two books which may be of interest to folks here: Concurrent programming in ERLANG Second Edition Robert Virding, Claes Wikstrom, Mike Williams, Joe Armstrong (Editor) ISBN: 013508301X Concurrency Theory Calculi and Automata for Modelling Untimed and Timed Concurrent Systems Howard Bowman, Rodolfo Gomez ISBN: 1852338954 Free to a good home! -- -Vance From nick@REDACTED Wed Jun 22 10:43:26 2011 From: nick@REDACTED (Niclas Eklund) Date: Wed, 22 Jun 2011 10:43:26 +0200 Subject: [erlang-questions] strftime for formatting dates and times In-Reply-To: <4E00D71E.30003@erix.ericsson.se> References: <4E00B8A9.5040708@erlang-solutions.com> <4E00D71E.30003@erix.ericsson.se> Message-ID: On Tue, 21 Jun 2011, Bj?rn-Egil Dahlberg wrote: > -record(time, { > year :: integer(), > month :: 1..12, > mday :: 1..31, > hour :: 0..23, > min :: 0..59, > sec :: 0..60, > wday :: 1..7, > yday :: 1..366, > is_dst :: -1 | 0 | 1 > }). > > Or what would suffice? and timezones? > > Just some thoughts. > > // Bj?rn-Egil > Remember that the UTC offset in some cases exceeds 12, e.g. Line Islands (UTC+14:00 - 2011-06-22T08:15:49+14:00) and Tonga (UTC+13:00 - 2011-06-22T07:15:49+13:00). It can also be off by half an hour/15 minutes, e.g. Venezuela (UTC-04:30 - 2011-06-21T13:45:49-04:30) and Nepal (UTC+05:45 - 2011-06-22T00:00:49+05:45). IMHO ISO 8601 notation should be an option, even though it might be a "fuzzy". Best Regards, Niclas @ Erlang/OTP From masklinn@REDACTED Wed Jun 22 10:49:13 2011 From: masklinn@REDACTED (Masklinn) Date: Wed, 22 Jun 2011 10:49:13 +0200 Subject: [erlang-questions] strftime for formatting dates and times In-Reply-To: References: Message-ID: <5007551D-1B33-48BE-804D-8E60D91C16DB@masklinn.net> On 2011-06-21, at 17:26 , Kenny Stone wrote: > https://github.com/kennystone/strftimerl > > I literally copied ruby's strftime docs into the readme and implemented > (almost) each type of formatting. > > I'm considering patching the calendar module with this functionality. > Thoughts? > > strftime:f(now(), "Printed on %m/%d/%Y"). %=> "Printed on 11/19/2007" > strftime:f(now(), "at %I:%M%p"). %=> "at 08:37AM" > strftime:f(now(), "at %I:%M%p", universal). %=> "at 02:37PM" > strftime:f(now(), "%D-%T.%N"). %=> > "11/19/2007-08:38:02.445443" > > -Kenny Why not implement Unicode LDML's Date Format Pattern[0] instead? A second note is that using Ruby's strftime as your base may not be a good idea: it's not compatible with BSD libc's strftime (the extensions are different), though it might match POSIX's un-extended spec. I'd suggest either sticking to POSIX's strftime or using libc's as your base, rather than Ruby's. [0] http://unicode.org/reports/tr35/tr35-6.html#Date_Format_Patterns From masklinn@REDACTED Wed Jun 22 10:53:16 2011 From: masklinn@REDACTED (Masklinn) Date: Wed, 22 Jun 2011 10:53:16 +0200 Subject: [erlang-questions] strftime for formatting dates and times In-Reply-To: <4E00D71E.30003@erix.ericsson.se> References: <4E00B8A9.5040708@erlang-solutions.com> <4E00D71E.30003@erix.ericsson.se> Message-ID: <91D3B83A-9F79-4B15-80E9-D9589F799AE5@masklinn.net> On 2011-06-21, at 19:38 , Bj?rn-Egil Dahlberg wrote: > On 2011-06-21 17:28, Adam Lindberg wrote: >> Would love this being integrated into calendar! (Or appropriate module, >> although I think calendar should be the one) > > I like this also and I probably would want something similar folded into calendar or another module. > > * Abit of topic * > > This weekend a wrote something similar looking at python, ruby and perl api:s for datetime, time and calendar functionality. Trying to get a feel for what would be ok. However, most of those api:s are emulations of standard C time conversions. gmtime, mktime, localtime etc. Horrible horrible syntax in my opinion. On the other hand it is rather familiar. > Though it's mostly object-oriented (and quite verbose owing to being Java), Joda-Time is generally considered a pretty nice API for an API handling just about everything related to date and time. See also JSR-310, which was authored by Joda's original developer and is an evolution of Joda (rather than a port or direct evolution) using lessons learned from Joda. From lukas.larsson@REDACTED Wed Jun 22 10:58:41 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Wed, 22 Jun 2011 08:58:41 +0000 (GMT) Subject: [erlang-questions] Trace/BPT trap on OS X when loading linked-in driver In-Reply-To: Message-ID: <116735684.171308733121154.JavaMail.root@zimbra> Hi, If I remember correctly the wx driver had the same issue on OS X, so you might want to have a look there are how Dan managed that. Lukas ----- Original Message ----- From: "Max Lapshin" To: "Erlang-Questions Questions" Sent: Tuesday, 21 June, 2011 19:54:36 GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: [erlang-questions] Trace/BPT trap on OS X when loading linked-in driver I've written linked-in driver for capturing sound with portaudio library. When I load it, erlang exits with Trace/BPT trap message. The following stacktrace in gdb: #0 0x00007fff88d1f570 in __CFInitialize () #1 0x00007fff5fc0d5ce in __dyld__ZN16ImageLoaderMachO11doImageInitERKN11ImageLoader11LinkContextE () #2 0x00007fff5fc0d607 in __dyld__ZN16ImageLoaderMachO16doInitializationERKN11ImageLoader11LinkContextE () #3 0x00007fff5fc0bcec in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEj () ... This happens because it is loading in non-main thread and CFInitialize _must_ be called in main thread. http://openradar.appspot.com/7209349 Is there any way to order erlang to load driver in main thread? Or maybe there is some way to do workaround with some DYLD environment variables? _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From sidcarter@REDACTED Wed Jun 22 11:03:41 2011 From: sidcarter@REDACTED (Sid Carter) Date: Wed, 22 Jun 2011 21:03:41 +1200 Subject: [erlang-questions] Beginner: What's wrong with this code? Message-ID: Hi Folks, New to Erlang and playing with code and trying to learn erlang. I have been reading the getting started guide and trying this code (on Mac OS X with erlang from MacPorts): -module(first). % call would look like format_temps([{moscow,{f,-40}}, {bangalore,{f,40}}, {wellington, {c,24}}]). -export([format_temps/1, start/0]). format_temps([]) -> ok; format_temps([City|Rest]) -> print_temp(convert_to_celsius(City)), format_temps(Rest). convert_to_celsius({City, {c, C}}) -> {City, {c, C}}; convert_to_celsius({City, {f, F}}) -> {City, {c, (F-32)*5/9}}. print_temp({City, {c, C}}) -> io:format("~p has a temperature of ~w C~n",[City,C]). start() -> Pid = spawn(?MODULE, format_temps, [{moscow,{f,-40}}, {bangalore,{f,40}}, {wellington, {c,24}}]), Pid. When I try to run: first:format_temps(args). I don't have any problems. I get the correct response. When I try to run as: first:start(). I get the following error: Error in process <0.49.0> with exit value: {undef,[{first,format_temps,[{moscow,{f,-40}},{bangalore,{f,40}},{wellington,{c,24}}]}]} What's wrong? Thanks Sid -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Wed Jun 22 11:09:13 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 22 Jun 2011 11:09:13 +0200 Subject: [erlang-questions] Beginner: What's wrong with this code? In-Reply-To: References: Message-ID: Hi, On Wed, Jun 22, 2011 at 11:03, Sid Carter wrote: > spawn(?MODULE, format_temps, [{moscow,{f,-40}}, {bangalore,{f,40}}, > {wellington, {c,24}}]), This tries to find a format_temps that takes three arguments (spawn's Args argument is a list of the arguments taken by the function). Try instead spawn(?MODULE, format_temps, [[{moscow,{f,-40}}, {bangalore,{f,40}}, {wellington, {c,24}}]]), regards, Vlad From koops.j@REDACTED Wed Jun 22 11:10:15 2011 From: koops.j@REDACTED (Jeroen Koops) Date: Wed, 22 Jun 2011 11:10:15 +0200 Subject: [erlang-questions] Beginner: What's wrong with this code? In-Reply-To: References: Message-ID: Hi Sid, The final argument to spawn is a list of arguments. In your case, the function format_temps takes one argument, which happens to be a list. Therefore, you should use a list containing a list as the final argument to spawn, as in: spawn(?MODULE, format_temps, [ [{moscow,{f,-40}}, {bangalore,{f,40}}, {wellington, {c,24}}] ]) On Wed, Jun 22, 2011 at 11:03 AM, Sid Carter wrote: > Hi Folks, > > New to Erlang and playing with code and trying to learn erlang. > > I have been reading the getting started guide and trying this code (on Mac > OS X with erlang from MacPorts): > > -module(first). > % call would look like format_temps([{moscow,{f,-40}}, {bangalore,{f,40}}, > {wellington, {c,24}}]). > -export([format_temps/1, start/0]). > format_temps([]) -> > ok; > format_temps([City|Rest]) -> > print_temp(convert_to_celsius(City)), > format_temps(Rest). > convert_to_celsius({City, {c, C}}) -> > {City, {c, C}}; > convert_to_celsius({City, {f, F}}) -> > {City, {c, (F-32)*5/9}}. > print_temp({City, {c, C}}) -> > io:format("~p has a temperature of ~w C~n",[City,C]). > start() -> > Pid = spawn(?MODULE, format_temps, [{moscow,{f,-40}}, {bangalore,{f,40}}, > {wellington, {c,24}}]), > Pid. > > When I try to run: > > first:format_temps(args). > > I don't have any problems. I get the correct response. > > When I try to run as: > > first:start(). > > I get the following error: > > Error in process <0.49.0> with exit value: > {undef,[{first,format_temps,[{moscow,{f,-40}},{bangalore,{f,40}},{wellington,{c,24}}]}]} > > What's wrong? > > Thanks > Sid > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve@REDACTED Wed Jun 22 11:10:15 2011 From: steve@REDACTED (Steve Strong) Date: Wed, 22 Jun 2011 11:10:15 +0200 Subject: [erlang-questions] Beginner: What's wrong with this code? In-Reply-To: References: Message-ID: <8926D9CC-71F4-4A64-A9E6-A544192B889F@srstrong.com> Your spawn command is passing the arguments incorrectly - you need: spawn(?MODULE, format_temps, [[{moscow,{f,-40}}, {bangalore,{f,40}}, {wellington, {c,24}}]]) without the outer list, you were trying to call first:format_temps({moscow,{f,-40}}, {bangalore,{f,40}}, {wellington, {c,24}}), which doesn't exist. Cheers, Steve -- Steve Strong, Director, id3as twitter.com/srstrong On 22 Jun 2011, at 11:03, Sid Carter wrote: > -module(first). > % call would look like format_temps([{moscow,{f,-40}}, {bangalore,{f,40}}, {wellington, {c,24}}]). > -export([format_temps/1, start/0]). > format_temps([]) -> > ok; > format_temps([City|Rest]) -> > print_temp(convert_to_celsius(City)), > format_temps(Rest). > convert_to_celsius({City, {c, C}}) -> > {City, {c, C}}; > convert_to_celsius({City, {f, F}}) -> > {City, {c, (F-32)*5/9}}. > print_temp({City, {c, C}}) -> > io:format("~p has a temperature of ~w C~n",[City,C]). > start() -> > Pid = spawn(?MODULE, format_temps, [{moscow,{f,-40}}, {bangalore,{f,40}}, {wellington, {c,24}}]), > Pid. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sidcarter@REDACTED Wed Jun 22 11:13:02 2011 From: sidcarter@REDACTED (Sid Carter) Date: Wed, 22 Jun 2011 21:13:02 +1200 Subject: [erlang-questions] Beginner: What's wrong with this code? In-Reply-To: <8926D9CC-71F4-4A64-A9E6-A544192B889F@srstrong.com> References: <8926D9CC-71F4-4A64-A9E6-A544192B889F@srstrong.com> Message-ID: <12482E86752A49A6BF2A267F42FB1CF1@gmail.com> Thanks all, that fixed it. Sid On Wednesday, 22 June 2011 at 9:10 PM, Steve Strong wrote: > Your spawn command is passing the arguments incorrectly - you need: > > spawn(?MODULE, format_temps, [[{moscow,{f,-40}}, {bangalore,{f,40}}, {wellington, {c,24}}]]) > > without the outer list, you were trying to call first:format_temps({moscow,{f,-40}}, {bangalore,{f,40}}, {wellington, {c,24}}), which doesn't exist. > > Cheers, > > Steve > -- > Steve Strong, Director, id3as > twitter.com/srstrong (http://twitter.com/srstrong) > > > > > On 22 Jun 2011, at 11:03, Sid Carter wrote: > > -module(first). > > % call would look like format_temps([{moscow,{f,-40}}, {bangalore,{f,40}}, {wellington, {c,24}}]). > > -export([format_temps/1, start/0]). > > format_temps([]) -> > > ok; > > format_temps([City|Rest]) -> > > print_temp(convert_to_celsius(City)), > > format_temps(Rest). > > convert_to_celsius({City, {c, C}}) -> > > {City, {c, C}}; > > convert_to_celsius({City, {f, F}}) -> > > {City, {c, (F-32)*5/9}}. > > print_temp({City, {c, C}}) -> > > io:format("~p has a temperature of ~w C~n",[City,C]). > > start() -> > > Pid = spawn(?MODULE, format_temps, [{moscow,{f,-40}}, {bangalore,{f,40}}, {wellington, {c,24}}]), > > Pid. > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Wed Jun 22 14:32:09 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 22 Jun 2011 15:32:09 +0300 Subject: [erlang-questions] Beginner: What's wrong with this code? In-Reply-To: References: Message-ID: <4E01E0C9.2090700@cs.ntua.gr> Sid Carter wrote: > > Hi Folks, > > New to Erlang and playing with code and trying to learn erlang. > > I have been reading the getting started guide and trying this code (on > Mac OS X with erlang from MacPorts): > > -module(first). > % call would look like format_temps([{moscow,{f,-40}}, > {bangalore,{f,40}}, {wellington, {c,24}}]). > -export([format_temps/1, start/0]). > format_temps([]) -> > ok; > format_temps([City|Rest]) -> > print_temp(convert_to_celsius(City)), > format_temps(Rest). > convert_to_celsius({City, {c, C}}) -> > {City, {c, C}}; > convert_to_celsius({City, {f, F}}) -> > {City, {c, (F-32)*5/9}}. > print_temp({City, {c, C}}) -> > io:format("~p has a temperature of ~w C~n",[City,C]). > start() -> > Pid = spawn(?MODULE, format_temps, [{moscow,{f,-40}}, > {bangalore,{f,40}}, {wellington, {c,24}}]), > Pid. > > When I try to run: > > first:format_temps(args). > > I don't have any problems. I get the correct response. > > When I try to run as: > > first:start(). > > I get the following error: > > Error in process <0.49.0> with exit value: > {undef,[{first,format_temps,[{moscow,{f,-40}},{bangalore,{f,40}},{wellington,{c,24}}]}]} > > What's wrong? Others have already explained the problem and suggested a fix for it. But these kinds of errors (forgetting one set of list brackets) are very common. They can easily be avoided if one writes in more modern Erlang rather than in Erlang of 10 years ago. In this particular case, you can use a spawn with a fun that makes the call to function format_temps/1 clear: spawn(fun () -> ?MODULE:format_temps([{moscow,{f,-40}}, {bangalore,{f,40}}, {wellington, {c,24}}]) end) An added bonus is that in this form you can choose to make a local call to the format_temp/1 function instead of a remote one. I.e. write it as follows: spawn(fun () -> format_temps([{moscow,{f,-40}}, {bangalore,{f,40}}, {wellington, {c,24}}]) end) Kostis From max.lapshin@REDACTED Wed Jun 22 15:11:23 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 22 Jun 2011 17:11:23 +0400 Subject: [erlang-questions] Trace/BPT trap on OS X when loading linked-in driver In-Reply-To: <7545A13D-D249-4EC2-9D67-C89DE9C5FEB9@semiocast.com> References: <7545A13D-D249-4EC2-9D67-C89DE9C5FEB9@semiocast.com> Message-ID: It is a very good idea, because erlang on mac is for development, so this is really a good idea to add two lines of code. From wsongcn@REDACTED Wed Jun 22 15:14:43 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Wed, 22 Jun 2011 21:14:43 +0800 Subject: [erlang-questions] Cowboy vs Misultin In-Reply-To: References: Message-ID: OK, the competition comes again. I finally wrote a client that performs decently. It's here . I tested my previous Cowboy first( I changed send_interval from 1000 to 900000 of line 64 of src/websocket_handler.erl in cowboy_examples ). Here is the version: *Cowboy_examples*: 343d002 Initial commit with a default and websocket handler. *Cowboy*: 5d69825 Add a max_connections transport option, Mon May 9 22:01:31 2011 The command line I use on the client is "./conn -c 10000 -n 10". It'll create 100K connections, 10K per IP to the server. It takes about 7 and half minutes to finish creating these connections. During the creation, server CPU usage is about 110%(dual core), client cpu usage is under 20%. Server memory usage(Cowboy residence memory) : Right after Cowboy starts: 525M After 100K connection established: 1842M After 5M packets passed: 2824M After 9M packets passed: 3231M After 87.7 M packets passed, 5746M [{total,4155518040}, {processes,1701318856}, {processes_used,1701299096}, {system,2454199184}, {atom,497457}, {atom_used,482279}, {binary,1787851232}, {code,4041219}, {ets,41231176}] There seems a memory leak. If I don't stop, the memory usage will keep going higher. 40% CPU, 1.5 MB/s traffic. *Latest Cowboy and Cowboy_examples* (same send_interval change). It takes about 7 and half minutes to finish creating 100K connections. During the creation, server CPU usage is about 110%(dual core), client cpu usage is under 20%. After 158.9M packets passed, 1464M residence memory used, 30% CPU, 1.5 MB/s traffic, . *Latest Misultin* from github, with patch (see attachment): It takes about 45s to finish creating 100K connections, during creation cpu usage is 120%. After creation, residence memory usage is 2684M. And it goes down slowly to 1416M after 40M packets are passed, cpu usage is about 50% Another thing worth to note is that when I hit CTRL-C on the client side, misultin will quickly destroy the connections and release memory and go back to normal. But Cowboy will consume almost all the cpu time and stay there for a long time (I didn't have the patience to measure how long, so just hit ctrl-c to quick erlang). Cheers Andy On Wed, Jun 1, 2011 at 8:26 PM, Andy W. Song wrote: > BTW, the latest version uses less memory too, about 3.2G for 200K > connections. > > > On Wed, Jun 1, 2011 at 8:23 PM, Andy W. Song wrote: > >> Missed one attachment. >> >> >> On Wed, Jun 1, 2011 at 8:21 PM, Andy W. Song wrote: >> >>> I tested out the latest Misultin with this change, CPU usage is about 2 >>> to 3 percent lower. >>> >>> With the attached Misultin package, I can still reproduce the large >>> packet problem. >>> >>> With latest version, Misultin doesn't report any error. But it doesn't >>> reply to the client either. Seems it just drops the packet. Another attached >>> file is the tcpdump result. >>> >>> I will try Mochiweb later just to compare. >>> >>> >>> On Wed, Jun 1, 2011 at 12:09 AM, Roberto Ostinelli wrote: >>> >>>> >>>> On May 31, 2011, at 12:54 PM, Andy W. Song wrote: >>>> >>>> Would you mind testing it using my C client? Search "->len" and give >>>> it a value between 1500 and 2000, hopefully you'll see the result I >>>> saw. >>>> >>>> Regards >>>> Andy >>>> >>>> >>>> Hi Andy, >>>> >>>> will do that. >>>> >>>> Meanwhile, I've added an undocumented option, {ws_no_header, true} so >>>> that headers are not duplicated in the record and should use less >>>> memory. plus, I've optimized binary code for websockets. >>>> >>>> can you please retry the test so we have a matter of comparison? >>>> >>>> the only thing you need to change is to add this option in misultin >>>> initialization, i.e.: >>>> >>>> % start misultin http server >>>> start(Port) -> >>>> misultin:start_link([ >>>> *{ws_no_header, true},* >>>> {port, Port}, {loop, fun(Req) -> handle_http(Req, Port) end}, >>>> {ws_loop, fun(Ws) -> handle_websocket(Ws) end}, {ws_autoexit, false} >>>> ]). >>>> >>>> please let me know... >>>> >>>> r. >>>> >>> >>> >>> >>> -- >>> --------------------------------------------------------------- >>> ???????????????????? >>> ???????????????????? >>> >>> >> >> >> -- >> --------------------------------------------------------------- >> ???????????????????? >> ???????????????????? >> >> > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > -- --------------------------------------------------------------- ???????????????????? ???????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: hibernate.patch Type: application/octet-stream Size: 4522 bytes Desc: not available URL: From paul.james.barry@REDACTED Wed Jun 22 15:19:45 2011 From: paul.james.barry@REDACTED (Paul Barry) Date: Wed, 22 Jun 2011 14:19:45 +0100 Subject: [erlang-questions] Books In-Reply-To: <20110622015124.GD49732@h216-235-12-167.host.egate.net> References: <20110622015124.GD49732@h216-235-12-167.host.egate.net> Message-ID: Have you considered donating them to the library associated with your local 3rd level college? That's what I do (but, then again, I work in such a place, so it's easy for me to do this). Paul. On 22 June 2011 02:51, Vance Shipley wrote: > Please excuse the noise ... > > I'm divesting myself of my paper based library and have at least > two books which may be of interest to folks here: > > ? ?Concurrent programming in ERLANG > ? ?Second Edition > ? ?Robert Virding, Claes Wikstrom, Mike Williams, Joe Armstrong (Editor) > ? ?ISBN: 013508301X > > ? ?Concurrency Theory > ? ?Calculi and Automata for Modelling Untimed and Timed Concurrent Systems > ? ?Howard Bowman, Rodolfo Gomez > ? ?ISBN: 1852338954 > > Free to a good home! > > -- > ? ? ? ?-Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Paul Barry, w: http://paulbarry.itcarlow.ie, e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From kennethstone@REDACTED Wed Jun 22 15:40:15 2011 From: kennethstone@REDACTED (Kenny Stone) Date: Wed, 22 Jun 2011 08:40:15 -0500 Subject: [erlang-questions] strftime for formatting dates and times In-Reply-To: <91D3B83A-9F79-4B15-80E9-D9589F799AE5@masklinn.net> References: <4E00B8A9.5040708@erlang-solutions.com> <4E00D71E.30003@erix.ericsson.se> <91D3B83A-9F79-4B15-80E9-D9589F799AE5@masklinn.net> Message-ID: I didn't know there was a standard for strftime implementations... I'd also never really considered it a DSL, although I can see that. I found myself writing a lot of the same code over and over again, so I wrote a simple library. Lots of languages have this method. It's handy, simple, usable, practical; all things I like in my methods. It uses established patterns that have worked for many years over many languages and many picky developers. Also things I like... :) 2011/6/22 Masklinn > On 2011-06-21, at 19:38 , Bj?rn-Egil Dahlberg wrote: > > On 2011-06-21 17:28, Adam Lindberg wrote: > >> Would love this being integrated into calendar! (Or appropriate module, > >> although I think calendar should be the one) > > > > I like this also and I probably would want something similar folded into > calendar or another module. > > > > * Abit of topic * > > > > This weekend a wrote something similar looking at python, ruby and perl > api:s for datetime, time and calendar functionality. Trying to get a feel > for what would be ok. However, most of those api:s are emulations of > standard C time conversions. gmtime, mktime, localtime etc. Horrible > horrible syntax in my opinion. On the other hand it is rather familiar. > > > Though it's mostly object-oriented (and quite verbose owing to being Java), > Joda-Time is generally considered a pretty nice API for an API handling just > about everything related to date and time. > > See also JSR-310, which was authored by Joda's original developer and is an > evolution of Joda (rather than a port or direct evolution) using lessons > learned from Joda. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wsongcn@REDACTED Wed Jun 22 15:49:30 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Wed, 22 Jun 2011 21:49:30 +0800 Subject: [erlang-questions] Call erlang:port_info/1 from shell Message-ID: Hi, If I run erlang:ports() from shell, it shows [#Port<0.1>,#Port<0.113>,#Port<0.118>,#Port<0.125>, #Port<0.513>,#Port<0.526>,#Port<0.201852>,#Port<0.202767>, #Port<0.202768>,#Port<0.202769>,#Port<0.202770>, #Port<0.202771>,#Port<0.202772>,#Port<0.202773>, #Port<0.202774>,#Port<0.202775>,#Port<0.202776>, #Port<0.202777>,#Port<0.202778>,#Port<0.202779>, #Port<0.202780>,#Port<0.202781>,#Port<0.202782>, #Port<0.202783>,#Port<0.202784>,#Port<0.202785>, #Port<0.202786>,#Port<0.202787>,#Port<0.202788>|...] Now I want to call erlang:port_info/1, but I don't know how to build the Port() parameter from say #Port<0.513>. Can anybody help? And BTW, where should I look for the document about this kind of thing? Thanks Andy -- --------------------------------------------------------------- ???????????????????? ???????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From allen.kim@REDACTED Wed Jun 22 16:00:27 2011 From: allen.kim@REDACTED (Allen Kim) Date: Wed, 22 Jun 2011 09:00:27 -0500 Subject: [erlang-questions] Call erlang:port_info/1 from shell In-Reply-To: Message-ID: [ erlang:port_info(Port) || Port <- erlang:ports()]. You can also use lists:foreach instead. From: "Andy W. Song" > Date: Wed, 22 Jun 2011 08:49:30 -0500 To: erlang-questions > Subject: [erlang-questions] Call erlang:port_info/1 from shell Hi, If I run erlang:ports() from shell, it shows [#Port<0.1>,#Port<0.113>,#Port<0.118>,#Port<0.125>, #Port<0.513>,#Port<0.526>,#Port<0.201852>,#Port<0.202767>, #Port<0.202768>,#Port<0.202769>,#Port<0.202770>, #Port<0.202771>,#Port<0.202772>,#Port<0.202773>, #Port<0.202774>,#Port<0.202775>,#Port<0.202776>, #Port<0.202777>,#Port<0.202778>,#Port<0.202779>, #Port<0.202780>,#Port<0.202781>,#Port<0.202782>, #Port<0.202783>,#Port<0.202784>,#Port<0.202785>, #Port<0.202786>,#Port<0.202787>,#Port<0.202788>|...] Now I want to call erlang:port_info/1, but I don't know how to build the Port() parameter from say #Port<0.513>. Can anybody help? And BTW, where should I look for the document about this kind of thing? Thanks Andy -- --------------------------------------------------------------- ???????????????????? ???????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jun 22 16:06:22 2011 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 22 Jun 2011 16:06:22 +0200 Subject: [erlang-questions] Cowboy vs Misultin In-Reply-To: References: Message-ID: <4E01F6DE.3040405@dev-extend.eu> Hello, Did you fix the many issues in your cowboy application code I reported in my previous emails? I see you removed the receive after from misultin, did you remove it from cowboy too? You could very well just be measuring Erlang's timer modules bad performance there. Also I see you're not using cowboy_http_req:compact and the hibernate option for cowboy that is doing it properly (unlike your patch). And again, you should probably come to IRC and ask for advice from the other people benchmarking it. Cheers. On 06/22/2011 03:14 PM, Andy W. Song wrote: > OK, the competition comes again. I finally wrote a client that performs > decently. It's here . > > I tested my previous Cowboy first( I changed send_interval from 1000 to > 900000 of line 64 of src/websocket_handler.erl in cowboy_examples ). > Here is the version: > *Cowboy_examples*: 343d002 Initial commit with a default and websocket > handler. > *Cowboy*: 5d69825 Add a max_connections transport option, Mon May 9 > 22:01:31 2011 > > The command line I use on the client is "./conn -c 10000 -n 10". It'll > create 100K connections, 10K per IP to the server. It takes about 7 and > half minutes to finish creating these connections. During the creation, > server CPU usage is about 110%(dual core), client cpu usage is under 20%. > > Server memory usage(Cowboy residence memory) : > Right after Cowboy starts: 525M > After 100K connection established: 1842M > After 5M packets passed: 2824M > After 9M packets passed: 3231M > After 87.7 M packets passed, 5746M > [{total,4155518040 }, > {processes,1701318856}, > {processes_used,1701299096}, > {system,2454199184}, > {atom,497457}, > {atom_used,482279}, > {binary,1787851232}, > {code,4041219}, > {ets,41231176}] > There seems a memory leak. If I don't stop, the memory usage will keep > going higher. 40% CPU, 1.5 MB/s traffic. > > *Latest Cowboy and Cowboy_examples* (same send_interval change). It > takes about 7 and half minutes to finish creating 100K connections. > During the creation, server CPU usage is about 110%(dual core), client > cpu usage is under 20%. After 158.9M packets passed, 1464M residence > memory used, 30% CPU, 1.5 MB/s traffic, . > > *Latest Misultin* from github, with patch (see attachment): > It takes about 45s to finish creating 100K connections, during creation > cpu usage is 120%. After creation, residence memory usage is 2684M. And > it goes down slowly to 1416M after 40M packets are passed, cpu usage is > about 50% > > Another thing worth to note is that when I hit CTRL-C on the client > side, misultin will quickly destroy the connections and release memory > and go back to normal. But Cowboy will consume almost all the cpu time > and stay there for a long time (I didn't have the patience to measure > how long, so just hit ctrl-c to quick erlang). > > Cheers > Andy > > > On Wed, Jun 1, 2011 at 8:26 PM, Andy W. Song > wrote: > > BTW, the latest version uses less memory too, about 3.2G for 200K > connections. > > > On Wed, Jun 1, 2011 at 8:23 PM, Andy W. Song > wrote: > > Missed one attachment. > > > On Wed, Jun 1, 2011 at 8:21 PM, Andy W. Song > wrote: > > I tested out the latest Misultin with this change, CPU usage > is about 2 to 3 percent lower. > > With the attached Misultin package, I can still reproduce > the large packet problem. > > With latest version, Misultin doesn't report any error. But > it doesn't reply to the client either. Seems it just drops > the packet. Another attached file is the tcpdump result. > > I will try Mochiweb later just to compare. > > > On Wed, Jun 1, 2011 at 12:09 AM, Roberto Ostinelli > > wrote: > > > On May 31, 2011, at 12:54 PM, Andy W. Song wrote: > >> Would you mind testing it using my C client? Search >> "->len" and give >> it a value between 1500 and 2000, hopefully you'll see >> the result I >> saw. >> >> Regards >> Andy > > Hi Andy, > > will do that. > > Meanwhile, I've added an undocumented option, > {ws_no_header, true} so that headers are not duplicated > in the record and should use less memory. plus, I've > optimized binary code for websockets. > > can you please retry the test so we have a matter of > comparison? > > the only thing you need to change is to add this option > in misultin initialization, i.e.: > > % start misultin http server > start(Port) -> > misultin:start_link([ > *{ws_no_header, true},* > {port, Port}, {loop, fun(Req) -> handle_http(Req, Port) > end}, > {ws_loop, fun(Ws) -> handle_websocket(Ws) end}, > {ws_autoexit, false} > ]). > > please let me know... > > r. > > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Dev:Extend From essen@REDACTED Wed Jun 22 16:11:47 2011 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 22 Jun 2011 16:11:47 +0200 Subject: [erlang-questions] Cowboy vs Misultin In-Reply-To: References: Message-ID: <4E01F823.6020605@dev-extend.eu> Sorry for the double post, but in your attached misultin patch: + erlang:ibernate(?MODULE, ws_loop, [ServerRef, Socket, L, WsHandleLoopPid, SocketMode, WsAutoExit]). you meant erlang:hibernate, right? On 06/22/2011 03:14 PM, Andy W. Song wrote: > OK, the competition comes again. I finally wrote a client that performs > decently. It's here . > > I tested my previous Cowboy first( I changed send_interval from 1000 to > 900000 of line 64 of src/websocket_handler.erl in cowboy_examples ). > Here is the version: > *Cowboy_examples*: 343d002 Initial commit with a default and websocket > handler. > *Cowboy*: 5d69825 Add a max_connections transport option, Mon May 9 > 22:01:31 2011 > > The command line I use on the client is "./conn -c 10000 -n 10". It'll > create 100K connections, 10K per IP to the server. It takes about 7 and > half minutes to finish creating these connections. During the creation, > server CPU usage is about 110%(dual core), client cpu usage is under 20%. > > Server memory usage(Cowboy residence memory) : > Right after Cowboy starts: 525M > After 100K connection established: 1842M > After 5M packets passed: 2824M > After 9M packets passed: 3231M > After 87.7 M packets passed, 5746M > [{total,4155518040 }, > {processes,1701318856}, > {processes_used,1701299096}, > {system,2454199184}, > {atom,497457}, > {atom_used,482279}, > {binary,1787851232}, > {code,4041219}, > {ets,41231176}] > There seems a memory leak. If I don't stop, the memory usage will keep > going higher. 40% CPU, 1.5 MB/s traffic. > > *Latest Cowboy and Cowboy_examples* (same send_interval change). It > takes about 7 and half minutes to finish creating 100K connections. > During the creation, server CPU usage is about 110%(dual core), client > cpu usage is under 20%. After 158.9M packets passed, 1464M residence > memory used, 30% CPU, 1.5 MB/s traffic, . > > *Latest Misultin* from github, with patch (see attachment): > It takes about 45s to finish creating 100K connections, during creation > cpu usage is 120%. After creation, residence memory usage is 2684M. And > it goes down slowly to 1416M after 40M packets are passed, cpu usage is > about 50% > > Another thing worth to note is that when I hit CTRL-C on the client > side, misultin will quickly destroy the connections and release memory > and go back to normal. But Cowboy will consume almost all the cpu time > and stay there for a long time (I didn't have the patience to measure > how long, so just hit ctrl-c to quick erlang). > > Cheers > Andy > > > On Wed, Jun 1, 2011 at 8:26 PM, Andy W. Song > wrote: > > BTW, the latest version uses less memory too, about 3.2G for 200K > connections. > > > On Wed, Jun 1, 2011 at 8:23 PM, Andy W. Song > wrote: > > Missed one attachment. > > > On Wed, Jun 1, 2011 at 8:21 PM, Andy W. Song > wrote: > > I tested out the latest Misultin with this change, CPU usage > is about 2 to 3 percent lower. > > With the attached Misultin package, I can still reproduce > the large packet problem. > > With latest version, Misultin doesn't report any error. But > it doesn't reply to the client either. Seems it just drops > the packet. Another attached file is the tcpdump result. > > I will try Mochiweb later just to compare. > > > On Wed, Jun 1, 2011 at 12:09 AM, Roberto Ostinelli > > wrote: > > > On May 31, 2011, at 12:54 PM, Andy W. Song wrote: > >> Would you mind testing it using my C client? Search >> "->len" and give >> it a value between 1500 and 2000, hopefully you'll see >> the result I >> saw. >> >> Regards >> Andy > > Hi Andy, > > will do that. > > Meanwhile, I've added an undocumented option, > {ws_no_header, true} so that headers are not duplicated > in the record and should use less memory. plus, I've > optimized binary code for websockets. > > can you please retry the test so we have a matter of > comparison? > > the only thing you need to change is to add this option > in misultin initialization, i.e.: > > % start misultin http server > start(Port) -> > misultin:start_link([ > *{ws_no_header, true},* > {port, Port}, {loop, fun(Req) -> handle_http(Req, Port) > end}, > {ws_loop, fun(Ws) -> handle_websocket(Ws) end}, > {ws_autoexit, false} > ]). > > please let me know... > > r. > > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Dev:Extend From allen.kim@REDACTED Wed Jun 22 16:34:50 2011 From: allen.kim@REDACTED (Allen Kim) Date: Wed, 22 Jun 2011 09:34:50 -0500 Subject: [erlang-questions] Call erlang:port_info/1 from shell In-Reply-To: Message-ID: I believe c:pid/3 is given to you for convenience of debugging. I don't think there is one that converts list(string) to port, and I also think we do not convert string to port in code. since it is shell environment you want to run on, I don't think performance really matters. If so, you can use a simple function like this. PortInfo = fun(Num) -> [MyPort] =lists:filter(fun(Port) -> {id,PortNum} = erlang:port_info(Port,id), PortNum==Num end, erlang:ports()), erlang:port_info(MyPort) end. PortInfo(267). Allen From: "Andy W. Song" > Date: Wed, 22 Jun 2011 09:07:18 -0500 To: Allen Kim > Subject: Re: [erlang-questions] Call erlang:port_info/1 from shell This is not convenient. I can use pid(0,22222,0) to get a pid from shell. I suppose there should be a similar way for ports. If have thousands of ports, which I do, and I just want to examine one of them from shell, I can't walk through them. 2011/6/22 Allen Kim > [ erlang:port_info(Port) || Port <- erlang:ports()]. You can also use lists:foreach instead. From: "Andy W. Song" > Date: Wed, 22 Jun 2011 08:49:30 -0500 To: erlang-questions > Subject: [erlang-questions] Call erlang:port_info/1 from shell Hi, If I run erlang:ports() from shell, it shows [#Port<0.1>,#Port<0.113>,#Port<0.118>,#Port<0.125>, #Port<0.513>,#Port<0.526>,#Port<0.201852>,#Port<0.202767>, #Port<0.202768>,#Port<0.202769>,#Port<0.202770>, #Port<0.202771>,#Port<0.202772>,#Port<0.202773>, #Port<0.202774>,#Port<0.202775>,#Port<0.202776>, #Port<0.202777>,#Port<0.202778>,#Port<0.202779>, #Port<0.202780>,#Port<0.202781>,#Port<0.202782>, #Port<0.202783>,#Port<0.202784>,#Port<0.202785>, #Port<0.202786>,#Port<0.202787>,#Port<0.202788>|...] Now I want to call erlang:port_info/1, but I don't know how to build the Port() parameter from say #Port<0.513>. Can anybody help? And BTW, where should I look for the document about this kind of thing? Thanks Andy -- --------------------------------------------------------------- ???????????????????? ???????????????????? -- --------------------------------------------------------------- ???????????????????? ???????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From joseph.stewart@REDACTED Wed Jun 22 16:48:06 2011 From: joseph.stewart@REDACTED (Joseph Stewart) Date: Wed, 22 Jun 2011 10:48:06 -0400 Subject: [erlang-questions] Books In-Reply-To: <20110622015124.GD49732@h216-235-12-167.host.egate.net> References: <20110622015124.GD49732@h216-235-12-167.host.egate.net> Message-ID: I'd REALLY love the " Concurrent programming in ERLANG" book... I'll pay shipping! -joe On Tue, Jun 21, 2011 at 9:51 PM, Vance Shipley wrote: > Please excuse the noise ... > > I'm divesting myself of my paper based library and have at least > two books which may be of interest to folks here: > > Concurrent programming in ERLANG > Second Edition > Robert Virding, Claes Wikstrom, Mike Williams, Joe Armstrong (Editor) > ISBN: 013508301X > > Concurrency Theory > Calculi and Automata for Modelling Untimed and Timed Concurrent Systems > Howard Bowman, Rodolfo Gomez > ISBN: 1852338954 > > Free to a good home! > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wsongcn@REDACTED Wed Jun 22 17:08:24 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Wed, 22 Jun 2011 23:08:24 +0800 Subject: [erlang-questions] Cowboy vs Misultin In-Reply-To: <4E01FD6F.1000607@dev-extend.eu> References: <4E01F6DE.3040405@dev-extend.eu> <4E01FD6F.1000607@dev-extend.eu> Message-ID: Thanks for the link. I removed the send_interval call and retested but it didn't make much of a difference. It still takes about 7 and half minutes to create 100K connections. During creation I can see several time outs from client side. And the creation speed gets slower when the number of connections gets higher. Although the average CPU usage doesn't change much. After creation, with 1.5MB/s traffic, the CPU usage is still about the same as before which is 30%. Cheers. Andy On Wed, Jun 22, 2011 at 10:34 PM, Lo?c Hoguin wrote: > Memory usage sounds a bit high for cowboy_http_req:compact. But maybe > not. Good point though it's already in the examples just like hibernate. > My bad. > > On the other hand if you have a timer per connection it's definitely not > going to scale, timers are very slow so having 100k timers isn't going > to work well. Remove the send_interval and it should work a lot better. > > For future reference: > http://www.erlang.org/doc/efficiency_guide/commoncaveats.html#id58959 > > Waiting for yet another comparison, have fun. :) > > On 06/22/2011 04:27 PM, Andy W. Song wrote: > > I actually didn't patch. I want to keep my change as small as possible. > > I will remove that send_interval and test it again. Sorry I didn't pay > > much attention about this. > > > > BTW, how can you tell I used or did not use cowboy_http_req:compact? And > > I see hibernate is already the default behavior, isn't it? > > > > Thanks > > Andy > > > > On Wed, Jun 22, 2011 at 10:06 PM, Lo?c Hoguin > > wrote: > > > > Hello, > > > > Did you fix the many issues in your cowboy application code I > reported > > in my previous emails? I see you removed the receive after from > > misultin, did you remove it from cowboy too? You could very well just > be > > measuring Erlang's timer modules bad performance there. > > > > Also I see you're not using cowboy_http_req:compact and the hibernate > > option for cowboy that is doing it properly (unlike your patch). > > > > And again, you should probably come to IRC and ask for advice from > the > > other people benchmarking it. > > > > Cheers. > > > > On 06/22/2011 03:14 PM, Andy W. Song wrote: > > > OK, the competition comes again. I finally wrote a client that > > performs > > > decently. It's here > > . > > > > > > I tested my previous Cowboy first( I changed send_interval from > > 1000 to > > > 900000 of line 64 of src/websocket_handler.erl in cowboy_examples > ). > > > Here is the version: > > > *Cowboy_examples*: 343d002 Initial commit with a default and > websocket > > > handler. > > > *Cowboy*: 5d69825 Add a max_connections transport option, Mon May 9 > > > 22:01:31 2011 > > > > > > The command line I use on the client is "./conn -c 10000 -n 10". > It'll > > > create 100K connections, 10K per IP to the server. It takes about > > 7 and > > > half minutes to finish creating these connections. During the > > creation, > > > server CPU usage is about 110%(dual core), client cpu usage is > > under 20%. > > > > > > Server memory usage(Cowboy residence memory) : > > > Right after Cowboy starts: 525M > > > After 100K connection established: 1842M > > > After 5M packets passed: 2824M > > > After 9M packets passed: 3231M > > > After 87.7 M packets passed, 5746M > > > [{total,4155518040 > >}, > > > {processes,1701318856}, > > > {processes_used,1701299096}, > > > {system,2454199184}, > > > {atom,497457}, > > > {atom_used,482279}, > > > {binary,1787851232}, > > > {code,4041219}, > > > {ets,41231176}] > > > There seems a memory leak. If I don't stop, the memory usage will > keep > > > going higher. 40% CPU, 1.5 MB/s traffic. > > > > > > *Latest Cowboy and Cowboy_examples* (same send_interval change). It > > > takes about 7 and half minutes to finish creating 100K connections. > > > During the creation, server CPU usage is about 110%(dual core), > client > > > cpu usage is under 20%. After 158.9M packets passed, 1464M > residence > > > memory used, 30% CPU, 1.5 MB/s traffic, . > > > > > > *Latest Misultin* from github, with patch (see attachment): > > > It takes about 45s to finish creating 100K connections, during > > creation > > > cpu usage is 120%. After creation, residence memory usage is > > 2684M. And > > > it goes down slowly to 1416M after 40M packets are passed, cpu > > usage is > > > about 50% > > > > > > Another thing worth to note is that when I hit CTRL-C on the client > > > side, misultin will quickly destroy the connections and release > memory > > > and go back to normal. But Cowboy will consume almost all the cpu > time > > > and stay there for a long time (I didn't have the patience to > measure > > > how long, so just hit ctrl-c to quick erlang). > > > > > > Cheers > > > Andy > > > > > > > > > On Wed, Jun 1, 2011 at 8:26 PM, Andy W. Song > > > > >> wrote: > > > > > > BTW, the latest version uses less memory too, about 3.2G for > 200K > > > connections. > > > > > > > > > On Wed, Jun 1, 2011 at 8:23 PM, Andy W. Song > > > > > >> wrote: > > > > > > Missed one attachment. > > > > > > > > > On Wed, Jun 1, 2011 at 8:21 PM, Andy W. Song > > > > > >> > wrote: > > > > > > I tested out the latest Misultin with this change, CPU > > usage > > > is about 2 to 3 percent lower. > > > > > > With the attached Misultin package, I can still > reproduce > > > the large packet problem. > > > > > > With latest version, Misultin doesn't report any > > error. But > > > it doesn't reply to the client either. Seems it just > drops > > > the packet. Another attached file is the tcpdump > result. > > > > > > I will try Mochiweb later just to compare. > > > > > > > > > On Wed, Jun 1, 2011 at 12:09 AM, Roberto Ostinelli > > > > > >> wrote: > > > > > > > > > On May 31, 2011, at 12:54 PM, Andy W. Song wrote: > > > > > >> Would you mind testing it using my C client? > Search > > >> "->len" and give > > >> it a value between 1500 and 2000, hopefully > > you'll see > > >> the result I > > >> saw. > > >> > > >> Regards > > >> Andy > > > > > > Hi Andy, > > > > > > will do that. > > > > > > Meanwhile, I've added an undocumented option, > > > {ws_no_header, true} so that headers are not > > duplicated > > > in the record and should use less memory. plus, > I've > > > optimized binary code for websockets. > > > > > > can you please retry the test so we have a matter > of > > > comparison? > > > > > > the only thing you need to change is to add this > > option > > > in misultin initialization, i.e.: > > > > > > % start misultin http server > > > start(Port) -> > > > misultin:start_link([ > > > *{ws_no_header, true},* > > > {port, Port}, {loop, fun(Req) -> handle_http(Req, > > Port) > > > end}, > > > {ws_loop, fun(Ws) -> handle_websocket(Ws) end}, > > > {ws_autoexit, false} > > > ]). > > > > > > please let me know... > > > > > > r. > > > > > > > > > > > > > > > -- > > > > > --------------------------------------------------------------- > > > ???????????????????? > > > ???????????????????? > > > > > > > > > > > > > > > -- > > > > > --------------------------------------------------------------- > > > ???????????????????? > > > ???????????????????? > > > > > > > > > > > > > > > -- > > > --------------------------------------------------------------- > > > ???????????????????? > > > ???????????????????? > > > > > > > > > > > > > > > -- > > > --------------------------------------------------------------- > > > ???????????????????? > > > ???????????????????? > > > > > > > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > > Lo?c Hoguin > > Dev:Extend > > > > > > > > > > -- > > --------------------------------------------------------------- > > ???????????????????? > > ???????????????????? > > > > > -- > Lo?c Hoguin > Dev:Extend > -- --------------------------------------------------------------- ???????????????????? ???????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jun 22 17:24:58 2011 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 22 Jun 2011 17:24:58 +0200 Subject: [erlang-questions] Cowboy vs Misultin In-Reply-To: References: <4E01F6DE.3040405@dev-extend.eu> <4E01FD6F.1000607@dev-extend.eu> Message-ID: <4E02094A.4080804@dev-extend.eu> Odd. I'll try to run the client you wrote to reproduce whenever I receive the hardware I'm waiting for. Not sure what could do this though. Do you have an idea of what's the number of connections before it starts slowing down? On 06/22/2011 05:08 PM, Andy W. Song wrote: > Thanks for the link. I removed the send_interval call and retested but > it didn't make much of a difference. It still takes about 7 and half > minutes to create 100K connections. During creation I can see several > time outs from client side. And the creation speed gets slower when the > number of connections gets higher. Although the average CPU usage > doesn't change much. > > After creation, with 1.5MB/s traffic, the CPU usage is still about the > same as before which is 30%. > > Cheers. > Andy > > On Wed, Jun 22, 2011 at 10:34 PM, Lo?c Hoguin > wrote: > > Memory usage sounds a bit high for cowboy_http_req:compact. But maybe > not. Good point though it's already in the examples just like hibernate. > My bad. > > On the other hand if you have a timer per connection it's definitely not > going to scale, timers are very slow so having 100k timers isn't going > to work well. Remove the send_interval and it should work a lot better. > > For future reference: > http://www.erlang.org/doc/efficiency_guide/commoncaveats.html#id58959 > > Waiting for yet another comparison, have fun. :) > > On 06/22/2011 04:27 PM, Andy W. Song wrote: > > I actually didn't patch. I want to keep my change as small as > possible. > > I will remove that send_interval and test it again. Sorry I didn't pay > > much attention about this. > > > > BTW, how can you tell I used or did not use > cowboy_http_req:compact? And > > I see hibernate is already the default behavior, isn't it? > > > > Thanks > > Andy > > > > On Wed, Jun 22, 2011 at 10:06 PM, Lo?c Hoguin > > >> wrote: > > > > Hello, > > > > Did you fix the many issues in your cowboy application code I > reported > > in my previous emails? I see you removed the receive after from > > misultin, did you remove it from cowboy too? You could very > well just be > > measuring Erlang's timer modules bad performance there. > > > > Also I see you're not using cowboy_http_req:compact and the > hibernate > > option for cowboy that is doing it properly (unlike your patch). > > > > And again, you should probably come to IRC and ask for advice > from the > > other people benchmarking it. > > > > Cheers. > > > > On 06/22/2011 03:14 PM, Andy W. Song wrote: > > > OK, the competition comes again. I finally wrote a client that > > performs > > > decently. It's here > > . > > > > > > I tested my previous Cowboy first( I changed send_interval from > > 1000 to > > > 900000 of line 64 of src/websocket_handler.erl in > cowboy_examples ). > > > Here is the version: > > > *Cowboy_examples*: 343d002 Initial commit with a default and > websocket > > > handler. > > > *Cowboy*: 5d69825 Add a max_connections transport option, > Mon May 9 > > > 22:01:31 2011 > > > > > > The command line I use on the client is "./conn -c 10000 -n > 10". It'll > > > create 100K connections, 10K per IP to the server. It takes > about > > 7 and > > > half minutes to finish creating these connections. During the > > creation, > > > server CPU usage is about 110%(dual core), client cpu usage is > > under 20%. > > > > > > Server memory usage(Cowboy residence memory) : > > > Right after Cowboy starts: 525M > > > After 100K connection established: 1842M > > > After 5M packets passed: 2824M > > > After 9M packets passed: 3231M > > > After 87.7 M packets passed, 5746M > > > [{total,4155518040 > > > >>}, > > > {processes,1701318856}, > > > {processes_used,1701299096}, > > > {system,2454199184}, > > > {atom,497457}, > > > {atom_used,482279}, > > > {binary,1787851232}, > > > {code,4041219}, > > > {ets,41231176}] > > > There seems a memory leak. If I don't stop, the memory usage > will keep > > > going higher. 40% CPU, 1.5 MB/s traffic. > > > > > > *Latest Cowboy and Cowboy_examples* (same send_interval > change). It > > > takes about 7 and half minutes to finish creating 100K > connections. > > > During the creation, server CPU usage is about 110%(dual > core), client > > > cpu usage is under 20%. After 158.9M packets passed, 1464M > residence > > > memory used, 30% CPU, 1.5 MB/s traffic, . > > > > > > *Latest Misultin* from github, with patch (see attachment): > > > It takes about 45s to finish creating 100K connections, during > > creation > > > cpu usage is 120%. After creation, residence memory usage is > > 2684M. And > > > it goes down slowly to 1416M after 40M packets are passed, cpu > > usage is > > > about 50% > > > > > > Another thing worth to note is that when I hit CTRL-C on the > client > > > side, misultin will quickly destroy the connections and > release memory > > > and go back to normal. But Cowboy will consume almost all > the cpu time > > > and stay there for a long time (I didn't have the patience > to measure > > > how long, so just hit ctrl-c to quick erlang). > > > > > > Cheers > > > Andy > > > > > > > > > On Wed, Jun 1, 2011 at 8:26 PM, Andy W. Song > > > > > > > > >>> wrote: > > > > > > BTW, the latest version uses less memory too, about 3.2G > for 200K > > > connections. > > > > > > > > > On Wed, Jun 1, 2011 at 8:23 PM, Andy W. Song > > > > > > > > >>> wrote: > > > > > > Missed one attachment. > > > > > > > > > On Wed, Jun 1, 2011 at 8:21 PM, Andy W. Song > > > > > > > > >>> wrote: > > > > > > I tested out the latest Misultin with this > change, CPU > > usage > > > is about 2 to 3 percent lower. > > > > > > With the attached Misultin package, I can still > reproduce > > > the large packet problem. > > > > > > With latest version, Misultin doesn't report any > > error. But > > > it doesn't reply to the client either. Seems it > just drops > > > the packet. Another attached file is the tcpdump > result. > > > > > > I will try Mochiweb later just to compare. > > > > > > > > > On Wed, Jun 1, 2011 at 12:09 AM, Roberto Ostinelli > > > > > > > >>> wrote: > > > > > > > > > On May 31, 2011, at 12:54 PM, Andy W. Song > wrote: > > > > > >> Would you mind testing it using my C > client? Search > > >> "->len" and give > > >> it a value between 1500 and 2000, hopefully > > you'll see > > >> the result I > > >> saw. > > >> > > >> Regards > > >> Andy > > > > > > Hi Andy, > > > > > > will do that. > > > > > > Meanwhile, I've added an undocumented option, > > > {ws_no_header, true} so that headers are not > > duplicated > > > in the record and should use less memory. > plus, I've > > > optimized binary code for websockets. > > > > > > can you please retry the test so we have a > matter of > > > comparison? > > > > > > the only thing you need to change is to add this > > option > > > in misultin initialization, i.e.: > > > > > > % start misultin http server > > > start(Port) -> > > > misultin:start_link([ > > > *{ws_no_header, true},* > > > {port, Port}, {loop, fun(Req) -> > handle_http(Req, > > Port) > > > end}, > > > {ws_loop, fun(Ws) -> handle_websocket(Ws) end}, > > > {ws_autoexit, false} > > > ]). > > > > > > please let me know... > > > > > > r. > > > > > > > > > > > > > > > -- > > > > > --------------------------------------------------------------- > > > ???????????????????? > > > ???????????????????? > > > > > > > > > > > > > > > -- > > > > > --------------------------------------------------------------- > > > ???????????????????? > > > ???????????????????? > > > > > > > > > > > > > > > -- > > > > --------------------------------------------------------------- > > > ???????????????????? > > > ???????????????????? > > > > > > > > > > > > > > > -- > > > --------------------------------------------------------------- > > > ???????????????????? > > > ???????????????????? > > > > > > > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > > Lo?c Hoguin > > Dev:Extend > > > > > > > > > > -- > > --------------------------------------------------------------- > > ???????????????????? > > ???????????????????? > > > > > -- > Lo?c Hoguin > Dev:Extend > > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > -- Lo?c Hoguin Dev:Extend From wsongcn@REDACTED Wed Jun 22 17:28:22 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Wed, 22 Jun 2011 23:28:22 +0800 Subject: [erlang-questions] Call erlang:port_info/1 from shell In-Reply-To: References: Message-ID: Thanks, it works very well. When I call PortInfo(868) I get: [{name,"tcp_inet"}, {links,[<0.234.0>]}, {id,868}, {connected,<0.234.0>}, {input,0}, {output,12788}] So here comes my next question: how can I find out the socket information associated to this port, such as ip and port? Thanks Andy 2011/6/22 Allen Kim > I believe c:pid/3 is given to you for convenience of debugging. > I don't think there is one that converts list(string) to port, and I also > think we do not convert string to port in code. > > since it is shell environment you want to run on, I don't think > performance really matters. > If so, you can use a simple function like this. > > PortInfo = fun(Num) -> > [MyPort] =lists:filter(fun(Port) -> {id,PortNum} = > erlang:port_info(Port,id), PortNum==Num end, erlang:ports()), > erlang:port_info(MyPort) > end. > > PortInfo(267). > > Allen > > From: "Andy W. Song" > Date: Wed, 22 Jun 2011 09:07:18 -0500 > To: Allen Kim > Subject: Re: [erlang-questions] Call erlang:port_info/1 from shell > > This is not convenient. I can use pid(0,22222,0) to get a pid from shell. I > suppose there should be a similar way for ports. If have thousands of ports, > which I do, and I just want to examine one of them from shell, I can't walk > through them. > > 2011/6/22 Allen Kim > >> [ erlang:port_info(Port) || Port <- erlang:ports()]. >> >> You can also use lists:foreach instead. >> >> From: "Andy W. Song" >> Date: Wed, 22 Jun 2011 08:49:30 -0500 >> To: erlang-questions >> Subject: [erlang-questions] Call erlang:port_info/1 from shell >> >> Hi, >> >> If I run erlang:ports() from shell, it shows >> >> [#Port<0.1>,#Port<0.113>,#Port<0.118>,#Port<0.125>, >> #Port<0.513>,#Port<0.526>,#Port<0.201852>,#Port<0.202767>, >> #Port<0.202768>,#Port<0.202769>,#Port<0.202770>, >> #Port<0.202771>,#Port<0.202772>,#Port<0.202773>, >> #Port<0.202774>,#Port<0.202775>,#Port<0.202776>, >> #Port<0.202777>,#Port<0.202778>,#Port<0.202779>, >> #Port<0.202780>,#Port<0.202781>,#Port<0.202782>, >> #Port<0.202783>,#Port<0.202784>,#Port<0.202785>, >> #Port<0.202786>,#Port<0.202787>,#Port<0.202788>|...] >> >> Now I want to call erlang:port_info/1, but I don't know how to build the >> Port() parameter from say #Port<0.513>. Can anybody help? And BTW, where >> should I look for the document about this kind of thing? >> >> Thanks >> Andy >> >> -- >> --------------------------------------------------------------- >> ???????????????????? >> ???????????????????? >> >> > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > -- --------------------------------------------------------------- ???????????????????? ???????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From wsongcn@REDACTED Wed Jun 22 17:30:01 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Wed, 22 Jun 2011 23:30:01 +0800 Subject: [erlang-questions] Cowboy vs Misultin In-Reply-To: <4E02094A.4080804@dev-extend.eu> References: <4E01F6DE.3040405@dev-extend.eu> <4E01FD6F.1000607@dev-extend.eu> <4E02094A.4080804@dev-extend.eu> Message-ID: I can roughly feel it after about 15K~20K. On Wed, Jun 22, 2011 at 11:24 PM, Lo?c Hoguin wrote: > Odd. > > I'll try to run the client you wrote to reproduce whenever I receive the > hardware I'm waiting for. Not sure what could do this though. Do you > have an idea of what's the number of connections before it starts > slowing down? > > On 06/22/2011 05:08 PM, Andy W. Song wrote: > > Thanks for the link. I removed the send_interval call and retested but > > it didn't make much of a difference. It still takes about 7 and half > > minutes to create 100K connections. During creation I can see several > > time outs from client side. And the creation speed gets slower when the > > number of connections gets higher. Although the average CPU usage > > doesn't change much. > > > > After creation, with 1.5MB/s traffic, the CPU usage is still about the > > same as before which is 30%. > > > > Cheers. > > Andy > > > > On Wed, Jun 22, 2011 at 10:34 PM, Lo?c Hoguin > > wrote: > > > > Memory usage sounds a bit high for cowboy_http_req:compact. But maybe > > not. Good point though it's already in the examples just like > hibernate. > > My bad. > > > > On the other hand if you have a timer per connection it's definitely > not > > going to scale, timers are very slow so having 100k timers isn't > going > > to work well. Remove the send_interval and it should work a lot > better. > > > > For future reference: > > > http://www.erlang.org/doc/efficiency_guide/commoncaveats.html#id58959 > > > > Waiting for yet another comparison, have fun. :) > > > > On 06/22/2011 04:27 PM, Andy W. Song wrote: > > > I actually didn't patch. I want to keep my change as small as > > possible. > > > I will remove that send_interval and test it again. Sorry I didn't > pay > > > much attention about this. > > > > > > BTW, how can you tell I used or did not use > > cowboy_http_req:compact? And > > > I see hibernate is already the default behavior, isn't it? > > > > > > Thanks > > > Andy > > > > > > On Wed, Jun 22, 2011 at 10:06 PM, Lo?c Hoguin > > > > >> wrote: > > > > > > Hello, > > > > > > Did you fix the many issues in your cowboy application code I > > reported > > > in my previous emails? I see you removed the receive after from > > > misultin, did you remove it from cowboy too? You could very > > well just be > > > measuring Erlang's timer modules bad performance there. > > > > > > Also I see you're not using cowboy_http_req:compact and the > > hibernate > > > option for cowboy that is doing it properly (unlike your > patch). > > > > > > And again, you should probably come to IRC and ask for advice > > from the > > > other people benchmarking it. > > > > > > Cheers. > > > > > > On 06/22/2011 03:14 PM, Andy W. Song wrote: > > > > OK, the competition comes again. I finally wrote a client > that > > > performs > > > > decently. It's here > > > . > > > > > > > > I tested my previous Cowboy first( I changed send_interval > from > > > 1000 to > > > > 900000 of line 64 of src/websocket_handler.erl in > > cowboy_examples ). > > > > Here is the version: > > > > *Cowboy_examples*: 343d002 Initial commit with a default and > > websocket > > > > handler. > > > > *Cowboy*: 5d69825 Add a max_connections transport option, > > Mon May 9 > > > > 22:01:31 2011 > > > > > > > > The command line I use on the client is "./conn -c 10000 -n > > 10". It'll > > > > create 100K connections, 10K per IP to the server. It takes > > about > > > 7 and > > > > half minutes to finish creating these connections. During the > > > creation, > > > > server CPU usage is about 110%(dual core), client cpu usage > is > > > under 20%. > > > > > > > > Server memory usage(Cowboy residence memory) : > > > > Right after Cowboy starts: 525M > > > > After 100K connection established: 1842M > > > > After 5M packets passed: 2824M > > > > After 9M packets passed: 3231M > > > > After 87.7 M packets passed, 5746M > > > > [{total,4155518040 > > > > > >>}, > > > > {processes,1701318856}, > > > > {processes_used,1701299096}, > > > > {system,2454199184}, > > > > {atom,497457}, > > > > {atom_used,482279}, > > > > {binary,1787851232}, > > > > {code,4041219}, > > > > {ets,41231176}] > > > > There seems a memory leak. If I don't stop, the memory usage > > will keep > > > > going higher. 40% CPU, 1.5 MB/s traffic. > > > > > > > > *Latest Cowboy and Cowboy_examples* (same send_interval > > change). It > > > > takes about 7 and half minutes to finish creating 100K > > connections. > > > > During the creation, server CPU usage is about 110%(dual > > core), client > > > > cpu usage is under 20%. After 158.9M packets passed, 1464M > > residence > > > > memory used, 30% CPU, 1.5 MB/s traffic, . > > > > > > > > *Latest Misultin* from github, with patch (see attachment): > > > > It takes about 45s to finish creating 100K connections, > during > > > creation > > > > cpu usage is 120%. After creation, residence memory usage is > > > 2684M. And > > > > it goes down slowly to 1416M after 40M packets are passed, > cpu > > > usage is > > > > about 50% > > > > > > > > Another thing worth to note is that when I hit CTRL-C on the > > client > > > > side, misultin will quickly destroy the connections and > > release memory > > > > and go back to normal. But Cowboy will consume almost all > > the cpu time > > > > and stay there for a long time (I didn't have the patience > > to measure > > > > how long, so just hit ctrl-c to quick erlang). > > > > > > > > Cheers > > > > Andy > > > > > > > > > > > > On Wed, Jun 1, 2011 at 8:26 PM, Andy W. Song > > > > > > > > > > > > >>> wrote: > > > > > > > > BTW, the latest version uses less memory too, about 3.2G > > for 200K > > > > connections. > > > > > > > > > > > > On Wed, Jun 1, 2011 at 8:23 PM, Andy W. Song > > > > > > > > > > > > >>> wrote: > > > > > > > > Missed one attachment. > > > > > > > > > > > > On Wed, Jun 1, 2011 at 8:21 PM, Andy W. Song > > > > > > > > > > > > >>> wrote: > > > > > > > > I tested out the latest Misultin with this > > change, CPU > > > usage > > > > is about 2 to 3 percent lower. > > > > > > > > With the attached Misultin package, I can still > > reproduce > > > > the large packet problem. > > > > > > > > With latest version, Misultin doesn't report any > > > error. But > > > > it doesn't reply to the client either. Seems it > > just drops > > > > the packet. Another attached file is the tcpdump > > result. > > > > > > > > I will try Mochiweb later just to compare. > > > > > > > > > > > > On Wed, Jun 1, 2011 at 12:09 AM, Roberto > Ostinelli > > > > > > > > > > > > >>> wrote: > > > > > > > > > > > > On May 31, 2011, at 12:54 PM, Andy W. Song > > wrote: > > > > > > > >> Would you mind testing it using my C > > client? Search > > > >> "->len" and give > > > >> it a value between 1500 and 2000, hopefully > > > you'll see > > > >> the result I > > > >> saw. > > > >> > > > >> Regards > > > >> Andy > > > > > > > > Hi Andy, > > > > > > > > will do that. > > > > > > > > Meanwhile, I've added an undocumented option, > > > > {ws_no_header, true} so that headers are not > > > duplicated > > > > in the record and should use less memory. > > plus, I've > > > > optimized binary code for websockets. > > > > > > > > can you please retry the test so we have a > > matter of > > > > comparison? > > > > > > > > the only thing you need to change is to add > this > > > option > > > > in misultin initialization, i.e.: > > > > > > > > % start misultin http server > > > > start(Port) -> > > > > misultin:start_link([ > > > > *{ws_no_header, true},* > > > > {port, Port}, {loop, fun(Req) -> > > handle_http(Req, > > > Port) > > > > end}, > > > > {ws_loop, fun(Ws) -> handle_websocket(Ws) > end}, > > > > {ws_autoexit, false} > > > > ]). > > > > > > > > please let me know... > > > > > > > > r. > > > > > > > > > > > > > > > > > > > > -- > > > > > > > --------------------------------------------------------------- > > > > ???????????????????? > > > > ???????????????????? > > > > > > > > > > > > > > > > > > > > -- > > > > > > > --------------------------------------------------------------- > > > > ???????????????????? > > > > ???????????????????? > > > > > > > > > > > > > > > > > > > > -- > > > > > > --------------------------------------------------------------- > > > > ???????????????????? > > > > ???????????????????? > > > > > > > > > > > > > > > > > > > > -- > > > > > --------------------------------------------------------------- > > > > ???????????????????? > > > > ???????????????????? > > > > > > > > > > > > > > > > _______________________________________________ > > > > erlang-questions mailing list > > > > erlang-questions@REDACTED > > > > > > > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > -- > > > Lo?c Hoguin > > > Dev:Extend > > > > > > > > > > > > > > > -- > > > --------------------------------------------------------------- > > > ???????????????????? > > > ???????????????????? > > > > > > > > > -- > > Lo?c Hoguin > > Dev:Extend > > > > > > > > > > -- > > --------------------------------------------------------------- > > ???????????????????? > > ???????????????????? > > > > > -- > Lo?c Hoguin > Dev:Extend > -- --------------------------------------------------------------- ???????????????????? ???????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From dale@REDACTED Wed Jun 22 17:45:03 2011 From: dale@REDACTED (Dale Harvey) Date: Wed, 22 Jun 2011 16:45:03 +0100 Subject: [erlang-questions] Documentation Format Changes (erldocs) In-Reply-To: <327898192.289261307959408715.JavaMail.root@zimbra> References: <327898192.289261307959408715.JavaMail.root@zimbra> Message-ID: It seems like if we need to cross reference the documentation xml with the type annotations in the source, what is the point in having docbuilder generate intermediary xml (with a constantly changing and non documented schema) at all? we could just generate the output format directly from the source. the only reason I dont do now is that a good chunk of the documentation (the important modules like lists etc) are only available in the xml format and not in the source. the old documentation used to take edoc formatted type annotations and embed them into the generated xml (like shown above). the format may be different but I cant see why that isnt far better than cross referencing stuff. Cheers Dale On 13 June 2011 11:03, Lukas Larsson wrote: > Hi Dale! > > All of the documentation is right now being reworked to generate type > information from the specs in the source instead of manually writing them. > tells docbuilder (I think) to generate > the type description from the specs instead of the XML file. If you dig > around in the source code of docbuilder/edoc there should be functions > somewhere there which you can use to generate type information for erldocs. > > Lukas > ----- Original Message ----- > From: "Dale Harvey" > To: "erlang-questions" > Sent: Sunday, 12 June, 2011 02:25:29 GMT +01:00 Amsterdam / Berlin / Bern / > Rome / Stockholm / Vienna > Subject: [erlang-questions] Documentation Format Changes (erldocs) > > > Hey all > > I was wondering about the changes in the format documentation sources, in > the last release I noticed a few function definitions changed from > > keymember(Key, N, TupleList) -> boolean() > Test for membership of a list of tuples > > Key = term() > N = 1..tuple_size(Tuple) > TupleList = [Tuple] >   Tuple = tuple() > to > > > Merge two key-sorted lists of tuples > 1..tuple_size(Tuple) > https://github.com/erlang/otp/blob/dev/lib/stdlib/doc/src/lists.xml#L299is the source for these changes, somehow the recent erlang documention ( > http://www.erlang.org/doc/man/lists.html#keymerge-3 ) shows the full > parameter list information and I am quite confused at how that can be > derived from the source xml. > > I was also just curious about the reason for the change and what was > planned for going forward > > since > http://erldocs.com/R14B03/stdlib/lists.html?i=0&search=lists%20keymerge#keymerge/3 > is a lot less useful than > http://erldocs.com/R14B02/stdlib/lists.html?i=0&search=lists%20keymerge#keymerge/3 > > Cheers, good work on the new release > Dale > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias@REDACTED Wed Jun 22 19:25:27 2011 From: matthias@REDACTED (Matthias Lang) Date: Wed, 22 Jun 2011 19:25:27 +0200 Subject: [erlang-questions] Gotta be an easier way... In-Reply-To: References: Message-ID: <20110622172527.GA2578@corelatus.se> On Wednesday, June 15, Jesper Louis Andersen wrote: > Ugh. You could wrap this up in a helper-function and then call the > helper when you want to do the update. The inliner should make short > work of any trouble there with call overhead. But remember: Inlining is never default; it must be explicitly enabled with a compiler option or a -compile() attribute in the source module. (the 'compile' module manual) so there isn't going to be any "short work" made unless you twiddle options. Matt From icarus.alive@REDACTED Wed Jun 22 19:44:00 2011 From: icarus.alive@REDACTED (Icarus Alive) Date: Wed, 22 Jun 2011 23:14:00 +0530 Subject: [erlang-questions] preferred looping style Message-ID: Hi, Of these 2 styles, which one is preferred, and why ? To me, the readability of 2nd style is better, tough it consumes 1 stack frame extra. Are there some cases where style-1 might be preferred, or be the only option ? %% style-1 ----------------------------------------- mainLoop(InitialRun) -> if InitialRun == true -> % do some initialization, mainLoop(false); true -> % do routine loop tasks, mainLoop(InitialRun) end. %% style-2 ----------------------------------------- loopInit() -> % do some intialization, mainLoop(). mainLoop() -> % do routing loop tasks, mainLoop(). cheers, Icarus -------------- next part -------------- An HTML attachment was scrubbed... URL: From rapsey@REDACTED Wed Jun 22 19:47:38 2011 From: rapsey@REDACTED (Rapsey) Date: Wed, 22 Jun 2011 19:47:38 +0200 Subject: [erlang-questions] preferred looping style In-Reply-To: References: Message-ID: If you are concerning yourself about 1 stack frame extra, you're using the wrong language. Style 2 is definitely the way to go. Sergej On Wed, Jun 22, 2011 at 7:44 PM, Icarus Alive wrote: > Hi, > > Of these 2 styles, which one is preferred, and why ? > To me, the readability of 2nd style is better, tough it consumes 1 stack > frame extra. > Are there some cases where style-1 might be preferred, or be the only > option ? > > %% style-1 ----------------------------------------- > mainLoop(InitialRun) -> > if InitialRun == true -> > % do some initialization, > mainLoop(false); > true -> > % do routine loop tasks, > mainLoop(InitialRun) > end. > > %% style-2 ----------------------------------------- > loopInit() -> > % do some intialization, > mainLoop(). > > mainLoop() -> > % do routing loop tasks, > mainLoop(). > > > cheers, > Icarus > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Wed Jun 22 19:47:48 2011 From: gumm@REDACTED (Jesse Gumm) Date: Wed, 22 Jun 2011 12:47:48 -0500 Subject: [erlang-questions] preferred looping style In-Reply-To: References: Message-ID: Style 2 certainly looks cleaner to me, and that's the way I tend to prefer. And beyond that, it removes the conditional from every iteration. -Jesse On Wed, Jun 22, 2011 at 12:44 PM, Icarus Alive wrote: > Hi, > > Of these 2 styles, which one is preferred, and why ? > To me, the readability of 2nd style is better, tough it consumes 1 stack > frame extra. > Are there some cases where style-1 might be preferred, or be the only option > ? > > %% style-1 ----------------------------------------- > mainLoop(InitialRun) -> > ?if InitialRun == true -> > ? ?% do some initialization, > ? ?mainLoop(false); > ?true -> > ? ?% do routine loop tasks, > ? ?mainLoop(InitialRun) > ?end. > > %% style-2 ----------------------------------------- > loopInit() -> > ?% do some intialization, > ?mainLoop(). > > mainLoop() -> > ?% do routing loop tasks, > ?mainLoop(). > > > cheers, > Icarus > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Jesse Gumm Sigma Star Systems 414.940.4866 gumm@REDACTED http://www.sigma-star.com From daniel@REDACTED Wed Jun 22 19:58:46 2011 From: daniel@REDACTED (Daniel Luna) Date: Wed, 22 Jun 2011 19:58:46 +0200 Subject: [erlang-questions] preferred looping style In-Reply-To: References: Message-ID: I'm not sure I understand what you mean with an extra stack frame. Version 2 has a tail call at the end of loopInit so will not use any extra memory. While version 1 adds an extra comparison operation for every time around the loop. Besides that, if you do any work at all within the loop, that will not matter at all. Go with the readable version. It's faster, uses less memory, and (most importantly) is easier to read. (If you want to be able to do hot code loading on this code you should use ?MODULE:mainLoop(), but most likely just convert whatever you do into a gen_server, supervisor, or gen_event.) /Daniel BTW. Erlang naming convention would have loop_init. CamelCase is for variables, while underscore (_) is used for function names. On 22 June 2011 19:44, Icarus Alive wrote: > Hi, > > Of these 2 styles, which one is preferred, and why ? > To me, the readability of 2nd style is better, tough it consumes 1 stack > frame extra. > Are there some cases where style-1 might be preferred, or be the only option > ? > > %% style-1 ----------------------------------------- > mainLoop(InitialRun) -> > ?if InitialRun == true -> > ? ?% do some initialization, > ? ?mainLoop(false); > ?true -> > ? ?% do routine loop tasks, > ? ?mainLoop(InitialRun) > ?end. > > %% style-2 ----------------------------------------- > loopInit() -> > ?% do some intialization, > ?mainLoop(). > > mainLoop() -> > ?% do routing loop tasks, > ?mainLoop(). > > > cheers, > Icarus > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- http://havebackpack.com - Exploring the world with a 30L backpack. Open to fun challenging assignments anywhere in the world. From icarus.alive@REDACTED Wed Jun 22 20:13:00 2011 From: icarus.alive@REDACTED (Icarus Alive) Date: Wed, 22 Jun 2011 23:43:00 +0530 Subject: [erlang-questions] preferred looping style In-Reply-To: References: Message-ID: Thanks folks - @Daniel, @Jesse and @Sergej. I see the point now. On Wed, Jun 22, 2011 at 11:28 PM, Daniel Luna wrote: > I'm not sure I understand what you mean with an extra stack frame. > Version 2 has a tail call at the end of loopInit so will not use any > extra memory. While version 1 adds an extra comparison operation for > every time around the loop. > > Besides that, if you do any work at all within the loop, that will not > matter at all. > > Go with the readable version. It's faster, uses less memory, and (most > importantly) is easier to read. > > (If you want to be able to do hot code loading on this code you should > use ?MODULE:mainLoop(), but most likely just convert whatever you do > into a gen_server, supervisor, or gen_event.) > > /Daniel > > BTW. Erlang naming convention would have loop_init. CamelCase is for > variables, while underscore (_) is used for function names. > > On 22 June 2011 19:44, Icarus Alive wrote: > > Hi, > > > > Of these 2 styles, which one is preferred, and why ? > > To me, the readability of 2nd style is better, tough it consumes 1 stack > > frame extra. > > Are there some cases where style-1 might be preferred, or be the only > option > > ? > > > > %% style-1 ----------------------------------------- > > mainLoop(InitialRun) -> > > if InitialRun == true -> > > % do some initialization, > > mainLoop(false); > > true -> > > % do routine loop tasks, > > mainLoop(InitialRun) > > end. > > > > %% style-2 ----------------------------------------- > > loopInit() -> > > % do some intialization, > > mainLoop(). > > > > mainLoop() -> > > % do routing loop tasks, > > mainLoop(). > > > > > > cheers, > > Icarus > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > http://havebackpack.com - Exploring the world with a 30L backpack. > Open to fun challenging assignments anywhere in the world. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Wed Jun 22 21:14:35 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 22 Jun 2011 20:14:35 +0100 Subject: [erlang-questions] rebar and eunit test prep Message-ID: Suppose my eunit tests require mnesia to be started and a schema to be installed. How do I incorporate that into ./rebar eunit such that the setup above is done only once? I definitely don't want to start mnesia and install the schema before every test. Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From bob@REDACTED Wed Jun 22 21:21:15 2011 From: bob@REDACTED (Bob Ippolito) Date: Wed, 22 Jun 2011 12:21:15 -0700 Subject: [erlang-questions] rebar and eunit test prep In-Reply-To: References: Message-ID: On Wed, Jun 22, 2011 at 12:14 PM, Joel Reymont wrote: > Suppose my eunit tests require mnesia to be started and a schema to be installed. > > How do I incorporate that into ./rebar eunit such that the setup above is done only once? > > I definitely don't want to start mnesia and install the schema before every test. Take a look at the {setup, ...} fixture [1] and put all of the Mnesia-requiring tests in there. Alternatively you can use meck or another mocking module to stub out Mnesia calls if that makes sense for some of the tests. [1] http://www.erlang.org/doc/apps/eunit/chapter.html#Fixtures -bob From ericbmerritt@REDACTED Wed Jun 22 21:42:17 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Wed, 22 Jun 2011 14:42:17 -0500 Subject: [erlang-questions] rebar and eunit test prep In-Reply-To: References: Message-ID: Bob has advice for how to do this in rebar. However, you want your unit tests to be small and focused. They shouldn't need mnesia started to run, mock up the mnesia calls with meck. These are more integration tests then unit tests, for that use something along the lines of common test. This distinction between self contained unit tests that just focus on the module functionality and then broader integration tests is an important one. Eric On Wed, Jun 22, 2011 at 2:14 PM, Joel Reymont wrote: > Suppose my eunit tests require mnesia to be started and a schema to be installed. > > How do I incorporate that into ./rebar eunit such that the setup above is done only once? > > I definitely don't want to start mnesia and install the schema before every test. > > ? ? ? ?Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From joelr1@REDACTED Wed Jun 22 21:43:42 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 22 Jun 2011 20:43:42 +0100 Subject: [erlang-questions] rebar and eunit test prep In-Reply-To: References: Message-ID: <843E7E12-084B-4FF3-B609-7D8BDF2866C7@gmail.com> On Jun 22, 2011, at 8:42 PM, Eric Merritt wrote: > However, you want your unit tests to be small and focused. They > shouldn't need mnesia started to run, mock up the mnesia calls with > meck. True. I've been using eunit for integration tests with OpenPoker. > These are more integration tests then unit tests, for that use > something along the lines of common test. Any existing projects (github) that use common test? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From roberto@REDACTED Wed Jun 22 21:44:30 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 22 Jun 2011 21:44:30 +0200 Subject: [erlang-questions] gen_tcp and large HTTP headers Message-ID: dear list, i'm using gen_tcp in {packet, http} and {active, once} modes to receive HTTPrequests. whenever i use a very large GET header, like such: http://localhost:8080/foo?var=foobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobar the controlling process receives from the socket the message: {SocketMode, Sock, {http_error, "oobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobar HTTP/1.1\r\n"}} this looks to me like a MTU issue: the GET header is larger than the MTU, and thus the erlang:decode_packet/3 returns the error of a badly formed HTTP header since it is not complete. since i'm in active mode, my loop looks like this: headers(Sock, H) -> gen_tcp:setopts(Sock, [{active, once}]), receive {SocketMode, Sock, {http_header, _, Header, _, Val} = _Head} -> headers(Sock, [{Header, Val}|H]); ... end. thus, should i use an accumulator? but then should i manually trigger the erlang:decode_packet/3 function? thank you for any insights on this. r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Wed Jun 22 21:53:18 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 22 Jun 2011 21:53:18 +0200 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: References: Message-ID: additional information: this can be solved simply by setting: {recbuf, 2000} any reasons for keeping this value to minimum sizes? 2011/6/22 Roberto Ostinelli > dear list, > > i'm using gen_tcp in {packet, http} and {active, once} modes to receive > HTTP requests. > > whenever i use a very large GET header, like such: > > http://localhost:8080/foo?var=foobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobar > > the controlling process receives from the socket the message: > {SocketMode, Sock, > {http_error, "oobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobar > HTTP/1.1\r\n"}} > > this looks to me like a MTU issue: the GET header is larger than the MTU, > and thus the erlang:decode_packet/3 returns the error of a badly formed > HTTP header since it is not complete. > > since i'm in active mode, my loop looks like this: > > headers(Sock, H) -> > gen_tcp:setopts(Sock, [{active, once}]), > receive > {SocketMode, Sock, {http_header, _, Header, _, Val} = _Head} -> > headers(Sock, [{Header, Val}|H]); > > ... > > end. > > thus, should i use an accumulator? but then should i manually trigger the erlang:decode_packet/3 > function? > > thank you for any insights on this. > > r. > -- ------------------------------------------------------------------- Roberto Ostinelli CTO, WideTag Inc. - Realtime, Social, Green widetag.com skype: rostinelli twitter: ostinelli mobile: +39 335 6 100 22 6 -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed Jun 22 21:55:46 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 22 Jun 2011 20:55:46 +0100 Subject: [erlang-questions] rebar and eunit test prep In-Reply-To: <843E7E12-084B-4FF3-B609-7D8BDF2866C7@gmail.com> References: <843E7E12-084B-4FF3-B609-7D8BDF2866C7@gmail.com> Message-ID: On 22 June 2011 20:43, Joel Reymont wrote: > True. I've been using eunit for integration tests with OpenPoker. > >> These are more integration tests then unit tests, for that use >> something along the lines of common test. > > Any existing projects (github) that use common test? https://github.com/hyperthunk/hamcrest-erlang uses common_test as a bootstrap for proper. :) I use common_test more often than not, just out of habit really as I never bothered to get into eunit. Eventually I started to include the eunit.hrl header and use the assert macros in my common_test suites, until finally I started using hamcrest instead. I'm getting more into quickcheck/proper now, which can be combined with all of the above. From bob@REDACTED Wed Jun 22 22:03:42 2011 From: bob@REDACTED (Bob Cowdery) Date: Wed, 22 Jun 2011 21:03:42 +0100 Subject: [erlang-questions] wx-erlang question In-Reply-To: References: <4DFB18BC.8070906@bobcowdery.plus.com> Message-ID: <4E024A9E.90005@bobcowdery.plus.com> Don't think its all working yet but a couple of things you may be able to help with. The wxAuiPaneInfo class has some public attributes, one of which is dock_proportion and is relevant to setting the horizontal proportion the pane will take in the dock. Is there some way to set this? savePerspective/1, loadPerspective/2 should work but doesn't. It works if done in the same session i.e. I can save the perspective and immediately load it again but in a new session I just get a blank window. Bob On 17/06/2011 10:19, Dan Gudmundsson wrote: > I don't know anything about it, I only wrote the wx<->wxWidgets > mapping, you will have to > search/ask in the wxWidgets forum/wiki directly. > > /Dan > > On Fri, Jun 17, 2011 at 11:05 AM, Bob Cowdery > wrote: > > Hi > > Are there any examples of using AUI where the size settings of the > panes > works. minSize, maxSize and bestSize seem to have limited effect, > especially on the X axis where it is completely ignored. I've also > tried > savePerspective and loadPerspective which look good when I look at the > string but attempting to load it back in gives a completely empty > window. > > I appreciate this is probably a wx rather than erlang problem > although I > can't find anything that says perspectives don't work. Should I be > doing > anything special with the string or doing something special when I > restore a perspective? I simply save it in an ETS table which is saved > as a file. I have set the {update, true} option. > > Regards > Bob > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sidcarter@REDACTED Wed Jun 22 22:12:44 2011 From: sidcarter@REDACTED (Sid Carter) Date: Thu, 23 Jun 2011 08:12:44 +1200 Subject: [erlang-questions] Beginner: What's wrong with this code? In-Reply-To: <4E01E0C9.2090700@cs.ntua.gr> References: <4E01E0C9.2090700@cs.ntua.gr> Message-ID: <8E5F6C0AB255461F8A3FAB6861CAE284@gmail.com> On Thursday, 23 June 2011 at 12:32 AM, Kostis Sagonas wrote: > Sid Carter wrote: > > I get the following error: > > > > Error in process <0.49.0> with exit value: > > {undef,[{first,format_temps,[{moscow,{f,-40}},{bangalore,{f,40}},{wellington,{c,24}}]}]} > > > > What's wrong? spawn(fun () -> > format_temps([{moscow,{f,-40}}, > {bangalore,{f,40}}, > {wellington, {c,24}}]) > end) Thank you for this. Is there any place where I can find more about how to write in Modern Erlang. I am currently reading the "Getting Started with Erlang User's Guide" and O'Reilly's "Erlang Programming". Sid -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Wed Jun 22 22:19:39 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 22 Jun 2011 16:19:39 -0400 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: References: Message-ID: IMO this is really a bug in the http packet parser. Setting recbuf might work for this example but if you get a larger URI then it will obviously fail, as you can't set it large enough. For this case the packet parser must realize the http request line is incomplete and complete the parsing only once the whole request line arrives. Getting this completely right might well require allowing a way for the application to control how long a URI it wants to allow, since if the parser just keeps reading in packets to find the end of the request line someone could deny service by just sending request packets that contain a really really long URI. --steve On Wed, Jun 22, 2011 at 3:53 PM, Roberto Ostinelli wrote: > additional information: this can be solved simply by setting: > {recbuf, 2000} > > any reasons for keeping this value to minimum sizes? > > > 2011/6/22 Roberto Ostinelli >> >> dear list, >> >> i'm using gen_tcp in {packet, http} and {active, once} modes to receive >> HTTP requests. >> >> whenever i use a very large GET header, like such: >> >> http://localhost:8080/foo?var=foobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobar >> >> the controlling process receives from the socket the message: >> {SocketMode, Sock, >> ??? {http_error, "oobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobar >> HTTP/1.1\r\n"}} >> >> this looks to me like a MTU issue: the GET header is larger than the MTU, >> and thus the erlang:decode_packet/3 returns the error of a badly formed HTTP >> header since it is not complete. >> >> since i'm in active mode, my loop looks like this: >> >> headers(Sock, H) -> >> ??? gen_tcp:setopts(Sock, [{active, once}]), >> ??? receive >> ??? ??? {SocketMode, Sock, {http_header, _, Header, _, Val} = _Head} -> >> ??? ??? ??? headers(Sock, [{Header, Val}|H]); >> >> ??????? ... >> >> ??? end. >> >> thus, should i use an accumulator? but then should i manually trigger the >> erlang:decode_packet/3 function? >> >> thank you for any insights on this. >> >> r. > > > > -- > ------------------------------------------------------------------- > Roberto Ostinelli > CTO, WideTag Inc. - Realtime, Social, Green > widetag.com > skype: rostinelli > twitter: ostinelli > mobile: +39 335 6 100 22 6 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From roberto@REDACTED Wed Jun 22 22:25:54 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 22 Jun 2011 22:25:54 +0200 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: References: Message-ID: hi steve, it does look so indeed. i actually do receive the http_request message: {http, Sock, {http_request, Method, Path, Version}} even though the GET request has obviously been shortened, so that the remaining part of the GET request gets passed over and then generates the http_error message while parsing headers. i guess i actually should not receive the http_request message, since the GET header is incomplete. 2011/6/22 Steve Vinoski > IMO this is really a bug in the http packet parser. Setting recbuf > might work for this example but if you get a larger URI then it will > obviously fail, as you can't set it large enough. > > For this case the packet parser must realize the http request line is > incomplete and complete the parsing only once the whole request line > arrives. Getting this completely right might well require allowing a > way for the application to control how long a URI it wants to allow, > since if the parser just keeps reading in packets to find the end of > the request line someone could deny service by just sending request > packets that contain a really really long URI. > > --steve > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Wed Jun 22 23:15:56 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 22 Jun 2011 17:15:56 -0400 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: References: Message-ID: Looking at the packet parser, I'm thinking it always treats this case as HTTP version 0.9, so that's wrong as well. Can you verify that major and minor are set to 0 and 9 respectively in the version tuple within the http_request tuple you get? --steve On Wed, Jun 22, 2011 at 4:25 PM, Roberto Ostinelli wrote: > hi steve, > > it does look so indeed. i actually do receive the http_request message: > > {http, Sock, {http_request, Method, Path, Version}} > > even though the GET request has obviously been shortened, so that the > remaining part of the GET request gets passed over and then generates the > http_error message while parsing headers. > > i guess i actually should not receive the http_request message, since the > GET header is incomplete. > > > 2011/6/22 Steve Vinoski >> >> IMO this is really a bug in the http packet parser. Setting recbuf >> might work for this example but if you get a larger URI then it will >> obviously fail, as you can't set it large enough. >> >> For this case the packet parser must realize the http request line is >> incomplete and complete the parsing only once the whole request line >> arrives. Getting this completely right might well require allowing a >> way for the application to control how long a URI it wants to allow, >> since if the parser just keeps reading in packets to find the end of >> the request line someone could deny service by just sending request >> packets that contain a really really long URI. >> >> --steve > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From essen@REDACTED Wed Jun 22 23:17:40 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 22 Jun 2011 23:17:40 +0200 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: References: Message-ID: <4E025BF4.4050505@dev-extend.eu> More info here: https://github.com/extend/cowboy/issues/3 We just switched to calling decode_packet directly. It's a bug in gen_tcp when in HTTP mode, not decode_packet, so it works fine now. I haven't had the time to find the exact source of this bug but it's on my todo list. Cheers. On 06/22/2011 10:25 PM, Roberto Ostinelli wrote: > hi steve, > > it does look so indeed. i actually do receive the http_request message: > > {http, Sock, {http_request, Method, Path, Version}} > > even though the GET request has obviously been shortened, so that the > remaining part of the GET request gets passed over and then generates > the http_error message while parsing headers. > > i guess i actually should not receive the http_request message, since > the GET header is incomplete. > > > 2011/6/22 Steve Vinoski > > > IMO this is really a bug in the http packet parser. Setting recbuf > might work for this example but if you get a larger URI then it will > obviously fail, as you can't set it large enough. > > For this case the packet parser must realize the http request line is > incomplete and complete the parsing only once the whole request line > arrives. Getting this completely right might well require allowing a > way for the application to control how long a URI it wants to allow, > since if the parser just keeps reading in packets to find the end of > the request line someone could deny service by just sending request > packets that contain a really really long URI. > > --steve > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Dev:Extend From vinoski@REDACTED Wed Jun 22 23:50:00 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 22 Jun 2011 17:50:00 -0400 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: <4E025BF4.4050505@dev-extend.eu> References: <4E025BF4.4050505@dev-extend.eu> Message-ID: Yep, I see the same. Looks like the inet driver isn't checking packet length properly like decode_packet does, and as a result it passes incomplete data to the packet parser, which isn't coded to handle that. --steve On Wed, Jun 22, 2011 at 5:17 PM, Lo?c Hoguin wrote: > More info here: > ?https://github.com/extend/cowboy/issues/3 > > We just switched to calling decode_packet directly. It's a bug in > gen_tcp when in HTTP mode, not decode_packet, so it works fine now. I > haven't had the time to find the exact source of this bug but it's on my > todo list. > > Cheers. > > On 06/22/2011 10:25 PM, Roberto Ostinelli wrote: >> hi steve, >> >> it does look so indeed. i actually do receive the http_request message: >> >> {http, Sock, {http_request, Method, Path, Version}} >> >> even though the GET request has obviously been shortened, so that the >> remaining part of the GET request gets passed over and then generates >> the http_error message while parsing headers. >> >> i guess i actually should not receive the http_request message, since >> the GET header is incomplete. >> >> >> 2011/6/22 Steve Vinoski > >> >> ? ? IMO this is really a bug in the http packet parser. Setting recbuf >> ? ? might work for this example but if you get a larger URI then it will >> ? ? obviously fail, as you can't set it large enough. >> >> ? ? For this case the packet parser must realize the http request line is >> ? ? incomplete and complete the parsing only once the whole request line >> ? ? arrives. Getting this completely right might well require allowing a >> ? ? way for the application to control how long a URI it wants to allow, >> ? ? since if the parser just keeps reading in packets to find the end of >> ? ? the request line someone could deny service by just sending request >> ? ? packets that contain a really really long URI. >> >> ? ? --steve >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > -- > Lo?c Hoguin > Dev:Extend > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mail@REDACTED Thu Jun 23 00:06:22 2011 From: mail@REDACTED (Tim Fletcher) Date: Wed, 22 Jun 2011 15:06:22 -0700 (PDT) Subject: [erlang-questions] Oauth 2.0 support In-Reply-To: References: <8980542.5113221308122823278.JavaMail.defaultUser@defaultHost> Message-ID: > I think oauth 2.0 is entirely an application level feature. It requires > application-specific storage, and application-specific implementation in > what you delegate. This is my experience. It's difficult to provide a "plug and play" solution for OAuth on the server side because of all the choices between web servers, frameworks, data storage etc. > That being said, building oauth 2.0 clients is super easy, and building 2.0 > servers is not hard if you already have a sane user authentication and > authorization system, because it all just relies on https. > I've never found oauth 2.0 "helper libraries" to be much help, personally. Exactly. The tricky thing with OAuth 1.0 was the signature calculation (particularly RSA-SHA1), and that was worth sharing. Take that away from the client side and you're left with an awkward wrapper around httpc, which isn't much use. My suggestion would be to implement OAuth 2.0 from scratch, and then look for ways to improve the underlying HTTP libraries to make it easier to write that functionality. Cheers, Tim From roberto@REDACTED Thu Jun 23 00:34:51 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 23 Jun 2011 00:34:51 +0200 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: References: Message-ID: steve, your intuition is correct: version is {0, 9}. i guess the only solution is to avoid {packet, http} and handle these manually? seems such a waste. 2011/6/22 Steve Vinoski > Looking at the packet parser, I'm thinking it always treats this case > as HTTP version 0.9, so that's wrong as well. Can you verify that > major and minor are set to 0 and 9 respectively in the version tuple > within the http_request tuple you get? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sidcarter@REDACTED Thu Jun 23 01:25:34 2011 From: sidcarter@REDACTED (Sid Carter) Date: Thu, 23 Jun 2011 11:25:34 +1200 Subject: [erlang-questions] Beginner: What's wrong with this code? In-Reply-To: <4E01E0C9.2090700@cs.ntua.gr> References: <4E01E0C9.2090700@cs.ntua.gr> Message-ID: On 23 June 2011 00:32, Kostis Sagonas wrote: > > Others have already explained the problem and suggested a fix for it. But > these kinds of errors (forgetting one set of list brackets) are very common. > They can easily be avoided if one writes in more modern Erlang rather than > in Erlang of 10 years ago. In this particular case, you can use a spawn with > a fun that makes the call to function format_temps/1 clear: And while searching, I found this -> http://www.erlang-factory.com/upload/presentations/347/Kostis.pdf I suppose that's a good start on modern Erlang. Are there more? :) Sid From ok@REDACTED Thu Jun 23 04:20:57 2011 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 23 Jun 2011 14:20:57 +1200 Subject: [erlang-questions] preferred looping style In-Reply-To: References: Message-ID: On 23/06/2011, at 5:44 AM, Icarus Alive wrote: > Hi, > > Of these 2 styles, which one is preferred, and why ? > To me, the readability of 2nd style is better, tough it consumes 1 stack frame extra. Why do you say style 2 consumes one stack frame extra? It's called via a tail call, so that when mainLoop/0 (whyDoYouWantToWriteUglyUnreadableNamesLikeThis? DidTheUnderscoreKeyBreakOffYourKeyboardOrWhat?) is running there isn't any frame for loopInit/0. From ixmatus@REDACTED Thu Jun 23 05:19:22 2011 From: ixmatus@REDACTED (Parnell Springmeyer) Date: Wed, 22 Jun 2011 20:19:22 -0700 Subject: [erlang-questions] Beginner: What's wrong with this code? In-Reply-To: (Sid Carter's message of "Thu, 23 Jun 2011 11:25:34 +1200") References: <4E01E0C9.2090700@cs.ntua.gr> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Read some of the popular Erlang libraries being written - I study code all the time; I generally find the code to be better documentation about how to do something than the documentation itself, too. Read code - that's how you will acculturate yourself :) Sid Carter writes: > On 23 June 2011 00:32, Kostis Sagonas wrote: >> >> Others have already explained the problem and suggested a fix for it. But >> these kinds of errors (forgetting one set of list brackets) are very common. >> They can easily be avoided if one writes in more modern Erlang rather than >> in Erlang of 10 years ago. In this particular case, you can use a spawn with >> a fun that makes the call to function format_temps/1 clear: > > And while searching, I found this -> > http://www.erlang-factory.com/upload/presentations/347/Kostis.pdf > > I suppose that's a good start on modern Erlang. Are there more? :) > > Sid > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > - -- Parnell "ixmatus" Springmeyer (http://ixmat.us) -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJOArC6AAoJEPvtlbpI1POLVdsH/34uAO3LCB0EhceRBbU5DieW MMbm4Fqg1q6JYD/r1x+ognCRs88ZvKBmLHFn/ZotlFSJCGWiWKWq3PIzEZp/x+XI HEZ4utJsGjQAQoGXKUvIW/argrSEw62svX7yvPA2dIy4einNH1SF5k3sdDHhDeTu 4z7y74JCuJgAbZG227H/wwK96oKmTpTxcFdrIL4HhvvFnAMVFnsT1PyLZRLi5Dwj l0jk3xpeMAtn0g+wqDROkufdfAVobcTGbLbrNCwY/2VbXNKaDhbODCqLYitko09Q obGSktoVvuiO0l6/ljWryD49aURHZgLpLvIr67a/7gVDl4GeXB6gbtce9BGY0lE= =93AZ -----END PGP SIGNATURE----- From vinoski@REDACTED Thu Jun 23 05:35:21 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 22 Jun 2011 23:35:21 -0400 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: References: Message-ID: Unfortunately, it appears that if you want to be completely safe and handle long request lines like your example, then yes, calling decode_packet is the only way to go. :-( I've stared at the inet driver code some more and the way it's calling the packet parser just doesn't allow it to be informed that a buffer is incomplete and more data from the next packet is needed. --steve On Wed, Jun 22, 2011 at 6:34 PM, Roberto Ostinelli wrote: > steve, > > your intuition is correct: version is {0, 9}. > > i guess the only solution is to avoid {packet, http} and handle these > manually? seems such a waste. > > 2011/6/22 Steve Vinoski >> >> Looking at the packet parser, I'm thinking it always treats this case >> as HTTP version 0.9, so that's wrong as well. Can you verify that >> major and minor are set to 0 and 9 respectively in the version tuple >> within the http_request tuple you get? > From wsongcn@REDACTED Thu Jun 23 05:39:50 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Thu, 23 Jun 2011 11:39:50 +0800 Subject: [erlang-questions] discrepancy of memory usage figures Message-ID: When I run my WebSocket test, I found the following interesting memory usage results: 1. Server stated, no connection [{total,573263528}, {processes,17375688}, {processes_used,17360240}, {system,555887840}, {atom,472297}, {atom_used,451576}, {binary,28944}, {code,3774097}, {ets,271016}] 44 processes, *System:705M, * *Erlang Residence:519M* 2. 100K Connections [{total,762564512}, {processes,130105104}, {processes_used,130089656}, {system,632459408}, {atom,476337}, {atom_used,456484}, {binary,50160}, {code,3925064}, {ets,7589160}] 100044 processes, *System: 1814M, * *Erlang Residence: 950M* 3. 200K Connections ( restart server and create from 0 connection, not continue from case 2) [{total,952040232}, {processes,243161192}, {processes_used,243139984}, {system,708879040}, {atom,476337}, {atom_used,456484}, {binary,70856}, {code,3925064}, {ets,14904760}] 200044 processes, *System:3383M, * *Erlang: 1837M* The figures in red are provided htop, others are output of memory() call from erlang shell. Please look at the total and erlang residence memory. When there is no connection, these two are roughly same, with 100K connections, residence memory is a little larger than total, with 200K connections, residence memory is almost double the total. Can anybody explain? Cheers Andy -------------- next part -------------- An HTML attachment was scrubbed... URL: From banibrata.dutta@REDACTED Thu Jun 23 06:12:41 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Thu, 23 Jun 2011 09:42:41 +0530 Subject: [erlang-questions] Beginner: What's wrong with this code? In-Reply-To: References: <4E01E0C9.2090700@cs.ntua.gr> Message-ID: On Thu, Jun 23, 2011 at 4:55 AM, Sid Carter wrote: > On 23 June 2011 00:32, Kostis Sagonas wrote: > > > > Others have already explained the problem and suggested a fix for it. But > > these kinds of errors (forgetting one set of list brackets) are very > common. > > They can easily be avoided if one writes in more modern Erlang rather > than > > in Erlang of 10 years ago. In this particular case, you can use a spawn > with > > a fun that makes the call to function format_temps/1 clear: > > And while searching, I found this -> > http://www.erlang-factory.com/upload/presentations/347/Kostis.pdf > > I suppose that's a good start on modern Erlang. Are there more? :) > > Great find. Also, wondering if there is a more generic 'Style Guide' (a.lathe PEP for Coding conventions, forgot the number). BTW, the modern erlang code (slide#23), looks bit more "intimidating and crowded" compared to the extremely simple look that old Erlang code has. One of the merits of Erlang code, cited in some language comparisons was based on the LoC comparison, somehow the modern Erlang code seems to add a lot of volume. Although some refactoring done by TidiEr do seem to reduce LoC as well. Anyhow, I'm certainly bookmarking and exploring all suggestions in that presentation. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sidcarter@REDACTED Thu Jun 23 07:07:29 2011 From: sidcarter@REDACTED (Sid Carter) Date: Thu, 23 Jun 2011 17:07:29 +1200 Subject: [erlang-questions] Beginner: What's wrong with this code? In-Reply-To: References: <4E01E0C9.2090700@cs.ntua.gr> Message-ID: <0895ACB38F104F59A344828FB0CBFCF0@gmail.com> On Thursday, 23 June 2011 at 4:12 PM, Banibrata Dutta wrote: > On Thu, Jun 23, 2011 at 4:55 AM, Sid Carter wrote: > > On 23 June 2011 00:32, Kostis Sagonas wrote: > > > > > > Others have already explained the problem and suggested a fix for it. But > > > these kinds of errors (forgetting one set of list brackets) are very common. > > > They can easily be avoided if one writes in more modern Erlang rather than > > > in Erlang of 10 years ago. In this particular case, you can use a spawn with > > > a fun that makes the call to function format_temps/1 clear: > > > > And while searching, I found this -> > > http://www.erlang-factory.com/upload/presentations/347/Kostis.pdf > > > > I suppose that's a good start on modern Erlang. Are there more? :) > > > Great find. Also, wondering if there is a more generic 'Style Guide' (a.la (http://a.la) the PEP for Coding conventions, forgot the number). Or like the node.js style guide -> http://nodeguide.com/style.html , considering I came from node.js to Erlang :) > BTW, the modern erlang code (slide#23), looks bit more "intimidating and crowded" compared to the extremely simple look that old Erlang code has. I agree. That's exactly what I thought once I saw that slide. > One of the merits of Erlang code, cited in some language comparisons was based on the LoC comparison, somehow the modern Erlang code seems to add a lot of volume. Although some refactoring done by TidiEr do seem to reduce LoC as well. Anyhow, I'm certainly bookmarking and exploring all suggestions in that presentation. Sid -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangy@REDACTED Thu Jun 23 07:59:19 2011 From: erlangy@REDACTED (ERLANG) Date: Thu, 23 Jun 2011 07:59:19 +0200 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: References: Message-ID: <746D50D8-6016-407B-A3BB-BDBF14DB200D@gmail.com> Hi Steve, Roberto, Maybe the Erlang team should consider and fix that problem on next releases! /Zab Le 23 juin 11 ? 05:35, Steve Vinoski a ?crit : > Unfortunately, it appears that if you want to be completely safe and > handle long request lines like your example, then yes, calling > decode_packet is the only way to go. :-( I've stared at the inet > driver code some more and the way it's calling the packet parser just > doesn't allow it to be informed that a buffer is incomplete and more > data from the next packet is needed. > > --steve > > On Wed, Jun 22, 2011 at 6:34 PM, Roberto Ostinelli > wrote: >> steve, >> >> your intuition is correct: version is {0, 9}. >> >> i guess the only solution is to avoid {packet, http} and handle these >> manually? seems such a waste. >> >> 2011/6/22 Steve Vinoski >>> >>> Looking at the packet parser, I'm thinking it always treats this >>> case >>> as HTTP version 0.9, so that's wrong as well. Can you verify that >>> major and minor are set to 0 and 9 respectively in the version tuple >>> within the http_request tuple you get? >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From paolo.negri@REDACTED Thu Jun 23 10:34:46 2011 From: paolo.negri@REDACTED (Paolo Negri) Date: Thu, 23 Jun 2011 10:34:46 +0200 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: <746D50D8-6016-407B-A3BB-BDBF14DB200D@gmail.com> References: <746D50D8-6016-407B-A3BB-BDBF14DB200D@gmail.com> Message-ID: Hi Everyone, Thanks for discussing this, we encountered the problem on our system and filed the bug that originated Roberto's email [1]. We're currently working around the problem by setting recbuf on the socket, this temporary solution is not too bad in real life if the need is only to cope with requests originated by browsers as currently fairly popular ones can't handle over 2048 characters [2]. While this is suitable as a temporary solution it really feels like the current implementation of inet is incorrect as it states "The packets are returned with the format according to HttpPacket described in erlang:decode_packet/3" [3] and instead the results are different from those one would get by calling erlang:decode_packet/3 [1] https://github.com/ostinelli/misultin/issues/40 [2] http://support.microsoft.com/kb/208427 [3] http://www.erlang.org/doc/man/inet.html#setopts-2 Thanks, Paolo On Thu, Jun 23, 2011 at 7:59 AM, ERLANG wrote: > Hi Steve, Roberto, > > Maybe the Erlang team should consider and fix that problem > on next releases! > > /Zab From roberto@REDACTED Thu Jun 23 10:54:00 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 23 Jun 2011 10:54:00 +0200 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: References: <746D50D8-6016-407B-A3BB-BDBF14DB200D@gmail.com> Message-ID: 2011/6/23 Paolo Negri > Hi Everyone, > > Thanks for discussing this, we encountered the problem on our system > and filed the bug that originated Roberto's email [1]. > indeed, i didn't refer you because i didn't associate the nickname 'hungryblank' to you :) > We're currently working around the problem by setting recbuf on the > socket, this temporary solution is not too bad in real life if the > need is only to cope with requests originated by browsers as currently > fairly popular ones can't handle over 2048 characters [2]. > i agree that recbuf can provide a solution on GET urls, however as seen on cowboy's issues list the same does apply on other headers, such as long Cookies for instance. ps: i've provided a patch exactly to do this, is it the one you're using or have you patched misultin yourself? > While this is suitable as a temporary solution it really feels like > the current implementation of inet is incorrect as it states "The > packets are returned with the format according to HttpPacket described > in erlang:decode_packet/3" [3] and instead the results are different > from those one would get by calling erlang:decode_packet/3 > > [1] https://github.com/ostinelli/misultin/issues/40 > [2] http://support.microsoft.com/kb/208427 > [3] http://www.erlang.org/doc/man/inet.html#setopts-2 > > Thanks, > > Paolo > let's see if the OTP team pops in in this discussion. it probably is a known issue. r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From als@REDACTED Thu Jun 23 13:06:51 2011 From: als@REDACTED (Anthony Shipman) Date: Thu, 23 Jun 2011 21:06:51 +1000 Subject: [erlang-questions] strange badarith error Message-ID: <201106232106.52038.als@iinet.net.au> Today I was getting this occasional error while stress-testing: 23-Jun-2011 18:24:20.290| ERROR fleetDevMgr: device 38: The process for the device has exited for reason: {badarith, [{fleetDevImpl,qualFromDevice, 2}, {fleetDevImpl, handle_cast, 2}, {asn1rt_ber_bin, cindex, 3}]} where the code in qualFromDevice is case streamUtils:computeSTEPQuality(Q, true) of {ok, RSUp, RSDown, RSUpBad, _, Blurb} -> ?DEBUG("fleetDevImpl: ~s: quality ~s", [deviceName(), Blurb]), T1 = State#state.traffic, T2 = T1#voiceTrafficStats{frameErr = (if RSUp == 0 -> 0.0; true -> RSUpBad/RSUp end)}, State#state{ currTraffic = {RSUp, RSDown}, % totals since the beginning of the call traffic = T2, trafficChg = true }; _ -> State end The only arithmetic in here is the division RSUpBad/RSUp and they are definitely integers. The DEBUG line is definitely executed. The code has been compiled with Hipe in R12B4 Erlang (BEAM) emulator version 5.6.4 [source] [async-threads:0] [hipe] [kernel-poll:true] Is it possible I've got bad machine code here? Why might asn1rt_ber_bin be in the stack trace when it isn't involved in this code at all? -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From jesper.louis.andersen@REDACTED Thu Jun 23 14:38:21 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 23 Jun 2011 14:38:21 +0200 Subject: [erlang-questions] discrepancy of memory usage figures In-Reply-To: References: Message-ID: On Thu, Jun 23, 2011 at 05:39, Andy W. Song wrote: > The figures in red are provided htop, others are output of memory() call > from erlang shell. Please look at the total and erlang residence memory. > When there is no connection, these two are roughly same, with 100K > connections, residence memory is a little larger than total, with 200K > connections, residence memory is almost double the total. > Can anybody explain? The first thing I would analyze in this situation is memory fragmentation obtained from the underlying memory allocation. If the memory fragments, then surely you can see stuff like this happening where your resident set size will be larger. I would definitely start out by setting the spotlight on the underlying allocator or the allocation pattern because as you see, there is no discrepancy if you ask the Erlang VM about its view on the matter. What speaks against fragmentation is the very young life of the program. Fragmentation problems usually only show up in long-running programs where pointers are set in stone (making compaction impossible). Another option is that there is some auxiliary data structure in the Erlang VM which are grown up in between the 100K and 200K connection count. If the structure is accessed randomly, then surely all the Virtual Memory pages will get allocated and assigned to the process, increasing the RSS of the program. It is also interesting to look for what happens if you simply ignore the program for a bit of time. Perhaps the allocated memory is not released right away, but over time. Apart from my ideas, I am afraid you will need an Erlang VM expert with more intrinsic knowledge than me. -- J. From the.malkolm@REDACTED Thu Jun 23 15:33:56 2011 From: the.malkolm@REDACTED (Alexander Krasnukhin) Date: Thu, 23 Jun 2011 15:33:56 +0200 Subject: [erlang-questions] preferred looping style In-Reply-To: References: Message-ID: Definitely the second one is better. It's more readable. On Thu, Jun 23, 2011 at 4:20 AM, Richard O'Keefe wrote: > > On 23/06/2011, at 5:44 AM, Icarus Alive wrote: > > > Hi, > > > > Of these 2 styles, which one is preferred, and why ? > > To me, the readability of 2nd style is better, tough it consumes 1 stack > frame extra. > > Why do you say style 2 consumes one stack frame extra? > It's called via a tail call, so that when mainLoop/0 > (whyDoYouWantToWriteUglyUnreadableNamesLikeThis? > DidTheUnderscoreKeyBreakOffYourKeyboardOrWhat?) > is running there isn't any frame for loopInit/0. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Regards, Alexander -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Thu Jun 23 16:09:23 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 23 Jun 2011 17:09:23 +0300 Subject: [erlang-questions] strange badarith error In-Reply-To: <201106232106.52038.als@iinet.net.au> References: <201106232106.52038.als@iinet.net.au> Message-ID: <4E034913.2020201@cs.ntua.gr> Anthony Shipman wrote: > Today I was getting this occasional error while stress-testing: > > 23-Jun-2011 18:24:20.290| ERROR fleetDevMgr: device 38: The process for the > device has exited for reason: {badarith, > [{fleetDevImpl,qualFromDevice, 2}, > {fleetDevImpl, handle_cast, 2}, > {asn1rt_ber_bin, cindex, 3}]} > > .... > > The only arithmetic in here is the division RSUpBad/RSUp and they are > definitely integers. The DEBUG line is definitely executed. > > The code has been compiled with Hipe in R12B4 > Erlang (BEAM) emulator version 5.6.4 [source] [async-threads:0] [hipe] > [kernel-poll:true] > > Is it possible I've got bad machine code here? > > Why might asn1rt_ber_bin be in the stack trace when it isn't involved in this > code at all? It's very difficult to say something intelligent here without being able to reproduce this and at least be able to form some educated guess on what might be the culprit. If you have the possibility, try to create a self-contained test that shows the behavior you are experiencing. Also, try to see whether this also happens in R14. (R12 is too old and many things which have been fixed by then may be responsible for what you are experiencing.) Kostis From sverker@REDACTED Thu Jun 23 16:16:07 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 23 Jun 2011 16:16:07 +0200 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: References: <746D50D8-6016-407B-A3BB-BDBF14DB200D@gmail.com> Message-ID: <4E034AA7.5030004@erix.ericsson.se> The bug is causing http lines to get truncated when the inet_drv in-buffer gets full. It would be nicer to dynamically expand the in-buffer and let inet:setopts option {packet_size, MaxSize} be used to protect against DoS attacks as it can for other packet types.** Steve Vinoski wrote: > Looking at the packet parser, I'm thinking it always treats this case > as HTTP version 0.9, so that's wrong as well. > What's wrong? I didn't understand that. /Sverker, Erlang/OTP Paolo Negri wrote: > Hi Everyone, > > Thanks for discussing this, we encountered the problem on our system > and filed the bug that originated Roberto's email [1]. > We're currently working around the problem by setting recbuf on the > socket, this temporary solution is not too bad in real life if the > need is only to cope with requests originated by browsers as currently > fairly popular ones can't handle over 2048 characters [2]. > While this is suitable as a temporary solution it really feels like > the current implementation of inet is incorrect as it states "The > packets are returned with the format according to HttpPacket described > in erlang:decode_packet/3" [3] and instead the results are different > from those one would get by calling erlang:decode_packet/3 > > [1] https://github.com/ostinelli/misultin/issues/40 > [2] http://support.microsoft.com/kb/208427 > [3] http://www.erlang.org/doc/man/inet.html#setopts-2 > > Thanks, > > Paolo > > On Thu, Jun 23, 2011 at 7:59 AM, ERLANG wrote: > >> Hi Steve, Roberto, >> >> Maybe the Erlang team should consider and fix that problem >> on next releases! >> >> /Zab >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From matthias@REDACTED Thu Jun 23 16:49:07 2011 From: matthias@REDACTED (Matthias Lang) Date: Thu, 23 Jun 2011 16:49:07 +0200 Subject: [erlang-questions] Automated Stripping of otp libraries / modules In-Reply-To: <20110620100128.GA3232@corelatus.se> References: <20110620100128.GA3232@corelatus.se> Message-ID: <20110623144907.GA20891@corelatus.se> Hi, Dale Harvey asked about how to figure out what modules are actually needed in a system. I promised to try out some different approaches on the control system for the hardware I work on. Executive summary: - Using my code:all_loaded() approach nets me 153 modules - Xref looks like a dead end. Xref gives me 475 modules in a simple analysis - Dialyzer might be able to do better. I don't know. The rest of this post is long. Sorry. I should probably get a personal blog, but then nobody would read it. Hacky approach: I previously described what we actually do: run our test suite and then call code:all_loaded(). Simple, but only pulls in modules the test suite touches. Seems unbeatable for my purposes. Use Xref to find module dependencies: 475 modules Xref is OTP's cross-referencing tool. There's an analyzer in xref which reads .beam files and produces a call graph. A call graph is just a (big) list of which function calls which other function. So we could use that to see which modules are actually needed in a system. The other part of xref is a query language which lets you determine things about the call graph. Anyway, let's just dive in. The call {ok, Modules} = xref:analyse(Xref, {module_call, [gth_mop]}) returns a list of modules used by gth_mop. gth_mop is the 'entry point' for my system. So if I keep calling {ok, More} = xref:analyse(Xref, {module_call, Modules}) until More == Modules, then I've got every module the system needs. The list produced this way is huge, 475 modules, and includes a bunch of things which are obviously _not_ needed, e.g. 'wx' on a system which can't possibly run 'wx'. Not so good. Use reltool (H?kan Mattsson's suggestion) It looks like reltool just uses 'xref'. Instead of using xref:analyse/2, it uses a query, xref:q(Pid, "UM"). I'm not sure if it does that recursively or not, but I can't see how it can solve this problem better than xref. But I'm no reltool expert. Can Xref do a better job? My first approach with xref is crap. Imagine this system with two modules: system entry point is m:f m:f calls n:f n:f calls nothing n:g calls o:f o:f can't be reached from m:f, but xref's module_call analysis will include 'o' in the results. So the module_call analysis is not the right way to go for this problem. We need to use the call graph. There's probably an xref query to do what I want, but thinking in terms of xref's query language is beyond me*. So I just get xref to give me the call graph edges, like this (E stands for edge): > xref:q(Xref, "E"). {ok,[{{m,f,0},{n,f,0}}, {{n,g,0},{o,f,0}}]} you can see from the call graph that o:f/0 isn't reachable from m:f/0 But this falls in a heap as soon as you use 'spawn' or M:F in even slightly tricky ways, e.g. go() -> B = b, spawn(fun() -> B:f() end). call graph: [{{mml,go,0},{'$M_EXPR',f,0}}, {{mml,go,0},{erlang,spawn,1}}]} '$M_EXPR' is xref-speak for "I don't know which module this is". Can Dialyzer do this better? Dialyzer is remarkably good at finding dead code, so I wonder if it can produce a call graph better than xref does. But I've already spent the better part of a day on xref so poking around dialyzer's will have to wait. Matt * The xref manpage says that xref has a "simple query language". The language has more than 20 predefined variables, a bunch of operators including |, || and |||, regular expressions and a cast syntax. I don't think that qualifies as "simple", unless your hobby is designing query languages. From robert.virding@REDACTED Thu Jun 23 22:10:22 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 23 Jun 2011 20:10:22 +0000 (GMT) Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: <673339621.9431308859775485.JavaMail.root@zimbra> Message-ID: <2071826284.9451308859821971.JavaMail.root@zimbra> Wouldn't it be better to rewrite the packet parser to work in a re-entrant fashion so as to be able to span over many packets? It would save a lot of fiddling with the recbuf size. Robert ----- "Sverker Eriksson" wrote: > The bug is causing http lines to get truncated when the inet_drv > in-buffer gets full. > > It would be nicer to dynamically expand the in-buffer and let > inet:setopts option {packet_size, MaxSize} be used to protect against > > DoS attacks as it can for other packet types.** > > Steve Vinoski wrote: > > > Looking at the packet parser, I'm thinking it always treats this > case > > as HTTP version 0.9, so that's wrong as well. > > > > What's wrong? I didn't understand that. > > /Sverker, Erlang/OTP > > > Paolo Negri wrote: > > Hi Everyone, > > > > Thanks for discussing this, we encountered the problem on our > system > > and filed the bug that originated Roberto's email [1]. > > We're currently working around the problem by setting recbuf on the > > socket, this temporary solution is not too bad in real life if the > > need is only to cope with requests originated by browsers as > currently > > fairly popular ones can't handle over 2048 characters [2]. > > While this is suitable as a temporary solution it really feels like > > the current implementation of inet is incorrect as it states "The > > packets are returned with the format according to HttpPacket > described > > in erlang:decode_packet/3" [3] and instead the results are > different > > from those one would get by calling erlang:decode_packet/3 > > > > [1] https://github.com/ostinelli/misultin/issues/40 > > [2] http://support.microsoft.com/kb/208427 > > [3] http://www.erlang.org/doc/man/inet.html#setopts-2 > > > > Thanks, > > > > Paolo > > > > On Thu, Jun 23, 2011 at 7:59 AM, ERLANG wrote: > > > >> Hi Steve, Roberto, > >> > >> Maybe the Erlang team should consider and fix that problem > >> on next releases! > >> > >> /Zab > >> > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From fred.hebert@REDACTED Thu Jun 23 23:05:46 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Thu, 23 Jun 2011 17:05:46 -0400 Subject: [erlang-questions] Spawnfest Erlang Contest reminder! Message-ID: <92323D9F-A131-48A5-9FD7-DCCEAB57AB17@erlang-solutions.com> Hey there everyone, This is a reminder for all interested people that spawnfest is going to take place in two weeks from today. Spawnfest is the first edition of an annual 48 hour Erlang development competition where you can compete against other developers in building the best, most breathtaking Erlang application, library or anything you like. The contest aims to let people flex their Erlang muscles and showcase awesome things they can do, but also aims to bring the community together and boost the interest towards the language. Teams of one to four members can subscribe to the contest by submitting the required info to our form (http://spawnfest.com/take-part/). Note that if you don't have a team and do not want to be a team of one, we have another form to help find potential teammates (http://spawnfest.com/find-a-team/). The contest commences at midnight Saturday 9th of July, 2011 00:00:00 GMT+0 and ends on Sunday 10th of July, 2011 23:59:59 GMT+0. The full set of rules is available at http://spawnfest.com/rules/ Our judging team is: - Bob Ippolito, CTO and co-founder of Mochi Media Inc; - Justin Sheehy, CTO at Basho Technologies; - Robert Virding, Co-Inventor of Erlang and Erlang Solutions Principal Language Expert; - Ulf Wiger, CTO of Erlang Solutions. And we have prizes! See them on our prize page (http://spawnfest.com/prizes/). We're always hunting for more prizes, so let us know if you're interested in providing any. I'll leave you with this little 'promotional' video: http://vimeo.com/22360540, and I hope we've sparked your interest. -- Fred H?bert http://www.erlang-solutions.com From romanshestakov@REDACTED Fri Jun 24 00:12:54 2011 From: romanshestakov@REDACTED (Roman Shestakov) Date: Thu, 23 Jun 2011 23:12:54 +0100 (BST) Subject: [erlang-questions] how to run PropEr tests from Eunit? Message-ID: <1308867174.83770.YahooMailRC@web25402.mail.ukl.yahoo.com> hello, I am trying to run PropEr tests from Eunit and I must admit, I can't figure out how to run them lets say I want to test the following three functions %%-------------------------------------------------------------------- %% @doc %% convert Date to atom %% @end %%-------------------------------------------------------------------- -spec date_to_atom(date()) -> atom(). date_to_atom({Year, Month, Day}) -> list_to_atom(date_to_string({Year, Month, Day})). %%-------------------------------------------------------------------- %% @doc %% Convert Date to string %% @end %%-------------------------------------------------------------------- -spec date_to_string(date()) -> string(). date_to_string({Year, Month, Day}) -> lists:flatten(io_lib:format("~4.10.0B~2.10.0B~2.10.0B", [Year, Month, Day])). %%-------------------------------------------------------------------- %% @doc %% convert Date represented as atom into Erlang date format %% @end %%-------------------------------------------------------------------- -spec atom_to_date(atom()) -> date(). atom_to_date(Date) -> Year = list_to_integer(lists:sublist(atom_to_list(Date),1,4)), Month = list_to_integer(lists:sublist(atom_to_list(Date),5,2)), Day = list_to_integer(lists:sublist(atom_to_list(Date),7,2)), {Year, Month, Day}. with the following property: proper_time_to_atom() -> ?FORALL(Date, date(), begin EncDate = ec_time_fns:atom_to_date(ec_time_fns:date_to_atom(Date)), EncDate =:= Date end). running the following from erl works fine proper:quickcheck(ec_time_fns_tests:proper_time_to_atom()). but how do I make eunit to run these test? tried this , but it doesn't seem to work time_to_atom_test_() -> ?_test(proper:quickcheck(proper_time_to_atom())). any ideas? Regards, Roman -------------- next part -------------- An HTML attachment was scrubbed... URL: From fred.hebert@REDACTED Fri Jun 24 00:54:36 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Thu, 23 Jun 2011 18:54:36 -0400 Subject: [erlang-questions] how to run PropEr tests from Eunit? In-Reply-To: <1308867174.83770.YahooMailRC@web25402.mail.ukl.yahoo.com> References: <1308867174.83770.YahooMailRC@web25402.mail.ukl.yahoo.com> Message-ID: <93BD232D-C14F-4B16-80D2-96C24EE44BB3@erlang-solutions.com> The simplest way I know to do them individually is with separate tests: proper_test_() -> [?_assert(proper:quickcheck(prop_some_property())), ?_assert(proper:quickcheck(prop_some_property())), ...]. You can then create macros to do it for you of the form -define(PROPTEST(A), ?_assert(proper:quickcheck(A()))). and then call proper_test_() -> [?PROPTEST(prop_something), ?PROPTEST(prop_something_else), ?PROPTEST(prop_other), ?PROPTEST(prop_last)]. or whatever you feel like. If you want to run them all, make sure your module's properties all start with the prefix 'prop_'. Then you can do something like proper_test() -> [] = proper:module(?MODULE). which will find all the properties of the current module and run them for you. Also, don't forget to always include the proper include file first, because both PropEr and EUnit define a LET macro, but the PropEr one is the only one I've seen actively used. -- Fred H?bert http://www.erlang-solutions.com On 2011-06-23, at 18:12 PM, Roman Shestakov wrote: > hello, > > I am trying to run PropEr tests from Eunit and I must admit, I can't figure out how to run them > > > lets say I want to test the following three functions > > %%-------------------------------------------------------------------- > %% @doc > %% convert Date to atom > %% @end > %%-------------------------------------------------------------------- > -spec date_to_atom(date()) -> atom(). > date_to_atom({Year, Month, Day}) -> > list_to_atom(date_to_string({Year, Month, Day})). > > %%-------------------------------------------------------------------- > %% @doc > %% Convert Date to string > %% @end > %%-------------------------------------------------------------------- > -spec date_to_string(date()) -> string(). > date_to_string({Year, Month, Day}) -> > lists:flatten(io_lib:format("~4.10.0B~2.10.0B~2.10.0B", [Year, Month, Day])). > > > %%-------------------------------------------------------------------- > %% @doc > %% convert Date represented as atom into Erlang date format > %% @end > %%-------------------------------------------------------------------- > -spec atom_to_date(atom()) -> date(). > atom_to_date(Date) -> > Year = list_to_integer(lists:sublist(atom_to_list(Date),1,4)), > Month = list_to_integer(lists:sublist(atom_to_list(Date),5,2)), > Day = list_to_integer(lists:sublist(atom_to_list(Date),7,2)), > {Year, Month, Day}. > > > > with the following property: > > proper_time_to_atom() -> > ?FORALL(Date, date(), > begin > EncDate = ec_time_fns:atom_to_date(ec_time_fns:date_to_atom(Date)), > EncDate =:= Date > end). > > running the following from erl works fine > proper:quickcheck(ec_time_fns_tests:proper_time_to_atom()). > > > but how do I make eunit to run these test? > > > tried this , but it doesn't seem to work > > time_to_atom_test_() -> > ?_test(proper:quickcheck(proper_time_to_atom())). > > > any ideas? > > Regards, Roman > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From wsongcn@REDACTED Fri Jun 24 03:57:20 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Fri, 24 Jun 2011 09:57:20 +0800 Subject: [erlang-questions] discrepancy of memory usage figures In-Reply-To: References: Message-ID: Thanks for the suggestions. I left the machine for more than 10 hours and the figures didn't change. I'm out of ideas. How can I approach to experts in stead of passively waiting on the mailing list? Thanks Andy On Thu, Jun 23, 2011 at 8:38 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Thu, Jun 23, 2011 at 05:39, Andy W. Song wrote: > > > The figures in red are provided htop, others are output of memory() call > > from erlang shell. Please look at the total and erlang residence memory. > > When there is no connection, these two are roughly same, with 100K > > connections, residence memory is a little larger than total, with 200K > > connections, residence memory is almost double the total. > > Can anybody explain? > > The first thing I would analyze in this situation is memory > fragmentation obtained from the underlying memory allocation. If the > memory fragments, then surely you can see stuff like this happening > where your resident set size will be larger. I would definitely start > out by setting the spotlight on the underlying allocator or the > allocation pattern because as you see, there is no discrepancy if you > ask the Erlang VM about its view on the matter. What speaks against > fragmentation is the very young life of the program. Fragmentation > problems usually only show up in long-running programs where pointers > are set in stone (making compaction impossible). > > Another option is that there is some auxiliary data structure in the > Erlang VM which are grown up in between the 100K and 200K connection > count. If the structure is accessed randomly, then surely all the > Virtual Memory pages will get allocated and assigned to the process, > increasing the RSS of the program. > > It is also interesting to look for what happens if you simply ignore > the program for a bit of time. Perhaps the allocated memory is not > released right away, but over time. > > Apart from my ideas, I am afraid you will need an Erlang VM expert > with more intrinsic knowledge than me. > > > -- > J. > -- --------------------------------------------------------------- ???????????????????? ???????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Jun 24 04:24:30 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 24 Jun 2011 06:24:30 +0400 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: <2071826284.9451308859821971.JavaMail.root@zimbra> References: <673339621.9431308859775485.JavaMail.root@zimbra> <2071826284.9451308859821971.JavaMail.root@zimbra> Message-ID: On Fri, Jun 24, 2011 at 12:10 AM, Robert Virding wrote: > Wouldn't it be better to rewrite the packet parser to work in a re-entrant fashion so as to be able to span over many packets? It would save a lot of fiddling with the recbuf size. > You always have to keep buffer: either in receiver, either in parser. Easier to use when parser is stateless, but it requires more work when you parse again and again old data on receiving new. From dmitriy.kargapolov@REDACTED Fri Jun 24 05:49:36 2011 From: dmitriy.kargapolov@REDACTED (Dmitriy Kargapolov) Date: Thu, 23 Jun 2011 23:49:36 -0400 Subject: [erlang-questions] GC memory consumption Message-ID: <4E040950.9000203@gmail.com> Hello, Erlang application we develop encountered memory lack problem (It's 32-bit unix system, so memory is limited by 2GB total per application). Analysing crash dumps I have found consistent scenario for the issue. Important facts: 1. Slogan: eheap_alloc: Cannot allocate 298930300 bytes of memory (of type "heap"). 2. All the processes are in 'waiting' state except one which is in 'Garbing' state. 3. The 'Garbing' process 'Stack+heap' is 59786060 bytes, OldHeap is 0 bytes, some fragment data available (174663 bytes or so). 4. System has also pretty large mnesia tables, this explains why there is not enough memory to allocate 300 MB segment at some point of execution. Looking at the GC code I found that it always allocates new heap segment to do its work. So in our consistent pattern GC tries to allocate 298930300 bytes to perform garbage collection for the process which heap is 59786060 bytes. Looks ridiculous - to de-fragment ~60 MB of the process heap, GC needs additional ~300 MB segment... This makes very difficult to design processes which have to keep large amount of data it its space (for example big tree-like structure or in-memory graph). Is there any known workaround for this (except moving it to 64 bit OS) ? Thanks. From garazdawi@REDACTED Fri Jun 24 11:47:58 2011 From: garazdawi@REDACTED (Lukas Larsson) Date: Fri, 24 Jun 2011 11:47:58 +0200 Subject: [erlang-questions] rebar and eunit test prep In-Reply-To: <843E7E12-084B-4FF3-B609-7D8BDF2866C7@gmail.com> References: <843E7E12-084B-4FF3-B609-7D8BDF2866C7@gmail.com> Message-ID: All of the Erlang/OTP tests also use common_test if you are looking for examples. On Jun 22, 2011 9:43 PM, "Joel Reymont" wrote: > > On Jun 22, 2011, at 8:42 PM, Eric Merritt wrote: > >> However, you want your unit tests to be small and focused. They >> shouldn't need mnesia started to run, mock up the mnesia calls with >> meck. > > True. I've been using eunit for integration tests with OpenPoker. > >> These are more integration tests then unit tests, for that use >> something along the lines of common test. > > Any existing projects (github) that use common test? > > Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomasl_erlang@REDACTED Fri Jun 24 12:44:29 2011 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Fri, 24 Jun 2011 03:44:29 -0700 (PDT) Subject: [erlang-questions] Automated Stripping of otp libraries / modules In-Reply-To: <20110623144907.GA20891@corelatus.se> References: <20110620100128.GA3232@corelatus.se> <20110623144907.GA20891@corelatus.se> Message-ID: <1308912269.97290.YahooMailNeo@web111406.mail.gq1.yahoo.com> >________________________________ >From: Matthias Lang >To: Dale Harvey >Cc: erlang-questions >Sent: Thursday, June 23, 2011 4:49 PM >Subject: Re: [erlang-questions] Automated Stripping of otp libraries / modules > >Hi, > >Dale Harvey asked about how to figure out what modules are actually >needed in a system. I promised to try out some different approaches on >the control system for the hardware I work on. > >Executive summary: > >???- Using my code:all_loaded() approach nets me 153 modules >???- Xref looks like a dead end. Xref gives me 475 modules in a simple analysis >???- Dialyzer might be able to do better. I don't know. Regarding static analysis, I seem to recall that Sven-Olof Nystr?m's soft typing analysis iterated control flow and dataflow analysis, and could thus (at least in theory) figure out the higher-order calls and get a tighter bound than xref. Here is the page:?http://user.it.uu.se/~svenolof/SA/ You will have to talk to Sven-Olof if there are any support issues. Unfortunately, the control flow analysis problem is difficult, so we can't expect too much in the general case. Just adding closures to the language makes control flow tricky. In erlang, we furthermore have some extra complications, e.g., constructing module and function names at runtime and invoking them. As you note later on, modules furthermore sometimes group unrelated functions and so the module call graph can yield irritating false dependences.?Some years ago, I saw exactly this problem when developing a high-level optimizer for erlang; it can get particularly annoying with libraries like lists. My solution then was to start off with a round of (automated) module splitting. That might not be for everyone, of course. Best, Thomas From mmzeeman@REDACTED Fri Jun 24 12:49:02 2011 From: mmzeeman@REDACTED (mmzeeman@REDACTED) Date: Fri, 24 Jun 2011 12:49:02 +0200 Subject: [erlang-questions] Charset conversion codec interface? Message-ID: <6c82214544842c8a3f43519d3948649c.squirrel@webmail.xs4all.nl> Hello, I have to convert incoming data from a lot of different character sets (iso8859-15, gb2312, ...) to utf8. Something Erlang isn't very good at at the moment. I have been using the iconv port library, which has some problems. It doesn't handle chunks of data > 64kb, and a handy interface to ignore missing characters and errors, multi-byte input conversions are unreliable (because of the 64kb chunking). All fixable... I want to spend some time to fix these shortcomings. What I was wondering. Is there a preferred interface for an erlang codec library. An EEP? Or are others working busy with this? A generic codec interface would make things so much simpler for a lot of others too. Kind regards, Maas From thomasl_erlang@REDACTED Fri Jun 24 12:52:33 2011 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Fri, 24 Jun 2011 03:52:33 -0700 (PDT) Subject: [erlang-questions] discrepancy of memory usage figures In-Reply-To: References: Message-ID: <1308912753.40419.YahooMailNeo@web111411.mail.gq1.yahoo.com> Do you mean contact on commercial terms? I think there are several organizations on the list who could help you. Off the top of my head, I believe Erlang/OTP offer support contracts. Erlang Solutions should be happy to take your call. Mikael Petterson, long-time Hipe guru at Uppsala, might be available for contracting work through his company (I don't know the company contact details, I'm afraid). No doubt there are others. Best, Thomas >________________________________ >From: Andy W. Song >To: Jesper Louis Andersen >Cc: erlang-questions >Sent: Friday, June 24, 2011 3:57 AM >Subject: Re: [erlang-questions] discrepancy of memory usage figures > > >Thanks for the suggestions. I left the machine for more than 10 hours and the figures didn't change.? > > >I'm out of ideas. How can I approach to experts in stead of?passively?waiting on the mailing list? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Fri Jun 24 13:53:42 2011 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 24 Jun 2011 12:53:42 +0100 Subject: [erlang-questions] behaviour X undefined Message-ID: <3D028942-BFD8-4044-AEC9-704CA0C058D6@gmail.com> Why is this happening? make ==> getopt (compile) Compiled src/getopt.erl ==> openpoker (compile) Compiled src/limit.erl Compiled src/exch.erl ... src/tourney.erl:22: Warning: behaviour exch undefined The top of exch.erl looks like this -module(exch, [Cbk, Context, Modules]). -behaviour(gen_server). -export([behaviour_info/1]). -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). And tourney.erl -module(tourney). -behaviour(exch). -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From jalanzo@REDACTED Fri Jun 24 13:55:07 2011 From: jalanzo@REDACTED (Juan Antonio Berrayarza =?ISO-8859-1?Q?Lanz=F3?=) Date: Fri, 24 Jun 2011 13:55:07 +0200 Subject: [erlang-questions] odbc blob datatype Message-ID: <1308916507.7555.7.camel@Ubuntu-Juan> Hi : I'm trying to access a table in Mysql that has TEXT fields. when i do a odbc:describe_table() i get {"field_name",ODBC_UNSUPPORTED_TYPE}. Reading documentation about erlang odbc, i see that TEXT fields should be mapped to SQL_LONGVARCHAR. Has anyone worked this out? how can i query tables with TEXT fields using odbc and erlang ? erlang : R14B02 odbc: 2.10.10 mysql: 4.1.49 odbc mysql driver : 5.1.8 Tx to all of you From torben.lehoff@REDACTED Fri Jun 24 13:58:19 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 24 Jun 2011 13:58:19 +0200 Subject: [erlang-questions] behaviour X undefined In-Reply-To: <3D028942-BFD8-4044-AEC9-704CA0C058D6@gmail.com> References: <3D028942-BFD8-4044-AEC9-704CA0C058D6@gmail.com> Message-ID: I think you are missing a path to the beam file for exch when you compile tourney. Cheers, Torben On Fri, Jun 24, 2011 at 13:53, Joel Reymont wrote: > Why is this happening? > > make > ==> getopt (compile) > Compiled src/getopt.erl > ==> openpoker (compile) > Compiled src/limit.erl > Compiled src/exch.erl > ... > src/tourney.erl:22: Warning: behaviour exch undefined > > The top of exch.erl looks like this > > -module(exch, [Cbk, Context, Modules]). > -behaviour(gen_server). > > -export([behaviour_info/1]). > > -export([init/1, handle_call/3, handle_cast/2, > handle_info/2, terminate/2, code_change/3]). > > And tourney.erl > > -module(tourney). > -behaviour(exch). > > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Fri Jun 24 14:07:15 2011 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 24 Jun 2011 13:07:15 +0100 Subject: [erlang-questions] behaviour X undefined In-Reply-To: References: <3D028942-BFD8-4044-AEC9-704CA0C058D6@gmail.com> Message-ID: <67894D58-E74F-42E4-BDC8-ABF94BABC330@gmail.com> On Jun 24, 2011, at 12:58 PM, Torben Hoffmann wrote: > I think you are missing a path to the beam file for exch when you compile tourney. rebar puts all the files into ebin, though. How do I tell rebar to include ebin during compilation? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From torben.lehoff@REDACTED Fri Jun 24 14:14:34 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 24 Jun 2011 14:14:34 +0200 Subject: [erlang-questions] behaviour X undefined In-Reply-To: <67894D58-E74F-42E4-BDC8-ABF94BABC330@gmail.com> References: <3D028942-BFD8-4044-AEC9-704CA0C058D6@gmail.com> <67894D58-E74F-42E4-BDC8-ABF94BABC330@gmail.com> Message-ID: On Fri, Jun 24, 2011 at 14:07, Joel Reymont wrote: > > On Jun 24, 2011, at 12:58 PM, Torben Hoffmann wrote: > > > I think you are missing a path to the beam file for exch when you compile > tourney. > > rebar puts all the files into ebin, though. > > How do I tell rebar to include ebin during compilation? > No clue - my Emakefiles have special lines for the behaviours that are at the top of the list. Surely there must be something like that for rebar... Cheers, Torben > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Fri Jun 24 14:44:56 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 24 Jun 2011 14:44:56 +0200 Subject: [erlang-questions] behaviour X undefined In-Reply-To: References: <3D028942-BFD8-4044-AEC9-704CA0C058D6@gmail.com> <67894D58-E74F-42E4-BDC8-ABF94BABC330@gmail.com> Message-ID: Sorry - forgot to post to the list. On Fri, Jun 24, 2011 at 14:37, Torben Hoffmann wrote: > > > On Fri, Jun 24, 2011 at 14:16, ERLANG wrote: > >> Hi Torben, >> >> >> No clue - my Emakefiles have special lines for the behaviours that are at >>> the top of the list. >>> >> >> >> >> Could you please share one of your Emakefile for learning? >> > > Sure - try to look at: > https://github.com/lehoff/erlware_commons/blob/master/Emakefile > > Cheers, > Torben > > >> >> Thanks >> Zab >> >> >> > > > -- > http://www.linkedin.com/in/torbenhoffmann > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From wsongcn@REDACTED Fri Jun 24 15:10:58 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Fri, 24 Jun 2011 21:10:58 +0800 Subject: [erlang-questions] discrepancy of memory usage figures In-Reply-To: <1308912753.40419.YahooMailNeo@web111411.mail.gq1.yahoo.com> References: <1308912753.40419.YahooMailNeo@web111411.mail.gq1.yahoo.com> Message-ID: I didn't mean commercial support. Thanks for the info though. I'm doing this purely as a hobby. Cheers Andy On Fri, Jun 24, 2011 at 6:52 PM, Thomas Lindgren wrote: > Do you mean contact on commercial terms? I think there are several > organizations on the list who could help you. Off the top of my head, I > believe Erlang/OTP offer support contracts. Erlang Solutions should be happy > to take your call. Mikael Petterson, long-time Hipe guru at Uppsala, might > be available for contracting work through his company (I don't know the > company contact details, I'm afraid). No doubt there are others. > > Best, > Thomas > > ------------------------------ > *From:* Andy W. Song > *To:* Jesper Louis Andersen > *Cc:* erlang-questions > *Sent:* Friday, June 24, 2011 3:57 AM > *Subject:* Re: [erlang-questions] discrepancy of memory usage figures > > Thanks for the suggestions. I left the machine for more than 10 hours and > the figures didn't change. > > I'm out of ideas. How can I approach to experts in stead > of passively waiting on the mailing list? > > -- --------------------------------------------------------------- ???????????????????? ???????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Fri Jun 24 15:11:57 2011 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 24 Jun 2011 14:11:57 +0100 Subject: [erlang-questions] behaviour X undefined In-Reply-To: References: <3D028942-BFD8-4044-AEC9-704CA0C058D6@gmail.com> Message-ID: On Jun 24, 2011, at 12:58 PM, Torben Hoffmann wrote: > I think you are missing a path to the beam file for exch when you compile tourney. Trying by hand... erlc -I ../include -pa ../ebin tourney.erl ./tourney.erl:22: Warning: behaviour exch undefined ls ../ebin/exch.beam ../ebin/exch.beam erlc -I ../include -I ../ebin -pa ../ebin tourney.erl ./tourney.erl:22: Warning: behaviour exch undefined Still doesn't work. Why? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From robert.virding@REDACTED Fri Jun 24 15:27:37 2011 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 24 Jun 2011 13:27:37 +0000 (GMT) Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: Message-ID: <362535126.11771308922057484.JavaMail.root@zimbra> Yes, of course you cannot get around some buffering. It is (should be) easier for the parser to keep track of its buffer as it can parse as it goes and only keep its already parsed data and perhaps that bit in the input buffer which was not enough for a parse, or partial parse. Robert ----- "Max Lapshin" wrote: > On Fri, Jun 24, 2011 at 12:10 AM, Robert Virding > wrote: > > Wouldn't it be better to rewrite the packet parser to work in a > re-entrant fashion so as to be able to span over many packets? It > would save a lot of fiddling with the recbuf size. > > > > You always have to keep buffer: either in receiver, either in parser. > > Easier to use when parser is stateless, but it requires more work > when > you parse again and again old data on receiving new. From juanjo@REDACTED Fri Jun 24 15:32:51 2011 From: juanjo@REDACTED (Juan Jose Comellas) Date: Fri, 24 Jun 2011 10:32:51 -0300 Subject: [erlang-questions] discrepancy of memory usage figures In-Reply-To: References: Message-ID: Have you tried the #erlang IRC channel on Freenode? You'll probably find a lot of Erlang developers there. Juanjo On Thu, Jun 23, 2011 at 10:57 PM, Andy W. Song wrote: > Thanks for the suggestions. I left the machine for more than 10 hours and > the figures didn't change. > > I'm out of ideas. How can I approach to experts in stead > of passively waiting on the mailing list? > > Thanks > Andy > > > On Thu, Jun 23, 2011 at 8:38 PM, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > >> On Thu, Jun 23, 2011 at 05:39, Andy W. Song wrote: >> >> > The figures in red are provided htop, others are output of memory() call >> > from erlang shell. Please look at the total and erlang residence memory. >> > When there is no connection, these two are roughly same, with 100K >> > connections, residence memory is a little larger than total, with 200K >> > connections, residence memory is almost double the total. >> > Can anybody explain? >> >> The first thing I would analyze in this situation is memory >> fragmentation obtained from the underlying memory allocation. If the >> memory fragments, then surely you can see stuff like this happening >> where your resident set size will be larger. I would definitely start >> out by setting the spotlight on the underlying allocator or the >> allocation pattern because as you see, there is no discrepancy if you >> ask the Erlang VM about its view on the matter. What speaks against >> fragmentation is the very young life of the program. Fragmentation >> problems usually only show up in long-running programs where pointers >> are set in stone (making compaction impossible). >> >> Another option is that there is some auxiliary data structure in the >> Erlang VM which are grown up in between the 100K and 200K connection >> count. If the structure is accessed randomly, then surely all the >> Virtual Memory pages will get allocated and assigned to the process, >> increasing the RSS of the program. >> >> It is also interesting to look for what happens if you simply ignore >> the program for a bit of time. Perhaps the allocated memory is not >> released right away, but over time. >> >> Apart from my ideas, I am afraid you will need an Erlang VM expert >> with more intrinsic knowledge than me. >> >> >> -- >> J. >> > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Jun 24 15:56:04 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Fri, 24 Jun 2011 15:56:04 +0200 Subject: [erlang-questions] behaviour X undefined In-Reply-To: <3D028942-BFD8-4044-AEC9-704CA0C058D6@gmail.com> References: <3D028942-BFD8-4044-AEC9-704CA0C058D6@gmail.com> Message-ID: <4E049774.5040500@dev-extend.eu> On 06/24/2011 01:53 PM, Joel Reymont wrote: > Why is this happening? > > make > ==> getopt (compile) > Compiled src/getopt.erl > ==> openpoker (compile) > Compiled src/limit.erl > Compiled src/exch.erl > ... > src/tourney.erl:22: Warning: behaviour exch undefined > > The top of exch.erl looks like this > > -module(exch, [Cbk, Context, Modules]). I don't think an evil parameterized module can be used as a behaviour; the behaviour_info/1 function will be undefined because you're implicitly adding 3 parameters to it. -- Lo?c Hoguin Dev:Extend From joelr1@REDACTED Fri Jun 24 15:59:27 2011 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 24 Jun 2011 14:59:27 +0100 Subject: [erlang-questions] behaviour X undefined In-Reply-To: <4E049774.5040500@dev-extend.eu> References: <3D028942-BFD8-4044-AEC9-704CA0C058D6@gmail.com> <4E049774.5040500@dev-extend.eu> Message-ID: <8D14D0A0-DEBB-4DCD-9CB8-BDCEA0B61B8C@gmail.com> On Jun 24, 2011, at 2:56 PM, Lo?c Hoguin wrote: > I don't think an evil parameterized module can be used as a behaviour; > the behaviour_info/1 function will be undefined because you're > implicitly adding 3 parameters to it. Bummer! :-( It seems to work otherwise, though, in all aspects but the compilation warning. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From torben.lehoff@REDACTED Fri Jun 24 16:03:35 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 24 Jun 2011 16:03:35 +0200 Subject: [erlang-questions] behaviour X undefined In-Reply-To: <8D14D0A0-DEBB-4DCD-9CB8-BDCEA0B61B8C@gmail.com> References: <3D028942-BFD8-4044-AEC9-704CA0C058D6@gmail.com> <4E049774.5040500@dev-extend.eu> <8D14D0A0-DEBB-4DCD-9CB8-BDCEA0B61B8C@gmail.com> Message-ID: On Fri, Jun 24, 2011 at 15:59, Joel Reymont wrote: > > On Jun 24, 2011, at 2:56 PM, Lo?c Hoguin wrote: > > > I don't think an evil parameterized module can be used as a behaviour; > > the behaviour_info/1 function will be undefined because you're > > implicitly adding 3 parameters to it. > > Bummer! :-( > > It seems to work otherwise, though, in all aspects but the compilation > warning. > I just tried to simulate what you were doing with a non-parameterised module and it works like a charm, so this is probably the reason. Cheers, Torben > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Fri Jun 24 18:52:08 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Fri, 24 Jun 2011 12:52:08 -0400 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: <362535126.11771308922057484.JavaMail.root@zimbra> References: <362535126.11771308922057484.JavaMail.root@zimbra> Message-ID: The underlying packet parser is capable of partial parsing, and that's how decode_packet uses it. It's just that the inet driver doesn't seem to call the packet parser the right way to take advantage of partial parsing. --steve On Fri, Jun 24, 2011 at 9:27 AM, Robert Virding wrote: > Yes, of course you cannot get around some buffering. It is (should be) easier for the parser to keep track of its buffer as it can parse as it goes and only keep its already parsed data and perhaps that bit in the input buffer which was not enough for a parse, or partial parse. > > Robert > > > ----- "Max Lapshin" wrote: > >> On Fri, Jun 24, 2011 at 12:10 AM, Robert Virding >> wrote: >> > Wouldn't it be better to rewrite the packet parser to work in a >> re-entrant fashion so as to be able to span over many packets? It >> would save a lot of fiddling with the recbuf size. >> > >> >> You always have to keep buffer: either in receiver, either in parser. >> >> Easier to use when parser is stateless, but it requires more work >> when >> you parse again and again old data on receiving new. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From bob@REDACTED Fri Jun 24 19:05:37 2011 From: bob@REDACTED (Bob Ippolito) Date: Fri, 24 Jun 2011 10:05:37 -0700 Subject: [erlang-questions] how to run PropEr tests from Eunit? In-Reply-To: <93BD232D-C14F-4B16-80D2-96C24EE44BB3@erlang-solutions.com> References: <1308867174.83770.YahooMailRC@web25402.mail.ukl.yahoo.com> <93BD232D-C14F-4B16-80D2-96C24EE44BB3@erlang-solutions.com> Message-ID: This is what I've been using: proper_test_() -> [{atom_to_list(F), fun () -> ?assert(proper:quickcheck(?MODULE:F(), [long_result])) end} || {F, 0} <- ?MODULE:module_info(exports), F > 'prop_', F < 'prop`']. 2011/6/23 Fr?d?ric Trottier-H?bert : > The simplest way I know to do them individually is with separate tests: > proper_test_() -> > ?? ?[?_assert(proper:quickcheck(prop_some_property())), > ?? ? ?_assert(proper:quickcheck(prop_some_property())), > ?? ? ...]. > You can then create macros to do it for you of the form > -define(PROPTEST(A), ?_assert(proper:quickcheck(A()))). > and then call > proper_test_() -> > ?? ?[?PROPTEST(prop_something), ?PROPTEST(prop_something_else), > ?? ? ?PROPTEST(prop_other), ?PROPTEST(prop_last)]. > or whatever you feel like. > If you want to run them all, make sure your module's properties all start > with the prefix 'prop_'. Then you can do something like > proper_test() -> > ?? ?[] = proper:module(?MODULE). > which will find all the properties of the current module and run them for > you. Also, don't forget to always include the proper include file first, > because both PropEr and EUnit define a LET macro, but the PropEr one is the > only one I've seen actively used. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > On 2011-06-23, at 18:12 PM, Roman Shestakov wrote: > > hello, > > I am trying to run PropEr tests from Eunit and I must admit, I can't figure > out how to run them > > > lets say I want to test the following three functions > > %%-------------------------------------------------------------------- > %% @doc > %% convert Date to atom > %% @end > %%-------------------------------------------------------------------- > -spec date_to_atom(date()) -> atom(). > date_to_atom({Year, Month, Day}) -> > ?? list_to_atom(date_to_string({Year, Month, Day})). > > %%-------------------------------------------------------------------- > %% @doc > %% Convert Date to string > %% @end > %%-------------------------------------------------------------------- > -spec date_to_string(date()) -> string(). > date_to_string({Year, Month, Day}) -> > ?? lists:flatten(io_lib:format("~4.10.0B~2.10.0B~2.10.0B", [Year, Month, > Day])). > > > %%-------------------------------------------------------------------- > %% @doc > %% convert Date represented as atom into Erlang date format > %% @end > %%-------------------------------------------------------------------- > -spec atom_to_date(atom()) -> date(). > atom_to_date(Date) -> > ?? Year = list_to_integer(lists:sublist(atom_to_list(Date),1,4)), > ?? Month = list_to_integer(lists:sublist(atom_to_list(Date),5,2)), > ?? Day = list_to_integer(lists:sublist(atom_to_list(Date),7,2)), > ?? {Year, Month, Day}. > > > > with the following property: > > proper_time_to_atom() -> > ?? ?FORALL(Date, date(), > ??? ??? begin > ??? ??? EncDate = ec_time_fns:atom_to_date(ec_time_fns:date_to_atom(Date)), > ??? ??? EncDate =:= Date > ??? ??? end). > > running the following from erl works fine > proper:quickcheck(ec_time_fns_tests:proper_time_to_atom()). > > > but how do I make eunit to run these test? > > > tried this , but it doesn't seem to work > > time_to_atom_test_() -> > ?? ?_test(proper:quickcheck(proper_time_to_atom())). > > > any ideas? > > Regards, Roman > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From romanshestakov@REDACTED Fri Jun 24 20:13:34 2011 From: romanshestakov@REDACTED (Roman Shestakov) Date: Fri, 24 Jun 2011 19:13:34 +0100 (BST) Subject: [erlang-questions] how to run PropEr tests from Eunit? In-Reply-To: References: <1308867174.83770.YahooMailRC@web25402.mail.ukl.yahoo.com> <93BD232D-C14F-4B16-80D2-96C24EE44BB3@erlang-solutions.com> Message-ID: <1308939214.73194.YahooMailRC@web25404.mail.ukl.yahoo.com> thanks a lot Bob, this is very helpful Regards, Roman ________________________________ From: Bob Ippolito To: Fr?d?ric Trottier-H?bert Cc: Roman Shestakov ; erlang-questions@REDACTED Sent: Fri, 24 June, 2011 18:05:37 Subject: Re: [erlang-questions] how to run PropEr tests from Eunit? This is what I've been using: proper_test_() -> [{atom_to_list(F), fun () -> ?assert(proper:quickcheck(?MODULE:F(), [long_result])) end} || {F, 0} <- ?MODULE:module_info(exports), F > 'prop_', F < 'prop`']. 2011/6/23 Fr?d?ric Trottier-H?bert : > The simplest way I know to do them individually is with separate tests: > proper_test_() -> > [?_assert(proper:quickcheck(prop_some_property())), > ?_assert(proper:quickcheck(prop_some_property())), > ...]. > You can then create macros to do it for you of the form > -define(PROPTEST(A), ?_assert(proper:quickcheck(A()))). > and then call > proper_test_() -> > [?PROPTEST(prop_something), ?PROPTEST(prop_something_else), > ?PROPTEST(prop_other), ?PROPTEST(prop_last)]. > or whatever you feel like. > If you want to run them all, make sure your module's properties all start > with the prefix 'prop_'. Then you can do something like > proper_test() -> > [] = proper:module(?MODULE). > which will find all the properties of the current module and run them for > you. Also, don't forget to always include the proper include file first, > because both PropEr and EUnit define a LET macro, but the PropEr one is the > only one I've seen actively used. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > On 2011-06-23, at 18:12 PM, Roman Shestakov wrote: > > hello, > > I am trying to run PropEr tests from Eunit and I must admit, I can't figure > out how to run them > > > lets say I want to test the following three functions > > %%-------------------------------------------------------------------- > %% @doc > %% convert Date to atom > %% @end > %%-------------------------------------------------------------------- > -spec date_to_atom(date()) -> atom(). > date_to_atom({Year, Month, Day}) -> > list_to_atom(date_to_string({Year, Month, Day})). > > %%-------------------------------------------------------------------- > %% @doc > %% Convert Date to string > %% @end > %%-------------------------------------------------------------------- > -spec date_to_string(date()) -> string(). > date_to_string({Year, Month, Day}) -> > lists:flatten(io_lib:format("~4.10.0B~2.10.0B~2.10.0B", [Year, Month, > Day])). > > > %%-------------------------------------------------------------------- > %% @doc > %% convert Date represented as atom into Erlang date format > %% @end > %%-------------------------------------------------------------------- > -spec atom_to_date(atom()) -> date(). > atom_to_date(Date) -> > Year = list_to_integer(lists:sublist(atom_to_list(Date),1,4)), > Month = list_to_integer(lists:sublist(atom_to_list(Date),5,2)), > Day = list_to_integer(lists:sublist(atom_to_list(Date),7,2)), > {Year, Month, Day}. > > > > with the following property: > > proper_time_to_atom() -> > ?FORALL(Date, date(), > begin > EncDate = ec_time_fns:atom_to_date(ec_time_fns:date_to_atom(Date)), > EncDate =:= Date > end). > > running the following from erl works fine > proper:quickcheck(ec_time_fns_tests:proper_time_to_atom()). > > > but how do I make eunit to run these test? > > > tried this , but it doesn't seem to work > > time_to_atom_test_() -> > ?_test(proper:quickcheck(proper_time_to_atom())). > > > any ideas? > > Regards, Roman > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Fri Jun 24 21:29:41 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 24 Jun 2011 21:29:41 +0200 Subject: [erlang-questions] behaviour X undefined In-Reply-To: <4E049774.5040500@dev-extend.eu> References: <3D028942-BFD8-4044-AEC9-704CA0C058D6@gmail.com> <4E049774.5040500@dev-extend.eu> Message-ID: <0D2ED51A-4C80-46B7-A465-20F6CF201A1E@erlang-solutions.com> Actually, it's only one extra parameter? But it still won't work, since the parameterized module compilation inserts a pattern match, so even if you would write: behaviour_info() -> behaviour_info(callbacks). behaviour_info(callbacks) -> ? this would lead to behaviour_info/1 being exported, but it would look something like this: behaviour_info({_,Cbk,Context,Modules} = THIS) -> behaviour_info(callbacks, THIS). behaviour_info(callbacks, {_,Cbk,Context,Modules} = THIS) -> ? so the linter calling exch:behaviour_info(callbacks) would still fail. BR, Ulf W On 24 Jun 2011, at 15:56, Lo?c Hoguin wrote: > On 06/24/2011 01:53 PM, Joel Reymont wrote: >> Why is this happening? >> >> make >> ==> getopt (compile) >> Compiled src/getopt.erl >> ==> openpoker (compile) >> Compiled src/limit.erl >> Compiled src/exch.erl >> ... >> src/tourney.erl:22: Warning: behaviour exch undefined >> >> The top of exch.erl looks like this >> >> -module(exch, [Cbk, Context, Modules]). > > I don't think an evil parameterized module can be used as a behaviour; > the behaviour_info/1 function will be undefined because you're > implicitly adding 3 parameters to it. > > -- > Lo?c Hoguin > Dev:Extend > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From anthonym@REDACTED Fri Jun 24 21:47:41 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Fri, 24 Jun 2011 12:47:41 -0700 Subject: [erlang-questions] Measuring Erlang Communication Overhead? Message-ID: <20110624194741.GA72067@alumni.caltech.edu> Hi, We put together a system which consists of a few thousand processes each of which holds a rule. A request comes in with a context, the context is sent to a subset of the processes (although this could be close to a thousand of them), which combines the context and the rule and sends back a result (both sends are done in a non-blocking fashion and a gen_fsm recieves all the results and collates then). What we see is that this caps out around 100-150 requests per second after which erlang becomes unresponsive and we need to kill it. We have an older server written in java which can do about an order of magnitude more request per second. I've looked over the code and the efficiency guide, and we've also profiled using fprof and not found any low hanging fruit. I'm thinking that maybe the scatter/gather approach might be bad if the context being passed to each process gets large and that instead it might be better to have a few large processes which can run through all rules with the context. However, its a largish undertaking to refactor like this, so I was wondering if there is any way to measure the overhead of communication? I've looked at message queue sizes and they are mostly zero (sometimes a few go up to 1, but nothing huge), but was wondering if there are other things that can be looked at, or tools which could be used to determine whether the limiting constraint is communication overhead (ie, copying around 1K to 400 processes), or whether it is something else. Thanks, -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From jack@REDACTED Fri Jun 24 22:03:05 2011 From: jack@REDACTED (Jack Moffitt) Date: Fri, 24 Jun 2011 14:03:05 -0600 Subject: [erlang-questions] Measuring Erlang Communication Overhead? In-Reply-To: <20110624194741.GA72067@alumni.caltech.edu> References: <20110624194741.GA72067@alumni.caltech.edu> Message-ID: > ?We put together a system which consists of a few thousand processes > each of which holds a rule. ?A request comes in with a context, the > context is sent to a subset of the processes (although this could be > close to a thousand of them), which combines the context and the rule > and sends back a result (both sends are done in a non-blocking fashion > and a gen_fsm recieves all the results and collates then). Since your unit of parallelization is a request, why not have a single process per request, which runs through determining the rules and executing them and returning the result? Does each rule have a lot of state or need information from more than one request? Your request is blocked until the results are all in anyway, so you might as well run all the results in the request's process to begin with. If each rule has lots of its own state, you could use ETS to manage that instead of processes. If rules depend on multiple requests, then I'm not sure what to suggest. jack. From max.lapshin@REDACTED Fri Jun 24 22:22:51 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 25 Jun 2011 00:22:51 +0400 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: References: <362535126.11771308922057484.JavaMail.root@zimbra> Message-ID: I have just thought, that proper web server should always use packet_decode, rather than {mode, http}. I'm currently trying to implement input of POST body as a stream and understood very bad thing: if client send second request very fast after sending first, it may happens so, that post body will come to us in the same packet with beginning of next header. If Content-Length is 2900 bytes, it may be such situation: 1400 bytes of post body 1400 bytes of post body 100 bytes of post body + 1000 bytes of next request. The only way to parse it is to keep buffer in web-server process and use erlang:decode_packet, because the second way is to do blocking gen_tcp:recv From jani.j.hakala@REDACTED Fri Jun 24 22:29:05 2011 From: jani.j.hakala@REDACTED (Jani Hakala) Date: Fri, 24 Jun 2011 23:29:05 +0300 Subject: [erlang-questions] Status of Windows platform support in Erlang/OTP In-Reply-To: (Amy Lear's message of "Sun, 19 Jun 2011 15:53:50 -0700") References: Message-ID: <87k4cbq8ha.fsf@pingviini.priv> Amy Lear writes: > Has anything happened since then as far as efforts to be able to fully > compile Erlang using mingw? The mingw/msys environment does provide a > working shell and the ability to run configure scripts, etc, without > bringing the whole cygwin cavalcade to the party. > One of the problems is that mingw32 header files lack definitions for GetLogicalProcessorInformation, GetLogicalProcessorInformationEx, and structures related to those two functions. I have been able to compile R14B0 using mingw32 but I haven't tested it thoroughly. Jani Hakala From anthonym@REDACTED Fri Jun 24 22:33:33 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Fri, 24 Jun 2011 13:33:33 -0700 Subject: [erlang-questions] Measuring Erlang Communication Overhead? In-Reply-To: References: <20110624194741.GA72067@alumni.caltech.edu> Message-ID: <20110624203333.GA72152@alumni.caltech.edu> On Fri, Jun 24, 2011 at 02:03:05PM -0600, Jack Moffitt wrote: > > ?We put together a system which consists of a few thousand processes > > each of which holds a rule. ?A request comes in with a context, the > > context is sent to a subset of the processes (although this could be > > close to a thousand of them), which combines the context and the rule > > and sends back a result (both sends are done in a non-blocking fashion > > and a gen_fsm recieves all the results and collates then). > > Since your unit of parallelization is a request, why not have a single > process per request, which runs through determining the rules and > executing them and returning the result? That's the gist of my question if you read further on. However, before I invest the time to do that I was wondering if there was a way to determine if it would help. So I have questions like is there a way to measure the cost of communication between processes on a running system, or a way to benchmark this with an existing system. I came across percept and am wondering if it helps on those lines, but I'd also be interested to know about ways to determine this sort of information with things like system_info or process_info. Mostly looking for strategies tools to determine if inter process communication is the issue > Does each rule have a lot of state or need information from more than > one request? Each rule has state and the state can be updated at any point by another process which gets updates from another management system. Now I've not measured memory used just by rules but as they make up the majority of my vms processes and memory() shows about 700M used by processes, if I consolidated all rules into one process I'd end up only being able to have a few of them running on a 24G machine. Now that may be okay as I only have 16 cores, but is a bit of an undertaking (as I have to figure out ways to update all the rules when the updates from the management system happen. > Your request is blocked until the results are all in anyway, so you > might as well run all the results in the request's process to begin > with. If each rule has lots of its own state, you could use ETS to > manage that instead of processes. If rules depend on multiple > requests, then I'm not sure what to suggest. ETS really doesn't help if the problem is copies, you have to copy stuff out of ETS when you look it up, so I'm not sure how that would help, but maybe I'm somehow mistaken there? I'm also wondering if it might not be better to have a module and take each rule and compile it as closure under that module or something like that, then I could assumable call rules:rule12345(Context) in a process and it would be faster than sending to a process and the context wouldn't need to be copied, but I'm not sure how feasible that is, nor if there's a limit to the number of functions in a module I'd run into. -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From vinoski@REDACTED Fri Jun 24 22:35:37 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Fri, 24 Jun 2011 16:35:37 -0400 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: References: <362535126.11771308922057484.JavaMail.root@zimbra> Message-ID: On Fri, Jun 24, 2011 at 4:22 PM, Max Lapshin wrote: > I have just thought, that proper web server should always use > packet_decode, rather than {mode, http}. > > I'm currently trying to implement input of POST body as a stream and > understood very bad thing: > if client send second request very fast after sending first, it may > happens so, that post body will come to > us in the same packet with beginning of next header. The client is broken in that case, since what you're describing is HTTP pipelining and POST cannot be pipelined (nor can other non-idempotent HTTP methods). The client has to wait for the POST reply before sending the next request. --steve From max.lapshin@REDACTED Fri Jun 24 22:44:49 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 25 Jun 2011 00:44:49 +0400 Subject: [erlang-questions] gen_tcp and large HTTP headers In-Reply-To: References: <362535126.11771308922057484.JavaMail.root@zimbra> Message-ID: On Sat, Jun 25, 2011 at 12:35 AM, Steve Vinoski wrote: > > The client is broken in that case, since what you're describing is > HTTP pipelining and POST cannot be pipelined (nor can other > non-idempotent HTTP methods). The client has to wait for the POST > reply before sending the next request. > So, I'm just frightened of a buggy behaviour, that shouldn't be handled and there is no problem. From dmercer@REDACTED Fri Jun 24 23:42:55 2011 From: dmercer@REDACTED (David Mercer) Date: Fri, 24 Jun 2011 16:42:55 -0500 Subject: [erlang-questions] Dialyzer Error: "Analysis failed with error" Message-ID: <000001cc32b7$ae8d94e0$0ba8bea0$@com> Inspired by http://www.erlang-factory.com/upload/presentations/347/Kostis.pdf, I decided to give Dialyzer a go. The first file I tried it on, however, failed with a rather unhelpful error. After trimming a few thousand lines, I've got it down to the following minimal case: -module(dialyzer_test). -export([test/1]). -record(rec1, {f1}). -record(rec2, {f2}). -type rec1(T) :: #rec1{f1 :: T}. -type rec2(T) :: #rec2{f2 :: T}. -spec test(rec2(rec1(A))) -> A. test(#rec2{f2 = #rec1{f1 = X}}) -> X. The Dialyzer error is: dialyzer: Analysis failed with error: Illegal declaration of #rec2{f2} Can anyone explain to me what's wrong with my types and spec? Please advise. Thank-you. David Mercer -------------- next part -------------- An HTML attachment was scrubbed... URL: From jws@REDACTED Sat Jun 25 02:50:35 2011 From: jws@REDACTED (Jeff Schultz) Date: Sat, 25 Jun 2011 10:50:35 +1000 Subject: [erlang-questions] Measuring Erlang Communication Overhead? In-Reply-To: <20110624194741.GA72067@alumni.caltech.edu> References: <20110624194741.GA72067@alumni.caltech.edu> Message-ID: <20110625005035.GA865@mulga.csse.unimelb.edu.au> On Fri, Jun 24, 2011 at 12:47:41PM -0700, Anthony Molinaro wrote: > What we see is that this caps out around 100-150 requests per second > after which erlang becomes unresponsive and we need to kill it. We > have an older server written in java which can do about an order of > magnitude more request per second. I've looked over the code and the > efficiency guide, and we've also profiled using fprof and not found > any low hanging fruit. I'm thinking that maybe the scatter/gather > approach might be bad if the context being passed to each process gets > large and that instead it might be better to have a few large processes > which can run through all rules with the context. However, its a largish > undertaking to refactor like this, so I was wondering if there is any > way to measure the overhead of communication? By "erlang becomes unresponsive and we need to kill it" do you mean that the VM doesn't recover when you stop sending requests? That's not a performance problem :-( > I've looked at message queue sizes and they are mostly zero (sometimes > a few go up to 1, but nothing huge), but was wondering if there are other > things that can be looked at, or tools which could be used to determine > whether the limiting constraint is communication overhead (ie, copying > around 1K to 400 processes), or whether it is something else. One thing that might be easy to do is make the communication bigger and see if you reach the performance limit earlier. Either make the messages bigger (perhaps doubling them, but not by just sending {Context, Context} because the implementation may preserve the sharing) or send each message twice ( P ! {look_at Context}, P ! {really_do Context} with the look_at ones read and ignored.) Jeff Schultz From octopusfluff@REDACTED Sat Jun 25 06:27:32 2011 From: octopusfluff@REDACTED (Amy Lear) Date: Fri, 24 Jun 2011 21:27:32 -0700 Subject: [erlang-questions] Status of Windows platform support in Erlang/OTP In-Reply-To: <87k4cbq8ha.fsf@pingviini.priv> References: <87k4cbq8ha.fsf@pingviini.priv> Message-ID: On Fri, Jun 24, 2011 at 1:29 PM, Jani Hakala wrote: > One of the problems is that mingw32 header files lack definitions for > GetLogicalProcessorInformation, GetLogicalProcessorInformationEx, and > structures related to those two functions. Interesting. Has the mingw32 team been approached on this deficiency? > I have been able to compile R14B0 using mingw32 but I haven't tested it > thoroughly. Did it take extensive changes? From kostis@REDACTED Sat Jun 25 09:32:15 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 25 Jun 2011 10:32:15 +0300 Subject: [erlang-questions] Dialyzer Error: "Analysis failed with error" In-Reply-To: <000001cc32b7$ae8d94e0$0ba8bea0$@com> References: <000001cc32b7$ae8d94e0$0ba8bea0$@com> Message-ID: <4E058EFF.8000807@cs.ntua.gr> David Mercer wrote: > Inspired by > http://www.erlang-factory.com/upload/presentations/347/Kostis.pdf, I > decided to give Dialyzer a go. The first file I tried it on, however, > failed with a rather unhelpful error. After trimming a few thousand > lines, I?ve got it down to the following minimal case: > > -module(dialyzer_test). > > -export([test/1]). > > -record(rec1, {f1}). > -record(rec2, {f2}). > > -type rec1(T) :: #rec1{f1 :: T}. > -type rec2(T) :: #rec2{f2 :: T}. > > -spec test(rec2(rec1(A))) -> A. > > test(#rec2{f2 = #rec1{f1 = X}}) -> X. > > The Dialyzer error is: > > dialyzer: Analysis failed with error: Illegal declaration of #rec2{f2} > > Can anyone explain to me what?s wrong with my types and spec? Please > advise. Thank-you. In principle there is nothing wrong with your types and specs. It seems you have hit a limitation (bug?) which makes parametric types involving records not handled properly by Erlang/OTP. (Record expressions are nasty, because they are handled by the preprocessor.) I'll try to see what can be done about it for R14B04. In the meantime, you can bypass this error by declaring the type you want as follows: -type rec2rec1(T) :: {rec2, rec1(T)}. -spec test(rec2rec1(A)) -> A. Far from ideal, I know... Thanks for the minimal test case! Kostis From jani.j.hakala@REDACTED Sat Jun 25 10:46:21 2011 From: jani.j.hakala@REDACTED (Jani Hakala) Date: Sat, 25 Jun 2011 11:46:21 +0300 Subject: [erlang-questions] Status of Windows platform support in Erlang/OTP In-Reply-To: (Amy Lear's message of "Fri, 24 Jun 2011 21:27:32 -0700") References: <87k4cbq8ha.fsf@pingviini.priv> Message-ID: <8739iyqowy.fsf@pingviini.priv> Amy Lear writes: > > Interesting. Has the mingw32 team been approached on this deficiency? > At least I haven't approached. I think I just copied the definitions from mingw-w64-headers. >> I have been able to compile R14B0 using mingw32 but I haven't tested it >> thoroughly. > > Did it take extensive changes? > Not in the actual code, most of the changes were related to Makefiles etc., http://users.jyu.fi/~jahakala/R14B-mingw32.diff Jani Hakala From mikpe@REDACTED Sat Jun 25 11:35:58 2011 From: mikpe@REDACTED (Mikael Pettersson) Date: Sat, 25 Jun 2011 11:35:58 +0200 Subject: [erlang-questions] Status of Windows platform support in Erlang/OTP In-Reply-To: <87k4cbq8ha.fsf@pingviini.priv> References: <87k4cbq8ha.fsf@pingviini.priv> Message-ID: <19973.44030.870151.449736@pilspetsen.it.uu.se> Jani Hakala writes: > Amy Lear writes: > > > Has anything happened since then as far as efforts to be able to fully > > compile Erlang using mingw? The mingw/msys environment does provide a > > working shell and the ability to run configure scripts, etc, without > > bringing the whole cygwin cavalcade to the party. > > > One of the problems is that mingw32 header files lack definitions for > GetLogicalProcessorInformation, GetLogicalProcessorInformationEx, and > structures related to those two functions. The mingw-w64 headers include those definitions. A work I've ditched the old mingw32 stuff and now build both 32 and 64-bit mingw GCCs with the mingw-w64 headers and libraries. From nick.sfx.1@REDACTED Sat Jun 25 13:39:57 2011 From: nick.sfx.1@REDACTED (Nick S) Date: Sat, 25 Jun 2011 14:39:57 +0300 Subject: [erlang-questions] Debug help Message-ID: Hi, I would like to write a separate module, which would catch/indicate, when some functions are executed in other modules. For example, otp/gen_server handle_call(new_msg,...) %% do not catch this line handle_call(....) -> %% catch this line i.e. when all above conditions doesn't match. Which debug tool would be best for my needs? Also would like to include some timing measurements, later on. Nick. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobias.rodaebel@REDACTED Sat Jun 25 15:55:29 2011 From: tobias.rodaebel@REDACTED (Tobias) Date: Sat, 25 Jun 2011 15:55:29 +0200 Subject: [erlang-questions] Debug help In-Reply-To: References: Message-ID: <095F5153-1CFC-4AB0-974C-9F9019CD08C2@googlemail.com> Hi Nick, For such a purpose, I'd take gen_event [1] and/or the dbg module [2], and erlang:statistics might also be of some help here. Since I'm quite new to Erlang, I'd be very interested in more/other suggestions, too. [1] http://www.erlang.org/doc/design_principles/events.html [2] http://stackoverflow.com/questions/1954894/using-trace-and-dbg-in-erlang/1954980#1954980 Cheers, Tobias On Jun 25, 2011, at 1:39 PM, Nick S wrote: > Hi, > > I would like to write a separate module, which would catch/indicate, when some functions are executed in other modules. > > For example, > > otp/gen_server > > handle_call(new_msg,...) %% do not catch this line > > handle_call(....) -> %% catch this line i.e. when all above conditions doesn't match. > > Which debug tool would be best for my needs? > Also would like to include some timing measurements, later on. > > Nick. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From joelr1@REDACTED Sun Jun 26 14:30:22 2011 From: joelr1@REDACTED (Joel Reymont) Date: Sun, 26 Jun 2011 13:30:22 +0100 Subject: [erlang-questions] eunit in common test Message-ID: <19DA9B0A-1409-4C1E-AC5C-19C1176BFA35@gmail.com> What is the benefit of using eunit in common test suites? I see this done but don't see the point. Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From banibrata.dutta@REDACTED Sun Jun 26 15:51:43 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Sun, 26 Jun 2011 19:21:43 +0530 Subject: [erlang-questions] eunit in common test In-Reply-To: <19DA9B0A-1409-4C1E-AC5C-19C1176BFA35@gmail.com> References: <19DA9B0A-1409-4C1E-AC5C-19C1176BFA35@gmail.com> Message-ID: On Sun, Jun 26, 2011 at 6:00 PM, Joel Reymont wrote: > What is the benefit of using eunit in common test suites? > > I see this done but don't see the point. > Ref: http://www.erlang.org/doc/apps/common_test/cover_chapter.html#cover Although Common Test was created primarly for the purpose of black box testing, nothing prevents it from working perfectly as a white box testing tool as well. This is especially true when the application to test is written in Erlang. Then the test ports are easily realized by means of Erlang function calls. Possibly leverage a common reporting framework ? Possibly make it easy to separate the nightly-build and automated (whitebox) test setup. I'd believe this is one of the several alternatives that developers have. -------------- next part -------------- An HTML attachment was scrubbed... URL: From greim@REDACTED Sun Jun 26 18:14:05 2011 From: greim@REDACTED (greim) Date: Sun, 26 Jun 2011 18:14:05 +0200 Subject: [erlang-questions] Where is embedded-erlang.org Message-ID: <4E075ACD.1050704@schleibinger.com> In February i downloaded Erlang for ARM from embedded-erlang.org . I tried to reach this page again, but i can't open it anymore. Has naybody any info about it? Regards Markus Greim From pfisher@REDACTED Sun Jun 26 20:20:31 2011 From: pfisher@REDACTED (Paul Fisher) Date: Sun, 26 Jun 2011 13:20:31 -0500 Subject: [erlang-questions] OTP roadmap slides from Erlang Factory? Message-ID: Kenneth, Can you publish your slides from your Erlang Factory talk on the OTP Roadmap? I apologize if you have already done so, I was looking on the Erlang Factory site and a link was not available. -- paul director, platform services alertlogic, inc. 713-484-8383 x2314 From francesco@REDACTED Sun Jun 26 21:07:24 2011 From: francesco@REDACTED (Francesco Cesarini) Date: Sun, 26 Jun 2011 20:07:24 +0100 Subject: [erlang-questions] Where is embedded-erlang.org In-Reply-To: <4E075ACD.1050704@schleibinger.com> References: <4E075ACD.1050704@schleibinger.com> Message-ID: <46781336-6b66-4655-895d-43adace5c7e7@email.android.com> It should be http://www.embedded-erlang.com Regards, F -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. greim wrote: In February i downloaded Erlang for ARM from embedded-erlang.org . I tried to reach this page again, but i can't open it anymore. Has naybody any info about it? Regards Markus Greim _____________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From wmacgyver@REDACTED Sun Jun 26 21:45:05 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Sun, 26 Jun 2011 15:45:05 -0400 Subject: [erlang-questions] Where is embedded-erlang.org In-Reply-To: <46781336-6b66-4655-895d-43adace5c7e7@email.android.com> References: <4E075ACD.1050704@schleibinger.com> <46781336-6b66-4655-895d-43adace5c7e7@email.android.com> Message-ID: Hmm, that url isn't working for me either. On Jun 26, 2011, at 3:07 PM, Francesco Cesarini wrote: > It should be http://www.embedded-erlang.com > > Regards, > F > -- > Sent from my Android phone with K-9 Mail. Please excuse my brevity. > > greim wrote: > In February i downloaded Erlang for ARM from embedded-erlang.org . > I tried to reach this page again, but i can't open it anymore. > > Has naybody any info about it? > > Regards > > Markus Greim > > > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From joseph.stewart@REDACTED Sun Jun 26 21:48:12 2011 From: joseph.stewart@REDACTED (Joseph Stewart) Date: Sun, 26 Jun 2011 15:48:12 -0400 Subject: [erlang-questions] Where is embedded-erlang.org In-Reply-To: References: <4E075ACD.1050704@schleibinger.com> <46781336-6b66-4655-895d-43adace5c7e7@email.android.com> Message-ID: Perhaps this is what you meant: http://www.erlang-embedded.com/ -joe On Sun, Jun 26, 2011 at 3:45 PM, Wilson MacGyver wrote: > Hmm, that url isn't working for me either. > > On Jun 26, 2011, at 3:07 PM, Francesco Cesarini < > francesco@REDACTED> wrote: > > It should be > http://www.embedded-erlang.com > > Regards, > F > -- > Sent from my Android phone with K-9 Mail. Please excuse my brevity. > > greim wrote: >> >> In February i downloaded Erlang for ARM from embedded-erlang.org . >> I tried to reach this page again, but i can't open it anymore. >> >> Has naybody any info about it? >> >> Regards >> >> Markus Greim >> >> ------------------------------ >> >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ > > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From acton@REDACTED Sun Jun 26 22:05:41 2011 From: acton@REDACTED (Brian Acton) Date: Sun, 26 Jun 2011 13:05:41 -0700 Subject: [erlang-questions] Trouble starting mnesia on R14B02/3 node with a R14B01 node connected In-Reply-To: References: Message-ID: Hi, We are similarly affected by this issue. We have a cluster of nodes running stock R14B01 We upgraded one node from R14B01 to R14B03. When we start mnesia, we get the same 'Failed to Merge Schema' error message that Ferenc got when he tried. Is there a solution for this? Does someone have a patch somewhere on github? For us, we can not take the downtime to convert the whole cluster at one time. We need to be able to restart one server at a time. Any info, pointers, or help appreciated. Thx, --b On Mon, May 30, 2011 at 6:17 AM, Ferenc Holzhauser < ferenc.holzhauser@REDACTED> wrote: > > Yes, I did try R14B03 first then R14B02 to see if that works. > > BR, > Ferenc > > > On 30 May 2011 15:00, Dan Gudmundsson wrote: > >> That is a bug in R14B03, I currently fail to understand how it can go >> wrong on R14B02.. >> >> Did you try R14B03 first? >> There is a missing make rule, which doesn't trigger a remake if the .hrl >> file is updated, >> old clearcase problem which fixed those kind of build dependencies >> automagicly. >> >> /Dan >> >> On Mon, May 30, 2011 at 11:59 AM, Ferenc Holzhauser < >> ferenc.holzhauser@REDACTED> wrote: >> >>> Hi, >>> >>> After updating Erlang from R14B01 to R14B02 (or B03, tried both) on one >>> node of a 2 node setup I had trouble starting mnesia. >>> >>> Eshell V5.8.3 (abort with >>> ^G) >>> >>> (node2)1> mnesia:start(). >>> >>> >>> =ERROR REPORT==== 26-May-2011::13:54:06 >>> === >>> Mnesia('node2'): ** ERROR ** (core dumped to file: >>> "/home/me/backend/MnesiaCore.node2_1306_410846_451973") >>> ** FATAL ** Failed to merge schema: >>> {aborted, >>> >>> {function_clause, >>> >>> [{mnesia_schema,cs2list, >>> >>> [{cstruct,schema,set,[], >>> ['node2', >>> 'node1'], >>> >>> [],0,read_write,[],[],false,schema, >>> >>> [table,cstruct], >>> >>> [],[], >>> >>> {{1300,896724,827143}, >>> 'node1'}, >>> >>> {{4,0}, >>> {'node2', >>> >>> {1302,780884,609311}}}}]}, >>> >>> {mnesia_schema,do_merge_schema,1}, >>> >>> {mnesia_tm,apply_fun,3}, >>> >>> {mnesia_tm,execute_transaction,5}, >>> >>> {mnesia_schema,schema_coordinator, >>> >>> 3}]}} >>> >>> >>> >>> At the end I stopped both nodes, updated Erlang then I could start both >>> of them. >>> >>> Is it a bug or did I do something wrong when trying to update them one by >>> one? >>> >>> Thanks in advance, >>> Ferenc >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Mon Jun 27 01:51:30 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 27 Jun 2011 00:51:30 +0100 Subject: [erlang-questions] incremental parsing Message-ID: <5BC1A1AD-A320-4D16-BE68-EF08D47B3899@gmail.com> Assuming an editor backend written in Erlang, any suggestions on how to implement incremental parsing of Erlang code? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From wsongcn@REDACTED Mon Jun 27 06:07:58 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Mon, 27 Jun 2011 12:07:58 +0800 Subject: [erlang-questions] question about Variable:fun() Message-ID: Hi, New to Erlang. I saw some program with statement such as V:fun() which I can't find from Joe's Erlang book. I know module:fun() is calling a function defined in a module. But do this to a variable is confusing. Does that variable define a function, and how? Thanks Andy -------------- next part -------------- An HTML attachment was scrubbed... URL: From jack@REDACTED Mon Jun 27 06:35:20 2011 From: jack@REDACTED (Jack Moffitt) Date: Sun, 26 Jun 2011 22:35:20 -0600 Subject: [erlang-questions] question about Variable:fun() In-Reply-To: References: Message-ID: > New to Erlang. I saw some program with statement such as V:fun() which I > can't find from Joe's Erlang book. I know module:fun() is calling a function > defined in a module. But do this to a variable is confusing. Does that > variable define a function, and how? It is probably bound to an atom, like in this example: X = dict, D = X:new(). jack. From wsongcn@REDACTED Mon Jun 27 07:28:58 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Mon, 27 Jun 2011 13:28:58 +0800 Subject: [erlang-questions] question about Variable:fun() In-Reply-To: References: Message-ID: In my case the variable is a record like V = {rec, field1, field2} where rec is indeed a module name with fun() defined in it. So it seems that calling to this record will in turn call the first element? Thanks Andy On Mon, Jun 27, 2011 at 12:35 PM, Jack Moffitt wrote: > > New to Erlang. I saw some program with statement such as V:fun() which I > > can't find from Joe's Erlang book. I know module:fun() is calling a > function > > defined in a module. But do this to a variable is confusing. Does that > > variable define a function, and how? > > It is probably bound to an atom, like in this example: > > X = dict, > D = X:new(). > > jack. > -- --------------------------------------------------------------- ???????????????????? ???????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From wsongcn@REDACTED Mon Jun 27 08:37:44 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Mon, 27 Jun 2011 14:37:44 +0800 Subject: [erlang-questions] question about Variable:fun() In-Reply-To: References: Message-ID: I guess I found the answer, see following code -module(abc). -export([joke/0, test_fun/1]). test_fun({abc, _, _}) -> io:format("Great, ~n"). joke() -> V = {abc, "tst", "filed2"}, V:test_fun(). If I call abc:joke() it will eventaully call test_fun/1, with V as the parameter. I would like to see some official documentation though. Thanks Andy On Mon, Jun 27, 2011 at 1:28 PM, Andy W. Song wrote: > In my case the variable is a record like V = {rec, field1, field2} where > rec is indeed a module name with fun() defined in it. So it seems that > calling to this record will in turn call the first element? > > Thanks > Andy > > > On Mon, Jun 27, 2011 at 12:35 PM, Jack Moffitt wrote: > >> > New to Erlang. I saw some program with statement such as V:fun() which I >> > can't find from Joe's Erlang book. I know module:fun() is calling a >> function >> > defined in a module. But do this to a variable is confusing. Does that >> > variable define a function, and how? >> >> It is probably bound to an atom, like in this example: >> >> X = dict, >> D = X:new(). >> >> jack. >> > > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > -- --------------------------------------------------------------- ???????????????????? ???????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Mon Jun 27 08:49:26 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 27 Jun 2011 08:49:26 +0200 Subject: [erlang-questions] incremental parsing In-Reply-To: <5BC1A1AD-A320-4D16-BE68-EF08D47B3899@gmail.com> References: <5BC1A1AD-A320-4D16-BE68-EF08D47B3899@gmail.com> Message-ID: Hi Joel, On Mon, Jun 27, 2011 at 01:51, Joel Reymont wrote: > Assuming an editor backend written in Erlang, > any suggestions on how to implement incremental parsing of Erlang code? We're doing a somewhat incremental parsing in erlide. The idea is that you keep track of the token stream created by the lexer and when there is a change in the text, you know that only some tokens can be affected: in the worst case all tokens following the place where the change was, but if you detect a token that is identical to what it was before then you don't need to continue. In a similar way, the syntax tree needs to keep track of the tokens it is created from and when there are changes only these parts need to get parsed again.This works ok when there are no syntax errors (which happen all the time while typing, until one is done). We try to be smart about that, for example keep track of all the forms in the module and not directly let the retokenization pass the form boundary: if the user starts typing a string literal, we don't want the rest of the file to become marked as a string, but we guess that the string will be ended in the same form that it was started in. I said we do "somewhat incremental" parsing because we are using coarser granularity than possible (i.e. at the form level). We found it's fast enough and the additional work isn't worth it. The parser would need deeper changes and I don't like having to maintain a parallel lexer and parser. The code is at https://github.com/erlide/erlide/tree/master/org.erlide.kernel.ide/src/, in erlide_scan.erl and the parser/ directory, hopefully it's understandable. If not and you want to know more, please ask. regards, Vlad From qrilka@REDACTED Mon Jun 27 09:02:01 2011 From: qrilka@REDACTED (Kirill Zaborsky) Date: Mon, 27 Jun 2011 11:02:01 +0400 Subject: [erlang-questions] question about Variable:fun() In-Reply-To: References: Message-ID: >From official docs: ExprF(Expr1,...,ExprN) ExprM:ExprF(Expr1,...,ExprN) In the first form of function calls, ExprM:ExprF(Expr1,...,ExprN), each of ExprM and ExprF must be an atom or an expression that evaluates to an atom. (http://www.erlang.org/doc/reference_manual/expressions.html#id75726) Though it is not clear what "evaluates to atom" may mean for a tuple. Kind regards, Kirill Zaborsky 2011/6/27 Andy W. Song > I guess I found the answer, see following code > -module(abc). > -export([joke/0, test_fun/1]). > > test_fun({abc, _, _}) -> > io:format("Great, ~n"). > > joke() -> > V = {abc, "tst", "filed2"}, > V:test_fun(). > > If I call abc:joke() it will eventaully call test_fun/1, with V as the > parameter. I would like to see some official documentation though. > > Thanks > Andy > > On Mon, Jun 27, 2011 at 1:28 PM, Andy W. Song wrote: > >> In my case the variable is a record like V = {rec, field1, field2} where >> rec is indeed a module name with fun() defined in it. So it seems that >> calling to this record will in turn call the first element? >> >> Thanks >> Andy >> >> >> On Mon, Jun 27, 2011 at 12:35 PM, Jack Moffitt wrote: >> >>> > New to Erlang. I saw some program with statement such as V:fun() which >>> I >>> > can't find from Joe's Erlang book. I know module:fun() is calling a >>> function >>> > defined in a module. But do this to a variable is confusing. Does that >>> > variable define a function, and how? >>> >>> It is probably bound to an atom, like in this example: >>> >>> X = dict, >>> D = X:new(). >>> >>> jack. >>> >> >> >> >> -- >> --------------------------------------------------------------- >> ???????????????????? >> ???????????????????? >> >> > > > -- > --------------------------------------------------------------- > ???????????????????? > ???????????????????? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpe@REDACTED Mon Jun 27 09:48:54 2011 From: mikpe@REDACTED (Mikael Pettersson) Date: Mon, 27 Jun 2011 09:48:54 +0200 Subject: [erlang-questions] question about Variable:fun() In-Reply-To: References: Message-ID: <19976.13798.893259.929824@pilspetsen.it.uu.se> On Mon, 27 Jun 2011 14:37:44 +0800, "Andy W. Song" wrote: > I guess I found the answer, see following code > -module(abc). > -export([joke/0, test_fun/1]). > > test_fun({abc, _, _}) -> > io:format("Great, ~n"). > > joke() -> > V = {abc, "tst", "filed2"}, > V:test_fun(). > > If I call abc:joke() it will eventaully call test_fun/1, with V as the > parameter. I would like to see some official documentation though. The V call looks like a fake parameterized module. They should be documented, perhaps as an experimental extension, however the specific usage with an explicit tuple doesn't look quite kosher. From paolo.negri@REDACTED Mon Jun 27 11:30:44 2011 From: paolo.negri@REDACTED (Paolo Negri) Date: Mon, 27 Jun 2011 11:30:44 +0200 Subject: [erlang-questions] Half word emulators and NIFs Message-ID: Dear list, We're running some test using the half word emulator and we're seeing few regressions. Our application runs with no problem on 32 and 64 VMs and regressions are specific to this emulator. While we're still looking at shrinking the broken test in order to identify exactly the failure, so far it seems - but we're not sure yet - that regressions are due to some interaction with a NIF [1] The behavior we're observing is that function clauses in pure erlang code (code not involving NIF calls) stop matching after the invocation of such a NIF. So far we couldn't build very simple ad hoc test cases to reproduce the failure, while more elaborated ones part of our application tests are consistent and always fail (while always pass using plain 32 or 64 bit VMs). All function clauses that we seem to "lose" are very simple matches like integers and atoms. What I would like to ask is - Is it possible for a NIF to somehow break such basic features of the VM because of the NIF non compliance with the half word machine? - How is it possible to recognize if a NIF is or not safe to be used with the half word emulator? While cutting out the minimal piece of code to reproduce the failure in order to publish it might take us some time, we're open to provide access to an environment where is possible to consistently reproduce the failure if someone is interested into looking into this matter. [1] https://github.com/davisp/jiffy Thanks in advance for your answers, Paolo From ingela@REDACTED Mon Jun 27 11:45:46 2011 From: ingela@REDACTED (Ingela Andin) Date: Mon, 27 Jun 2011 11:45:46 +0200 Subject: [erlang-questions] odbc blob datatype In-Reply-To: <1308916507.7555.7.camel@Ubuntu-Juan> References: <1308916507.7555.7.camel@Ubuntu-Juan> Message-ID: Hi! Well it is mapped to SQL_WLONGVARCHAR which has been unsupported but we have a patch on its way in that will add support for SQL_WLONGVARCHAR. See my github repository git@REDACTED:IngelaAndin/otp.git branch hs/odbc/sql_wlongvarchar_support Regards Ingela Erlang/OTP team - Ericsson AB 2011/6/24 Juan Antonio Berrayarza Lanz? : > Hi : > > I'm trying to access a table in Mysql that has TEXT fields. > when i do a odbc:describe_table() i get > {"field_name",ODBC_UNSUPPORTED_TYPE}. > > Reading documentation about erlang odbc, i see that TEXT fields should > be mapped to SQL_LONGVARCHAR. > > Has anyone worked this out? how can i query tables with TEXT fields > using odbc and erlang ? > > > erlang : R14B02 > odbc: 2.10.10 > mysql: 4.1.49 > > odbc mysql driver : 5.1.8 > > Tx to all of you > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From dangud@REDACTED Mon Jun 27 10:06:56 2011 From: dangud@REDACTED (Dan Gudmundsson) Date: Mon, 27 Jun 2011 10:06:56 +0200 Subject: [erlang-questions] Trouble starting mnesia on R14B02/3 node with a R14B01 node connected In-Reply-To: References: Message-ID: You have to back out Ulf majority patches, from R14B03 branch I have not made it yet but will do in R14B04, plus add some code so we can introduce the functionality in R15. Sorry I messed up the testing on that patch, it shouldn't have been included since both I and Ulf miss-interpreted how the schema upgrade code worked, it didn't. /Dan On Sun, Jun 26, 2011 at 10:05 PM, Brian Acton wrote: > Hi, > > We are similarly affected by this issue. > > We have a cluster of nodes running stock R14B01 > > We upgraded one node from R14B01 to R14B03. > > When we start mnesia, we get the same 'Failed to Merge Schema' error > message that Ferenc got when he tried. > > Is there a solution for this? Does someone have a patch somewhere on > github? > > For us, we can not take the downtime to convert the whole cluster at one > time. We need to be able to restart one server at a time. > > Any info, pointers, or help appreciated. > > Thx, > > --b > > > > On Mon, May 30, 2011 at 6:17 AM, Ferenc Holzhauser < > ferenc.holzhauser@REDACTED> wrote: > >> >> Yes, I did try R14B03 first then R14B02 to see if that works. >> >> BR, >> Ferenc >> >> >> On 30 May 2011 15:00, Dan Gudmundsson wrote: >> >>> That is a bug in R14B03, I currently fail to understand how it can go >>> wrong on R14B02.. >>> >>> Did you try R14B03 first? >>> There is a missing make rule, which doesn't trigger a remake if the .hrl >>> file is updated, >>> old clearcase problem which fixed those kind of build dependencies >>> automagicly. >>> >>> /Dan >>> >>> On Mon, May 30, 2011 at 11:59 AM, Ferenc Holzhauser < >>> ferenc.holzhauser@REDACTED> wrote: >>> >>>> Hi, >>>> >>>> After updating Erlang from R14B01 to R14B02 (or B03, tried both) on one >>>> node of a 2 node setup I had trouble starting mnesia. >>>> >>>> Eshell V5.8.3 (abort with >>>> ^G) >>>> >>>> (node2)1> mnesia:start(). >>>> >>>> >>>> =ERROR REPORT==== 26-May-2011::13:54:06 >>>> === >>>> Mnesia('node2'): ** ERROR ** (core dumped to file: >>>> "/home/me/backend/MnesiaCore.node2_1306_410846_451973") >>>> ** FATAL ** Failed to merge schema: >>>> {aborted, >>>> >>>> {function_clause, >>>> >>>> [{mnesia_schema,cs2list, >>>> >>>> [{cstruct,schema,set,[], >>>> ['node2', >>>> 'node1'], >>>> >>>> [],0,read_write,[],[],false,schema, >>>> >>>> [table,cstruct], >>>> >>>> [],[], >>>> >>>> {{1300,896724,827143}, >>>> 'node1'}, >>>> >>>> {{4,0}, >>>> {'node2', >>>> >>>> {1302,780884,609311}}}}]}, >>>> >>>> {mnesia_schema,do_merge_schema,1}, >>>> >>>> {mnesia_tm,apply_fun,3}, >>>> >>>> {mnesia_tm,execute_transaction,5}, >>>> >>>> {mnesia_schema,schema_coordinator, >>>> >>>> 3}]}} >>>> >>>> >>>> >>>> At the end I stopped both nodes, updated Erlang then I could start both >>>> of them. >>>> >>>> Is it a bug or did I do something wrong when trying to update them one >>>> by one? >>>> >>>> Thanks in advance, >>>> Ferenc >>>> >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker@REDACTED Mon Jun 27 12:03:15 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Mon, 27 Jun 2011 12:03:15 +0200 Subject: [erlang-questions] Half word emulators and NIFs In-Reply-To: References: Message-ID: <4E085563.1050106@erix.ericsson.se> Have you tested with a debug built emulator. That may catch faulty NIF behaviors earlier. Otherwise if a NIF builds broken terms on the process heap it might not be discovered until later when those terms are matched or maybe garbage collected. There is no difference to make NIF's for the halfword emulator. Just use the NIF API as intended and don't cheat (make assumptions about sizeof(ERL_NIF_TERM) for example). /Sverker, Erlang/OTP Paolo Negri wrote: > Dear list, > > We're running some test using the half word emulator and we're seeing > few regressions. > Our application runs with no problem on 32 and 64 VMs and regressions > are specific to this emulator. > While we're still looking at shrinking the broken test in order to > identify exactly the failure, so far it seems - but we're not sure yet > - that regressions are due to some interaction with a NIF [1] > The behavior we're observing is that function clauses in pure erlang > code (code not involving NIF calls) stop matching after the invocation > of such a NIF. > So far we couldn't build very simple ad hoc test cases to reproduce > the failure, while more elaborated ones part of our application tests > are consistent and always fail (while always pass using plain 32 or 64 > bit VMs). > All function clauses that we seem to "lose" are very simple matches > like integers and atoms. > What I would like to ask is > > - Is it possible for a NIF to somehow break such basic features of the > VM because of the NIF non compliance with the half word machine? > - How is it possible to recognize if a NIF is or not safe to be used > with the half word emulator? > > While cutting out the minimal piece of code to reproduce the failure > in order to publish it might take us some time, we're open to provide > access to an environment where is possible to consistently reproduce > the failure if someone is interested into looking into this matter. > > [1] https://github.com/davisp/jiffy > > Thanks in advance for your answers, > > Paolo > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From toby@REDACTED Mon Jun 27 14:24:44 2011 From: toby@REDACTED (Toby Thain) Date: Mon, 27 Jun 2011 08:24:44 -0400 Subject: [erlang-questions] incremental parsing In-Reply-To: References: <5BC1A1AD-A320-4D16-BE68-EF08D47B3899@gmail.com> Message-ID: <4E08768C.50801@telegraphics.com.au> On 27/06/11 2:49 AM, Vlad Dumitrescu wrote: > Hi Joel, > > On Mon, Jun 27, 2011 at 01:51, Joel Reymont wrote: >> Assuming an editor backend written in Erlang, >> any suggestions on how to implement incremental parsing of Erlang code? > > We're doing a somewhat incremental parsing in erlide. The idea is that > you keep track of the token stream created by the lexer and when there > is a change in the text, you know that only some tokens can be > affected: in the worst case all tokens following the place where the > change was, but if you detect a token that is identical to what it was > before then you don't need to continue. In a similar way, the syntax > tree needs to keep track of the tokens it is created from and when > there are changes only these parts need to get parsed again.This works > ok when there are no syntax errors (which happen all the time while > typing, until one is done). We try to be smart about that, for example > keep track of all the forms in the module and not directly let the > retokenization pass the form boundary: if the user starts typing a > string literal, we don't want the rest of the file to become marked as > a string, but we guess that the string will be ended in the same form > that it was started in. I wish all syntax highlighting editors were clever like this. --Toby > > I said we do "somewhat incremental" parsing because we are using > coarser granularity than possible (i.e. at the form level). We found > it's fast enough and the additional work isn't worth it. ... > > regards, > Vlad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From v@REDACTED Mon Jun 27 16:12:02 2011 From: v@REDACTED (Valentin Micic) Date: Mon, 27 Jun 2011 16:12:02 +0200 Subject: [erlang-questions] Sending a tcp packet containing null data Message-ID: <97A9D51D-9FBD-4825-8BC8-3F121BF489E9@pharos-avantgard.com> Hi I'd like to implement a generic link-test facility for TCP connections -- something that would work similarly to TCP keep-alive; however, under complete control of the calling ERLANG process with respect to sending and receiving of control packets. By the same token, I'd like this test to be transparent (as in: non-intrusive) to the peer application. In other words, I would *not* like such an empty (null data) packet to be propagated to the remote peer application, but expect the remote TCP stack to issue an equally empty "ACK" packet (or stream of data, if available), which would be propagated back to the caller. Thus, I have a number of problems on multiple levels, but first and foremost, I have no idea how to achieve this (other than hoping that TCP keep-alive would be supported on a particular machine) :-) So, how can I force a null data packet to leave the local machine. Did try gen_tcp:send(Socket, <<>>), however, this didn't work (well... why should it?) Maybe the whole problem can be reduced to the above -- if the remote TCP stack receives such a packet, it will be obliged to send an ACK, failing which the local TCP driver would eventually tear-down the connection, therefore informing the local ERLANG process about such an action, right? (as it would in a case of TCP keep-alive) Another valid question can be: Is it worth-while going to a trouble of trying to do this? My current feeling is -- Yes; largely because I had (did I say had? I meant still have) to do application-level link testing with almost every project that requires some form of connection pool management. Why is this important? Well, not much if you prefer to ignore the golden rule about customer always being right (and we all know they can be quite left from time to time); unfortunately, if one is to observe important (evolution related) rules, one would need to consider easier ways to do this -- that is, provide for a link-testing even if the remote peer application does not support a link testing primitive. Kind regards, V/ From ngreco@REDACTED Mon Jun 27 16:53:08 2011 From: ngreco@REDACTED (Nahuel Greco) Date: Mon, 27 Jun 2011 11:53:08 -0300 Subject: [erlang-questions] Sending a tcp packet containing null data In-Reply-To: <97A9D51D-9FBD-4825-8BC8-3F121BF489E9@pharos-avantgard.com> References: <97A9D51D-9FBD-4825-8BC8-3F121BF489E9@pharos-avantgard.com> Message-ID: The operating system TCP keepalive support is just what you want. It sends an empty-data TCP packet with the ACK flag on. You need to enable it with the flag {keepalive, true} when creating the socket. But also you need to specify the timing parameters (the default to detect disconnection is two hours in Linux, unusable for most purposes); inets.erl currently doesn't support them but you can pass raw setsockopt() parameters. So, on Linux, you need to do something like: get_tcp_keepalive_opts(KeepIdle, KeepInterval, KeepCount) -> [{keepalive, true}, {raw, 1, 4, <>}, % SOL_SOCKET, TCP_KEEPIDLE {raw, 1, 5, <>}, % SOL_SOCKET, TCP_KEEPINTVL {raw, 1, 6, <>} % SOL_SOCKET, TCP_KEEPCNT ]. test_socket() -> gen_tcp:connect("127.0.0.1", 22, [binary, {packet, 0}] ++ get_tcp_keepalive_opts(2,2,2)). See "man 7 tcp". Btw, would be nice to patch inets.erl to support this. Saludos, Nahuel Greco. On Mon, Jun 27, 2011 at 11:12 AM, Valentin Micic wrote: > Hi > > I'd like to implement a generic link-test facility for TCP connections -- something that would work similarly to TCP keep-alive; however, under complete control of the calling ERLANG process with respect to sending and receiving of control packets. > By the same token, I'd like this test to be transparent (as in: non-intrusive) to the peer application. In other words, I would *not* like such an empty (null data) packet to be propagated to the remote peer application, but expect the remote TCP stack to issue an equally empty "ACK" packet (or stream of data, if available), which would be propagated back to the caller. > > Thus, ?I have a number of problems on multiple levels, but first and foremost, I have no idea how to achieve this (other than hoping that TCP keep-alive would be supported on a particular machine) :-) > > So, how can I force a null data packet to leave the local machine. Did try gen_tcp:send(Socket, <<>>), however, this didn't work (well... why should it?) > > Maybe the whole problem can be reduced to the above -- if the remote TCP stack receives such a packet, it will be obliged to send an ACK, failing which the local TCP driver would eventually tear-down the connection, therefore informing the local ERLANG process about such an action, right? (as it would in a case of TCP keep-alive) > > Another valid question can be: Is it worth-while going to a trouble of trying to do this? > > My current feeling is -- Yes; largely because I had (did I say had? I meant still have) to do application-level link testing with almost every project that requires some form of connection pool management. > Why is this important? Well, not much if you prefer to ignore the golden rule about customer always being right (and we all know they can be quite left from time to time); unfortunately, if one is to observe important (evolution related) rules, one would need to consider easier ways to do this -- that is, provide for a link-testing even if the remote peer application does not support a link testing primitive. > > Kind regards, > > V/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From martynasp@REDACTED Mon Jun 27 17:01:17 2011 From: martynasp@REDACTED (Martynas Pumputis) Date: Mon, 27 Jun 2011 18:01:17 +0300 Subject: [erlang-questions] problems with PropEr includes from eunit Message-ID: Hello, I have problems running PropEr:check_spec/1 from Eunit. With this line in module file: -include("bp_app/include/log_utils.hrl"). PropEr stops working. After removing it, PropEr starts working properly. Error message I get from PropEr: motiejus:~/mll/gateway_app$ make test_gateway_specs_tests ERL_FLAGS="-sname eunit -config ../bp_app/config/default.config +P1000000 -env ERL_MAX_ETS_TABLES 20000" ../rebar verbose=2 skip_deps=true suite=gateway_specs_tests eunit ==> gateway_app (eunit) Compiled tests/gateway_specs_tests.erl Compiled tests/gateway_app_tests.erl ======================== EUnit ======================== gateway_specs_tests: yadda_test (module 'gateway_specs_tests')... =INFO REPORT==== 27-Jun-2011::17:03:39 === pid: <0.475.0> source: "tests/gateway_specs_tests.erl:29" message: "checkspec says" error: {typeserver, {cant_load_code,gateway_helpers, {cant_find_object_file, {cant_compile_source_file, [{"/home/motiejus/code/mll/gateway_app/.eunit/gateway_helpers.erl", [{16,epp, {include,file, "bp_app/include/log_utils.hrl"}}]}]}}}} [0.029 s] ok ======================================================= Test passed. I suspect PropEr recompiles those modules and does not add include paths properly. Relevant Code: ===== gateway_helpers.erl ===== -module(gateway_helpers). -include("bp_app/include/log_utils.hrl"). -export([demo/1]). -spec demo(integer()) -> integer(). demo(Item) -> case Item of 1 -> undefined; R when is_integer(R) -> Item end. ===== gateway_specs_tests.erl ===== -module(gateway_specs_tests). -include_lib("proper/include/proper.hrl"). -include_lib("eunit/include/eunit.hrl"). -include("bp_app/include/log_utils.hrl"). yadda_test() -> ?info("checkspec says", [proper:check_spec({gateway_helpers, demo, 1})]). bp_app ??? include ? ??? log_utils.hrl gateway_app ??? include ? ??? consts.hrl ? ??? types.hrl ??? src ? ??? gateway_helpers.erl ??? tests ??? gateway_specs_tests.erl Versions: Erlang R14A (from Squeeze apt) PropEr v1.0-1-g5fe56e9 Please let me know if you need any more information. Motiejus From v@REDACTED Mon Jun 27 17:19:53 2011 From: v@REDACTED (Valentin Micic) Date: Mon, 27 Jun 2011 17:19:53 +0200 Subject: [erlang-questions] Sending a tcp packet containing null data In-Reply-To: References: <97A9D51D-9FBD-4825-8BC8-3F121BF489E9@pharos-avantgard.com> Message-ID: <6E6A8781-F691-4458-BAD3-977655392975@pharos-avantgard.com> Nice! This was my initial point of departure, however, got discouraged by loosely stipulated support for TCP keep-alive (even in TCP RFC 1122). After reading your comment, I think I am enriched with a new fond enthusiasm :-). Thank you! Have you ever used this approach in practice? Regards, V/ On 27 Jun 2011, at 4:53 PM, Nahuel Greco wrote: > The operating system TCP keepalive support is just what you want. It > sends an empty-data TCP packet with the ACK flag on. You need to > enable it with the flag {keepalive, true} when creating the socket. > But also you need to specify the timing parameters (the default to > detect disconnection is two hours in Linux, unusable for most > purposes); inets.erl currently doesn't support them but you can pass > raw setsockopt() parameters. So, on Linux, you need to do something > like: > > get_tcp_keepalive_opts(KeepIdle, KeepInterval, KeepCount) -> > [{keepalive, true}, > {raw, 1, 4, <>}, % SOL_SOCKET, TCP_KEEPIDLE > {raw, 1, 5, <>}, % SOL_SOCKET, TCP_KEEPINTVL > {raw, 1, 6, <>} % SOL_SOCKET, TCP_KEEPCNT > ]. > > > test_socket() -> > gen_tcp:connect("127.0.0.1", 22, [binary, {packet, 0}] ++ > get_tcp_keepalive_opts(2,2,2)). > > See "man 7 tcp". Btw, would be nice to patch inets.erl to support this. > > Saludos, > Nahuel Greco. > > > > On Mon, Jun 27, 2011 at 11:12 AM, Valentin Micic wrote: >> Hi >> >> I'd like to implement a generic link-test facility for TCP connections -- something that would work similarly to TCP keep-alive; however, under complete control of the calling ERLANG process with respect to sending and receiving of control packets. >> By the same token, I'd like this test to be transparent (as in: non-intrusive) to the peer application. In other words, I would *not* like such an empty (null data) packet to be propagated to the remote peer application, but expect the remote TCP stack to issue an equally empty "ACK" packet (or stream of data, if available), which would be propagated back to the caller. >> >> Thus, I have a number of problems on multiple levels, but first and foremost, I have no idea how to achieve this (other than hoping that TCP keep-alive would be supported on a particular machine) :-) >> >> So, how can I force a null data packet to leave the local machine. Did try gen_tcp:send(Socket, <<>>), however, this didn't work (well... why should it?) >> >> Maybe the whole problem can be reduced to the above -- if the remote TCP stack receives such a packet, it will be obliged to send an ACK, failing which the local TCP driver would eventually tear-down the connection, therefore informing the local ERLANG process about such an action, right? (as it would in a case of TCP keep-alive) >> >> Another valid question can be: Is it worth-while going to a trouble of trying to do this? >> >> My current feeling is -- Yes; largely because I had (did I say had? I meant still have) to do application-level link testing with almost every project that requires some form of connection pool management. >> Why is this important? Well, not much if you prefer to ignore the golden rule about customer always being right (and we all know they can be quite left from time to time); unfortunately, if one is to observe important (evolution related) rules, one would need to consider easier ways to do this -- that is, provide for a link-testing even if the remote peer application does not support a link testing primitive. >> >> Kind regards, >> >> V/ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ngreco@REDACTED Mon Jun 27 17:41:51 2011 From: ngreco@REDACTED (Nahuel Greco) Date: Mon, 27 Jun 2011 12:41:51 -0300 Subject: [erlang-questions] Sending a tcp packet containing null data In-Reply-To: <6E6A8781-F691-4458-BAD3-977655392975@pharos-avantgard.com> References: <97A9D51D-9FBD-4825-8BC8-3F121BF489E9@pharos-avantgard.com> <6E6A8781-F691-4458-BAD3-977655392975@pharos-avantgard.com> Message-ID: RFC 1122 is from 1989, the protocol implementors elves did some work since then :) I used it many times and never found an stack that does not support TCP keepalives. Saludos, Nahuel Greco. On Mon, Jun 27, 2011 at 12:19 PM, Valentin Micic wrote: > Nice! > > This was my initial point of departure, however, got discouraged by loosely stipulated support for TCP keep-alive (even in TCP RFC 1122). > After reading your comment, I think I am enriched with a new fond enthusiasm ?:-). > Thank you! > > Have you ever used this approach in practice? > > Regards, > > V/ > > On 27 Jun 2011, at 4:53 PM, Nahuel Greco wrote: > >> The operating system TCP keepalive support is just what you want. It >> sends an empty-data TCP packet with the ACK flag on. You need to >> enable it with the flag {keepalive, true} when creating the socket. >> But also you need to specify the timing parameters (the default to >> detect disconnection is two hours in Linux, unusable for most >> purposes); inets.erl currently doesn't support them but you can pass >> raw setsockopt() parameters. So, on Linux, you need to do something >> like: >> >> get_tcp_keepalive_opts(KeepIdle, KeepInterval, KeepCount) -> >> ? ?[{keepalive, true}, >> ? ? {raw, 1, 4, <>}, ? ? ? % SOL_SOCKET, TCP_KEEPIDLE >> ? ? {raw, 1, 5, <>}, ? % SOL_SOCKET, TCP_KEEPINTVL >> ? ? {raw, 1, 6, <>} ? ? ? % SOL_SOCKET, TCP_KEEPCNT >> ? ? ]. >> >> >> test_socket() -> >> ? ?gen_tcp:connect("127.0.0.1", 22, [binary, {packet, 0}] ++ >> get_tcp_keepalive_opts(2,2,2)). >> >> See "man 7 tcp". Btw, would be nice to patch inets.erl to support this. >> >> Saludos, >> Nahuel Greco. >> >> >> >> On Mon, Jun 27, 2011 at 11:12 AM, Valentin Micic wrote: >>> Hi >>> >>> I'd like to implement a generic link-test facility for TCP connections -- something that would work similarly to TCP keep-alive; however, under complete control of the calling ERLANG process with respect to sending and receiving of control packets. >>> By the same token, I'd like this test to be transparent (as in: non-intrusive) to the peer application. In other words, I would *not* like such an empty (null data) packet to be propagated to the remote peer application, but expect the remote TCP stack to issue an equally empty "ACK" packet (or stream of data, if available), which would be propagated back to the caller. >>> >>> Thus, ?I have a number of problems on multiple levels, but first and foremost, I have no idea how to achieve this (other than hoping that TCP keep-alive would be supported on a particular machine) :-) >>> >>> So, how can I force a null data packet to leave the local machine. Did try gen_tcp:send(Socket, <<>>), however, this didn't work (well... why should it?) >>> >>> Maybe the whole problem can be reduced to the above -- if the remote TCP stack receives such a packet, it will be obliged to send an ACK, failing which the local TCP driver would eventually tear-down the connection, therefore informing the local ERLANG process about such an action, right? (as it would in a case of TCP keep-alive) >>> >>> Another valid question can be: Is it worth-while going to a trouble of trying to do this? >>> >>> My current feeling is -- Yes; largely because I had (did I say had? I meant still have) to do application-level link testing with almost every project that requires some form of connection pool management. >>> Why is this important? Well, not much if you prefer to ignore the golden rule about customer always being right (and we all know they can be quite left from time to time); unfortunately, if one is to observe important (evolution related) rules, one would need to consider easier ways to do this -- that is, provide for a link-testing even if the remote peer application does not support a link testing primitive. >>> >>> Kind regards, >>> >>> V/ >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From dan@REDACTED Mon Jun 27 17:52:44 2011 From: dan@REDACTED (Daniel Dormont) Date: Mon, 27 Jun 2011 11:52:44 -0400 Subject: [erlang-questions] What is allowed in a pattern in terms of string concatenation? Message-ID: <64C19587-7ACB-4BCB-B901-EBBC5664547F@greywallsoftware.com> Suppose I want to test that a certain string String1 ends with a certain other string String2. The best I could come up with is: case string:substr(lists:reverse(String1), 1, length(String2)) of String2 -> ok; _ -> not_ok end I was thinking there must be a better way, but various attempts along the lines of case String1 of _ ++ String2 -> ok; all returned "illegal pattern" errors. I couldn't find a clear explanation online of when exactly ++ is allowed in a pattern and when it's not, so I'm a bit stumped. Is there a better way? thanks, Dan PS I'm aware of regexp and re, but my current environment is a hybrid of R12 and R13B so I'm a bit hesitant to involve either one right now. From montuori@REDACTED Mon Jun 27 18:00:59 2011 From: montuori@REDACTED (kevin montuori) Date: Mon, 27 Jun 2011 12:00:59 -0400 Subject: [erlang-questions] What is allowed in a pattern in terms of string concatenation? In-Reply-To: <64C19587-7ACB-4BCB-B901-EBBC5664547F@greywallsoftware.com> References: <64C19587-7ACB-4BCB-B901-EBBC5664547F@greywallsoftware.com> Message-ID: On Mon, Jun 27, 2011 at 11:52 AM, Daniel Dormont wrote: > Suppose I want to test that a certain string String1 ends with a certain other string String2. Although it doesn't actually answer the question you asked, you might try something like lists:suffix(String2, String1). Cheers. k. -- kevin montuori From reynaldomic@REDACTED Mon Jun 27 18:12:39 2011 From: reynaldomic@REDACTED (Reynaldo Baquerizo) Date: Mon, 27 Jun 2011 11:12:39 -0500 Subject: [erlang-questions] What is allowed in a pattern in terms of string concatenation? In-Reply-To: <64C19587-7ACB-4BCB-B901-EBBC5664547F@greywallsoftware.com> References: <64C19587-7ACB-4BCB-B901-EBBC5664547F@greywallsoftware.com> Message-ID: <20110627111239.540efc0f@bertello.localdomain> On Mon, 27 Jun 2011 11:52:44 -0400 Daniel Dormont wrote: > Suppose I want to test that a certain string String1 ends with a > certain other string String2. The best I could come up with is: > > case string:substr(lists:reverse(String1), 1, length(String2)) of > String2 -> ok; > _ -> not_ok > end > > I was thinking there must be a better way, but various attempts along > the lines of case String1 of > _ ++ String2 -> ok; > > all returned "illegal pattern" errors. I couldn't find a clear > explanation online of when exactly ++ is allowed in a pattern and > when it's not, so I'm a bit stumped. Is there a better way? I can think of one using pattern matching, is_substring(Substr, Substr) -> true; is_substring(String, Substr) -> [_|T] = String, p5(T, Substr). you'd have to wrap it in a try ... catch for badmatch -- Reynaldo From watson.timothy@REDACTED Mon Jun 27 18:21:07 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 27 Jun 2011 17:21:07 +0100 Subject: [erlang-questions] question about Variable:fun() In-Reply-To: <19976.13798.893259.929824@pilspetsen.it.uu.se> References: <19976.13798.893259.929824@pilspetsen.it.uu.se> Message-ID: > > The V call looks like a fake parameterized module. ?They should > be documented, perhaps as an experimental extension, however the > specific usage with an explicit tuple doesn't look quite kosher. > _______________________________________________ Parameterised modules do exist, but are an undocumented feature that could be removed at any time - indeed many people on this list have called for their removal, thought others would like them to stay. I tend to use them only when interacting with an API that requires them, opting instead to use a process (such as a gen_server) to encapsulate state. As previous answers mentioned, the general form M:func() will work whenever M evaluates to a module - an atom in most cases but for parameterized modules, a tuple where the first element is the name of the parameterised module and the remaining places contain its state. The example you've given is not a parameterised module per se, as it's module annotation doesn't contain a second element defining state variables in a list. e.g., -module(abc, [Var1, Var2, Etc]). But I suppose (from your example) that the use of parameterised modules at the call site is dealt with independently from their definitions, at least as far as the compiler is concerned. From reynaldomic@REDACTED Mon Jun 27 18:16:57 2011 From: reynaldomic@REDACTED (Reynaldo Baquerizo) Date: Mon, 27 Jun 2011 11:16:57 -0500 Subject: [erlang-questions] What is allowed in a pattern in terms of string concatenation? In-Reply-To: <20110627111239.540efc0f@bertello.localdomain> References: <64C19587-7ACB-4BCB-B901-EBBC5664547F@greywallsoftware.com> <20110627111239.540efc0f@bertello.localdomain> Message-ID: <20110627111657.2d5757d8@bertello.localdomain> On Mon, 27 Jun 2011 11:12:39 -0500 Reynaldo Baquerizo wrote: > On Mon, 27 Jun 2011 11:52:44 -0400 > Daniel Dormont wrote: > > > Suppose I want to test that a certain string String1 ends with a > > certain other string String2. The best I could come up with is: > > > > case string:substr(lists:reverse(String1), 1, length(String2)) of > > String2 -> ok; > > _ -> not_ok > > end > > > > I was thinking there must be a better way, but various attempts > > along the lines of case String1 of > > _ ++ String2 -> ok; > > > > all returned "illegal pattern" errors. I couldn't find a clear > > explanation online of when exactly ++ is allowed in a pattern and > > when it's not, so I'm a bit stumped. Is there a better way? > > I can think of one using pattern matching, > > is_substring(Substr, Substr) -> > true; > is_substring(String, Substr) -> > [_|T] = String, > p5(T, Substr). Oops, that should be: is_substring(String, Substr) -> [_|T] = String, is_substring(T, Substr). -- Reynaldo From freza@REDACTED Mon Jun 27 19:23:36 2011 From: freza@REDACTED (Jachym Holecek) Date: Mon, 27 Jun 2011 18:23:36 +0100 Subject: [erlang-questions] What is allowed in a pattern in terms of string concatenation? In-Reply-To: <64C19587-7ACB-4BCB-B901-EBBC5664547F@greywallsoftware.com> References: <64C19587-7ACB-4BCB-B901-EBBC5664547F@greywallsoftware.com> Message-ID: <20110627172336.GA1848@hanele> # Daniel Dormont 2011-06-27: > Suppose I want to test that a certain string String1 ends with a certain > other string String2. The best I could come up with is: There are no strings in Erlang! :-) > case string:substr(lists:reverse(String1), 1, length(String2)) of > String2 -> ok; > _ -> not_ok > end Kevin already pointed you to lists:suffix/2 which is good if suffix isn't statically known, and very easy on the eyes too. If suffix is known you could do: has_suffix(L) -> has_suffix2(lists:reverse(L)). has_suffix2("xiffus" ++ _) -> true; has_suffix2(_) -> false. Which I think answers your other question. More precisely, AFAIU this has_suffix2("xiffus" ++ _) is just syntactic sugar for has_suffix2([$x, $i, $f, $f, $u, $s | _]) So in a way you could perhaps say "++" isn't really supported in pattern matching (but other will know this with more certainty). One doesn't see "++" used very often overall, in my experience. Also, this would have been so much cheaper (but harder to read) if you used binaries to represent strings: has_suffix(S, B) -> has_suffix2(byte_size(B) - byte_size(S), S, B). has_suffix2(N, _, _) when N < 0 -> false; has_suffix2(N, S, <<_:N/binary, D/binary>>) -> S == D. Or somesuch, you get the idea. HTH, -- Jachym From dan@REDACTED Mon Jun 27 20:27:59 2011 From: dan@REDACTED (Daniel Dormont) Date: Mon, 27 Jun 2011 14:27:59 -0400 Subject: [erlang-questions] What is allowed in a pattern in terms of string concatenation? In-Reply-To: <20110627172336.GA1848@hanele> References: <64C19587-7ACB-4BCB-B901-EBBC5664547F@greywallsoftware.com> <20110627172336.GA1848@hanele> Message-ID: <4D51A73A-3AA7-4016-86BA-89A3116A7782@greywallsoftware.com> On Jun 27, 2011, at 1:23 PM, Jachym Holecek wrote: > # Daniel Dormont 2011-06-27: >> Suppose I want to test that a certain string String1 ends with a certain >> other string String2. The best I could come up with is: > > There are no strings in Erlang! :-) > Right, fair enough. >> case string:substr(lists:reverse(String1), 1, length(String2)) of >> String2 -> ok; >> _ -> not_ok >> end > > Kevin already pointed you to lists:suffix/2 which is good if suffix > isn't statically known, and very easy on the eyes too. If suffix is > known you could do: > > has_suffix(L) -> > has_suffix2(lists:reverse(L)). > > has_suffix2("xiffus" ++ _) -> > true; > has_suffix2(_) -> > false. > I was not familiar with lists:suffix. I am now :) and it works fine for my purposes. > Which I think answers your other question. More precisely, AFAIU this > > has_suffix2("xiffus" ++ _) > > is just syntactic sugar for > > has_suffix2([$x, $i, $f, $f, $u, $s | _]) > > So in a way you could perhaps say "++" isn't really supported in pattern > matching (but other will know this with more certainty). One doesn't see > "++" used very often overall, in my experience. > Ok. > Also, this would have been so much cheaper (but harder to read) if you > used binaries to represent strings: > > has_suffix(S, B) -> > has_suffix2(byte_size(B) - byte_size(S), S, B). > > has_suffix2(N, _, _) when N < 0 -> > false; > has_suffix2(N, S, <<_:N/binary, D/binary>>) -> > S == D. > Right, unfortunately in the place where my module will be called, the string values are in lists, not binaries. > Or somesuch, you get the idea. > Dan From moxford@REDACTED Mon Jun 27 20:38:37 2011 From: moxford@REDACTED (Mike Oxford) Date: Mon, 27 Jun 2011 11:38:37 -0700 Subject: [erlang-questions] Trie fun Message-ID: Erlang's MR against a trie seems ideal for parallelizing a word filter. However, the filter-terms becomes the bottleneck. In many languages we'd put it into memory and acquire a pointer/reference to it, spawn the worker threads and pass them the address of the filter list. This is efficient as it's static-data and only in memory in one place. In Erlang we build up the trie and then send it messages, leading to a bottleneck at the filter-list's message queue. Okay, so we can spawn multiple filters, and ship words to them, but every filter has to build up the filter-list from storage (compiled in, in this case, to minimize overhead.) Now we have 'm' words going to 'n' filters. Except for the startup costs, that doesn't seem too bad. The Trie is static, prebuild it and store it as a binary "variable", so it's just blasted into memory instead of actually being built? Any downsides? (Theory; I don't know if you can "image" memory this way in Erlang yet.) Is there a better way? Yes, I could do it in C and link it in but I'm interested in the Erlang way. :) Thanks! -mox From gleber.p@REDACTED Mon Jun 27 20:47:29 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 27 Jun 2011 20:47:29 +0200 Subject: [erlang-questions] Trie fun In-Reply-To: References: Message-ID: Maybe creating a module in runtime, where a function returns a prebuild Trie as a static term, will be a good solution? This way processess will be able to share the same Trie in the same area of memory. AFAIK static terms from modules are not copied from module's "static term storage" to each process which uses it. On Mon, Jun 27, 2011 at 20:38, Mike Oxford wrote: > Erlang's MR against a trie seems ideal for parallelizing a word filter. > However, the filter-terms becomes the bottleneck. > > In many languages we'd put it into memory and acquire a > pointer/reference to it, spawn the worker threads and pass them the > address of the filter list. > This is efficient as it's static-data and only in memory in one place. > > In Erlang we build up the trie and then send it messages, leading to a > bottleneck at the filter-list's message queue. > Okay, so we can spawn multiple filters, and ship words to them, but > every filter has to build up the filter-list from storage (compiled > in, in this case, to minimize overhead.) > Now we have 'm' words going to 'n' filters. ?Except for the startup > costs, that doesn't seem too bad. > > The Trie is static, prebuild it and store it as a binary "variable", > so it's just blasted into memory instead of actually being built? ?Any > downsides? (Theory; I don't know if you > can "image" memory this way in Erlang yet.) > > Is there a better way? ?Yes, I could do it in C and link it in but I'm > interested in the Erlang way. ?:) > > Thanks! > > -mox > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From neumark.peter@REDACTED Mon Jun 27 20:50:45 2011 From: neumark.peter@REDACTED (Peter Neumark) Date: Mon, 27 Jun 2011 20:50:45 +0200 Subject: [erlang-questions] erlang thrit bindings and TJSONProtocol In-Reply-To: References: <2B9E17AF-1B25-4902-B0B8-779C43CB688A@gmail.com> <20110617210922.GA50776@alumni.caltech.edu> Message-ID: I finished a thrift json protocol implementation for erlang: My implementation of the protocol may be found here: https://github.com/postlock/thrift/blob/trunk/lib/erl/src/thrift_json_protocol.erl A modified version of the erlang thrift tutorial which uses my implementation over HTTP to talk to the same server that the javascript tutorial uses: https://github.com/postlock/thrift/blob/trunk/tutorial/erl/json_client.erl Finally, it should be noted that the code depends on JSX: https://github.com/talentdeficit/jsx I haven't had time to run a huge amount of tests on it yet, but the tutorial seems to work fine. Improvements are welcome! Peter Neumark On Sun, Jun 19, 2011 at 8:48 PM, Peter Neumark wrote: > I'm in the process of writing one now, but I'm not sure when it will > be finished. > If all goes well maybe sometime this week. > > Peter > > > On Fri, Jun 17, 2011 at 11:09 PM, Anthony Molinaro > wrote: > > I don't believe so, currently only base64 and binary protocols. However, > > if you feel like taking on creation of this protocol, I'm willing to look > > it over and possibly merge it (I'm a maintainer/committer). > > > > -Anthony > > > > On Fri, Jun 17, 2011 at 09:39:59PM +0100, Joel Reymont wrote: > >> Do Erlang bindings for Thrift support TJSONProtocol? > >> > >> > http://sundresh.org/docs/thrift-0.2.0-javadoc/org/apache/thrift/protocol/TJSONProtocol.html > >> > >> > -------------------------------------------------------------------------- > >> - for hire: mac osx device driver ninja, kernel extensions and usb > drivers > >> > ---------------------+------------+--------------------------------------- > >> http://wagerlabs.com | @wagerlabs | > http://www.linkedin.com/in/joelreymont > >> > ---------------------+------------+--------------------------------------- > >> > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > > ------------------------------------------------------------------------ > > Anthony Molinaro > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Mon Jun 27 22:38:49 2011 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 27 Jun 2011 13:38:49 -0700 Subject: [erlang-questions] Trie fun In-Reply-To: References: Message-ID: <4E08EA59.3070905@gmail.com> If you use https://github.com/okeuday/trie with https://github.com/esl/parse_trans/blob/master/src/ct_expand.erl you could have a static prebuild trie at compile-time. That should be the best solution for a static trie in Erlang. On 06/27/2011 11:38 AM, Mike Oxford wrote: > Erlang's MR against a trie seems ideal for parallelizing a word filter. > However, the filter-terms becomes the bottleneck. > > In many languages we'd put it into memory and acquire a > pointer/reference to it, spawn the worker threads and pass them the > address of the filter list. > This is efficient as it's static-data and only in memory in one place. > > In Erlang we build up the trie and then send it messages, leading to a > bottleneck at the filter-list's message queue. > Okay, so we can spawn multiple filters, and ship words to them, but > every filter has to build up the filter-list from storage (compiled > in, in this case, to minimize overhead.) > Now we have 'm' words going to 'n' filters. Except for the startup > costs, that doesn't seem too bad. > > The Trie is static, prebuild it and store it as a binary "variable", > so it's just blasted into memory instead of actually being built? Any > downsides? (Theory; I don't know if you > can "image" memory this way in Erlang yet.) > > Is there a better way? Yes, I could do it in C and link it in but I'm > interested in the Erlang way. :) > > Thanks! > > -mox > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From essen@REDACTED Tue Jun 28 00:02:01 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 28 Jun 2011 00:02:01 +0200 Subject: [erlang-questions] How to help improve specs in OTP? Message-ID: <4E08FDD9.8060804@dev-extend.eu> Hello, I'm looking into improving even more the specs I have in cowboy and am wondering what are the guidelines on how to help improve the specs in OTP so that applications can make better use of the types already defined or the library functions. Note that all my comments are about R14B03 and might not be up to date anymore. There's some types in the calendar module that define the datetime, year, months, days, and so on. They are not exported. Would a patch adding -export_type to the useful types be accepted? I'm assuming here that useful means a type used for the arguments and/or return values of the module's exported functions. Same comment about inet:ip_port(), it would be better if we could just use it without redefining it. Especially considering other types in that module are exported, like inet:socket(). What about opaque types? Should they be exported too? There's also modules that have no typespecs at all yet. How should we go about adding them? Any guidelines to follow? Style varies depending on the module, too, like 'atom' versus atom. Finally, what's the best way to test that kind of changes? I'm guessing I don't need to recompile everything all the time... Please send any guidelines or advices and I'll start working on a few patches. -- Lo?c Hoguin Dev:Extend From ok@REDACTED Tue Jun 28 03:34:47 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 28 Jun 2011 13:34:47 +1200 Subject: [erlang-questions] What is allowed in a pattern in terms of string concatenation? In-Reply-To: <64C19587-7ACB-4BCB-B901-EBBC5664547F@greywallsoftware.com> References: <64C19587-7ACB-4BCB-B901-EBBC5664547F@greywallsoftware.com> Message-ID: <216B4AB7-9832-4B40-B057-164B59DF14F0@cs.otago.ac.nz> On 28/06/2011, at 3:52 AM, Daniel Dormont wrote: > Suppose I want to test that a certain string String1 ends with a certain other string String2. The best I could come up with is: > > case string:substr(lists:reverse(String1), 1, length(String2)) of > String2 -> ok; > _ -> not_ok > end > > I was thinking there must be a better way, but various attempts along the lines of > case String1 of > _ ++ String2 -> ok; > > all returned "illegal pattern" errors. I couldn't find a clear explanation online of when exactly ++ is allowed in a pattern and when it's not, so I'm a bit stumped. Section 7.4 of the reference manual is the place where it says ++ is allowed as a pattern, and the only thing it allows on the left of ++ is an explicit string literal. A pattern like "foo"++X is just shorthand for [$f,$o,$o|X] If you want to check whether String1 ends with String2, (a) you're going to need an even number of calls to lists:reverse/1 (b) you are going to be kicking yourself that you didn't read the documentation for the 'lists' module http://www.erlang.org/doc/man/lists.html because lists:suffix(String2, String1) does exactly what you want (modulo returning 'true' or 'false'). From wmacgyver@REDACTED Tue Jun 28 05:11:59 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Mon, 27 Jun 2011 23:11:59 -0400 Subject: [erlang-questions] recommended redis client? Message-ID: there are a few redis clients for erlang. is there one that stands out from the rest? What do you use/recommend? Thanks -- Omnem crede diem tibi diluxisse supremum. From dan@REDACTED Tue Jun 28 05:58:03 2011 From: dan@REDACTED (Daniel Dormont) Date: Mon, 27 Jun 2011 23:58:03 -0400 Subject: [erlang-questions] What is allowed in a pattern in terms of string concatenation? In-Reply-To: <216B4AB7-9832-4B40-B057-164B59DF14F0@cs.otago.ac.nz> References: <64C19587-7ACB-4BCB-B901-EBBC5664547F@greywallsoftware.com> <216B4AB7-9832-4B40-B057-164B59DF14F0@cs.otago.ac.nz> Message-ID: On Mon, Jun 27, 2011 at 9:34 PM, Richard O'Keefe wrote: > > On 28/06/2011, at 3:52 AM, Daniel Dormont wrote: > > > Suppose I want to test that a certain string String1 ends with a certain > other string String2. The best I could come up with is: > > > > case string:substr(lists:reverse(String1), 1, length(String2)) of > > String2 -> ok; > > _ -> not_ok > > end > > > > I was thinking there must be a better way, but various attempts along the > lines of > > case String1 of > > _ ++ String2 -> ok; > > > > all returned "illegal pattern" errors. I couldn't find a clear > explanation online of when exactly ++ is allowed in a pattern and when it's > not, so I'm a bit stumped. > > Section 7.4 of the reference manual is the place where it says ++ is > allowed > as a pattern, and the only thing it allows on the left of ++ is an explicit > string literal. A pattern like > "foo"++X > is just shorthand for > [$f,$o,$o|X] > > I see. I'd found that section in my searching earlier, but hadn't made the connection that it has to be literal (in my defense, the manual doesn't exactly say that). > If you want to check whether String1 ends with String2, > (a) you're going to need an even number of calls to lists:reverse/1 > (b) you are going to be kicking yourself that you didn't read the > documentation for the 'lists' module > http://www.erlang.org/doc/man/lists.html > because > lists:suffix(String2, String1) > does exactly what you want (modulo returning 'true' or 'false'). > Sometimes the challenge is not knowing where to look. I've used lists of course, but Erlang's lists is a pretty big module; it's easy to miss a few things here and there :) Anyway, thanks. dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal.ptaszek@REDACTED Tue Jun 28 09:45:36 2011 From: michal.ptaszek@REDACTED (Michal Ptaszek) Date: Tue, 28 Jun 2011 09:45:36 +0200 Subject: [erlang-questions] recommended redis client? In-Reply-To: References: Message-ID: <5F7D6595-B630-45CF-A660-41BC76BDEED4@erlang-solutions.com> I would definitely look at: https://github.com/JacobVorreuter/redo as it seems to be the most mature one. Cheers, Michal Ptaszek On Jun 28, 2011, at 5:11 AM, Wilson MacGyver wrote: > there are a few redis clients for erlang. > > is there one that stands out from the rest? What do you use/recommend? > > Thanks > > -- > Omnem crede diem tibi diluxisse supremum. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From knut.nesheim@REDACTED Tue Jun 28 10:12:10 2011 From: knut.nesheim@REDACTED (Knut Nesheim) Date: Tue, 28 Jun 2011 10:12:10 +0200 Subject: [erlang-questions] recommended redis client? In-Reply-To: References: Message-ID: Hi Wilson, As you say, there are a few options: * erldis, older, implements an API on top of redis(sets, lists, dicts), not so good error handling(tries to reconnect on every request, clients will pile up), not very good performance for multibulk replies, maybe a bit bloated, https://github.com/cstar/erldis * redo, very new, has support for pipelining requests, ie. performing multiple requests in one go, no abstraction on top of redis, not so good error handling (similar to erldis), probably good performance for multibulk replies, used by heroku, https://github.com/JacobVorreuter/redo * erl-redis, mature, implements pooling, sharding across databases, never used myself, https://github.com/litaocheng/erl-redis * eredis, very new, focus on performance and error handling, no abstraction on top of redis, no pubsub, used by wooga (online gaming), https://github.com/wooga/eredis (Disclaimer: I am the author of eredis) For our use case, we want good performance and good error handling. This is why we wrote eredis. The eredis repo contains eredis and erldis drivers for basho bench, so you can run benchmarks on your own hardware with your own queries and data. The error handling in eredis allows us to loose the connection to redis, without having requests timing out just waiting for it to come back up. It will try to reconnect in the background, while still serving clients. We handle sharding inside our application. Regards Knut On Tue, Jun 28, 2011 at 5:11 AM, Wilson MacGyver wrote: > there are a few redis clients for erlang. > > is there one that stands out from the rest? What do you use/recommend? > > Thanks > > -- > Omnem crede diem tibi diluxisse supremum. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Engineering http://www.wooga.com | phone +49 151 57202523 | fax +49-30-8964 9064 wooga GmbH | Saarbruecker Str. 38 | 10405 Berlin | Germany Sitz der Gesellschaft: Berlin; HRB 117846 B Registergericht Berlin-Charlottenburg Geschaeftsfuehrung: Jens Begemann, Philipp Moeser From pablo.platt@REDACTED Tue Jun 28 11:58:46 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 28 Jun 2011 02:58:46 -0700 (PDT) Subject: [erlang-questions] bug in binary:split with trim? Message-ID: <1309255126.30623.YahooMailNeo@web112610.mail.gq1.yahoo.com> binary:split with the trim option doesn't remove empty results from the beginning of the binary: binary:split(<<"/a/b/">>, <<"/">>, [global, trim]). [<<>>,<<"a">>,<<"b">>] binary:split(<<"//a/b///">>, <<"/">>, [trim, global]). [<<>>,<<>>,<<"a">>,<<"b">>] Is this a bug? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Tue Jun 28 12:16:57 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Tue, 28 Jun 2011 11:16:57 +0100 Subject: [erlang-questions] A simple NIF so: compiles on Ubuntu, but not on MacOSX. Is something missing? Message-ID: <4E09AA19.20901@llaisdy.com> Dear All I'm writing a simple NIF. This is what I'm using to compile it: gcc -Wall \ -I/usr/local/lib/erlang/usr/include \ -fPIC \ -shared \ -o simple_nif.so \ simple.c simple_nif.c This compiles on Ubuntu, but trying it on a Mac I get this error: > Undefined symbols: > "_enif_make_badarg", referenced from: > _get_params_nif in ccNKt95s.o > "_enif_make_tuple_from_array", referenced from: > _get_params_nif in ccNKt95s.o > "_enif_make_int", referenced from: > _get_params_nif in ccNKt95s.o > "_enif_get_tuple", referenced from: > _get_params_nif in ccNKt95s.o > ld: symbol(s) not found > collect2: ld returned 1 exit status Both machines are running erlang 14B, installed from source (the -I path is the same for both). I don't have my Ubuntu machine with me atm, but this is what's in .../usr/include on the Mac: > $ l /usr/local/lib/erlang/usr/include/ > > driver_int.h > erl_driver.h > erl_drv_nif.h > erl_fixed_size_int_types.h > erl_int_sizes_config.h > erl_memory_trace_parser.h > erl_nif.h > erl_nif_api_funcs.h Is something missing? Does the Mac need extra arguments to compile? With thanks and best wishes Ivan -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From lukas.larsson@REDACTED Tue Jun 28 12:26:10 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Tue, 28 Jun 2011 10:26:10 +0000 (GMT) Subject: [erlang-questions] A simple NIF so: compiles on Ubuntu, but not on MacOSX. Is something missing? In-Reply-To: <1688912529.21001309256635317.JavaMail.root@zimbra> Message-ID: <1600710057.21041309256769984.JavaMail.root@zimbra> Hi! On OS X you want to give "-bundle -flat_namespace -undefined suppress" and remove "-shared" for the linker in order for it to link successfully. Lukas ----- Original Message ----- From: "Ivan Uemlianin" To: "Erlang Questions" Sent: Tuesday, June 28, 2011 12:16:57 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: [erlang-questions] A simple NIF so: compiles on Ubuntu, but not on MacOSX. Is something missing? Dear All I'm writing a simple NIF. This is what I'm using to compile it: gcc -Wall \ -I/usr/local/lib/erlang/usr/include \ -fPIC \ -shared \ -o simple_nif.so \ simple.c simple_nif.c This compiles on Ubuntu, but trying it on a Mac I get this error: > Undefined symbols: > "_enif_make_badarg", referenced from: > _get_params_nif in ccNKt95s.o > "_enif_make_tuple_from_array", referenced from: > _get_params_nif in ccNKt95s.o > "_enif_make_int", referenced from: > _get_params_nif in ccNKt95s.o > "_enif_get_tuple", referenced from: > _get_params_nif in ccNKt95s.o > ld: symbol(s) not found > collect2: ld returned 1 exit status Both machines are running erlang 14B, installed from source (the -I path is the same for both). I don't have my Ubuntu machine with me atm, but this is what's in .../usr/include on the Mac: > $ l /usr/local/lib/erlang/usr/include/ > > driver_int.h > erl_driver.h > erl_drv_nif.h > erl_fixed_size_int_types.h > erl_int_sizes_config.h > erl_memory_trace_parser.h > erl_nif.h > erl_nif_api_funcs.h Is something missing? Does the Mac need extra arguments to compile? With thanks and best wishes Ivan -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From ivan@REDACTED Tue Jun 28 12:32:34 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Tue, 28 Jun 2011 11:32:34 +0100 Subject: [erlang-questions] SOLVED Re: A simple NIF so: compiles on Ubuntu, but not on MacOSX. Is something missing? In-Reply-To: <1600710057.21041309256769984.JavaMail.root@zimbra> References: <1600710057.21041309256769984.JavaMail.root@zimbra> Message-ID: <4E09ADC2.2000004@llaisdy.com> Dear Lukas Thanks very much! Now it compiles on the Mac. Best wishes Ivan On 28/06/2011 11:26, Lukas Larsson wrote: > Hi! > > On OS X you want to give "-bundle -flat_namespace -undefined suppress" and remove "-shared" for the linker in order for it to link successfully. > > Lukas > ----- Original Message ----- > From: "Ivan Uemlianin" > To: "Erlang Questions" > Sent: Tuesday, June 28, 2011 12:16:57 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna > Subject: [erlang-questions] A simple NIF so: compiles on Ubuntu, but not on MacOSX. Is something missing? > > Dear All > > I'm writing a simple NIF. This is what I'm using to compile it: > > gcc -Wall \ > -I/usr/local/lib/erlang/usr/include \ > -fPIC \ > -shared \ > -o simple_nif.so \ > simple.c simple_nif.c > > This compiles on Ubuntu, but trying it on a Mac I get this error: > >> Undefined symbols: >> "_enif_make_badarg", referenced from: >> _get_params_nif in ccNKt95s.o >> "_enif_make_tuple_from_array", referenced from: >> _get_params_nif in ccNKt95s.o >> "_enif_make_int", referenced from: >> _get_params_nif in ccNKt95s.o >> "_enif_get_tuple", referenced from: >> _get_params_nif in ccNKt95s.o >> ld: symbol(s) not found >> collect2: ld returned 1 exit status > > Both machines are running erlang 14B, installed from source (the -I path > is the same for both). I don't have my Ubuntu machine with me atm, but > this is what's in .../usr/include on the Mac: > >> $ l /usr/local/lib/erlang/usr/include/ >> >> driver_int.h >> erl_driver.h >> erl_drv_nif.h >> erl_fixed_size_int_types.h >> erl_int_sizes_config.h >> erl_memory_trace_parser.h >> erl_nif.h >> erl_nif_api_funcs.h > > Is something missing? Does the Mac need extra arguments to compile? > > With thanks and best wishes > > Ivan > > -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From spawn.think@REDACTED Tue Jun 28 12:43:42 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Tue, 28 Jun 2011 12:43:42 +0200 Subject: [erlang-questions] bug in binary:split with trim? In-Reply-To: <1309255126.30623.YahooMailNeo@web112610.mail.gq1.yahoo.com> References: <1309255126.30623.YahooMailNeo@web112610.mail.gq1.yahoo.com> Message-ID: According to documentation it's not *trim**Removes trailing empty parts of the result (as does trim in re:split/3)** * http://www.erlang.org/doc/man/binary.html#split-2 On Tue, Jun 28, 2011 at 11:58 AM, Pablo Platt wrote: > binary:split with the trim option doesn't remove empty results from the > beginning of the binary: > > binary:split(<<"/a/b/">>, <<"/">>, [global, trim]). > [<<>>,<<"a">>,<<"b">>] > > binary:split(<<"//a/b///">>, <<"/">>, [trim, global]). > [<<>>,<<>>,<<"a">>,<<"b">>] > > Is this a bug? > > Thanks > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think -------------- next part -------------- An HTML attachment was scrubbed... URL: From jalanzo@REDACTED Tue Jun 28 13:12:45 2011 From: jalanzo@REDACTED (Juan Antonio Berrayarza =?ISO-8859-1?Q?Lanz=F3?=) Date: Tue, 28 Jun 2011 13:12:45 +0200 Subject: [erlang-questions] odbc blob datatype In-Reply-To: <1705425771.21151309257682092.JavaMail.root@zimbra> References: <1705425771.21151309257682092.JavaMail.root@zimbra> Message-ID: <1309259565.4236.4.camel@Ubuntu-Juan> Buenos dias Juan : Gracias por la info, pero con Varchar funciona correctamente. Estoy mirando una respuesta que me envio Ingela Andin en la que me indica que efectivamente SQL_WLONGVARCHAR no est? soportado actualmente, pero que est?n trabajando en un parche para soportarlo. De nuevo, gracias por la info. Un saludo ---------------- Hi Juan : Thanks for the info, but SQL_VARCHAR works fine. I'm looking out a response sent by Ingela Andin, where she tell's me that SQL_WLONGVARCHAR is actually UNSUPPORTED but they are working on a path to supported. Tx again for the info Greetings. El mar, 28-06-2011 a las 12:41 +0200, Juan Puig escribi?: > Hola Juan Antonio, > > Te puedo decir que VARCHAR se mapea correctamente a SQL_VARCHAR (postgres), > > 8> odbc:describe_table(C2, "t_proxy_info"). > {ok,[{"proxy_info_id",{sql_numeric,16,0}}, > {"proxy",{sql_varchar,75}}, > {"proxy_supp",{sql_varchar,50}}, > {"proxy_type_id",{sql_numeric,16,0}}, > {"party_id",{sql_numeric,16,0}}, > {"valid_from",sql_timestamp}, > {"valid_to",sql_timestamp}]} > > Has probado VARCHAR por un momento para descartar? > > Juan Puig > Erlang Solutions Ltd - London > www.erlang-solutions.com > > ----- "Juan Antonio Berrayarza Lanz?" wrote: > > > Hi : > > > > I'm trying to access a table in Mysql that has TEXT fields. > > when i do a odbc:describe_table() i get > > {"field_name",ODBC_UNSUPPORTED_TYPE}. > > > > Reading documentation about erlang odbc, i see that TEXT fields > > should > > be mapped to SQL_LONGVARCHAR. > > > > Has anyone worked this out? how can i query tables with TEXT fields > > using odbc and erlang ? > > > > > > erlang : R14B02 > > odbc: 2.10.10 > > mysql: 4.1.49 > > > > odbc mysql driver : 5.1.8 > > > > Tx to all of you > > > > > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions From fred.hebert@REDACTED Tue Jun 28 13:56:55 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Tue, 28 Jun 2011 07:56:55 -0400 Subject: [erlang-questions] Issues with reltool vs. init:stop() Message-ID: <541B4472-6E43-4F45-8F33-4233277977B0@erlang-solutions.com> I'm having quite peculiar issues when it comes to shutting down the VM from within a release. I'm writing a little demo application to play with reltools. The application is using files from LYSE's last chapters on applications, but I believe my problem to be independent from there. I've modified my code a bit so that when it's done running, 'init:stop()' is called from within one of the gen_servers I have. Whenever I do "application:start(ppool), application:start(erlcount)." and wait, the erlcount application will run, call init:stop(), and everything will shut down fine. I then made a release (.rel file not included here) with systools: systools:make_script("erlcount-1.0", [local]). systools:make_tar("erlcount-1.0", [{erts,"/local/lib/erlang"}]) And ran it with $ ./erts-5.8.4/bin/erl -boot ./releases/1.0.0/start from the right directory, which worked well. Init:stop() managed to take the VM down. Then I decided to switch to reltool. This is my current config file (not yet having all modules stripped down): {sys, [ {lib_dirs, ["/home/ferd/code/learn-you-some-erlang/release/"]}, {rel, "erlcount", "1.0.0", [kernel, stdlib, {ppool, permanent}, {erlcount, permanent}]}, {boot_rel, "erlcount"}, {relocatable, true}, {profile, standalone}, {app, ppool, [{vsn, "1.0.0"}, {app_file, all}, {debug_info, keep}]}, {app, erlcount, [{vsn, "1.0.0"}, {incl_cond, include}, {app_file, strip}, {debug_info, strip}]} ]}. I then run the following commands, without an error: 1> {ok, Conf} = file:consult("erlcount-1.0.config"). 2> {ok, Spec} = reltool:get_target_spec(Conf). 3> reltool:eval_target_spec(Spec, code:root_dir(), "./rel"). Then, when I call "./rel/bin/erl" from the shell, the application starts normally, does everything it needs to do, but init:stop() will hang forever and the VM will only stop if I start it with -shutdown_timeout SomeDelay that will kill everything. The same problem remains if I make a more stripped-down reltools release by using incl_cond on the apps I want to only keep the core of what's necessary. I've googled around a bit and have not found anything that might explain this behaviour. My application's code doesn't seem to be the culprit given it works normally from the shell, and also works normally with systools, but fails to quit with reltools. I've also tried calling './rel/bin/erl -run init stop' -- it seems that it kills part of the VM's apps (no output coming, or only partial output from the app), but will still fail to take the whole VM down. Anyone has an idea or has had similar problems? Pointers or idea of what to explore to resolve this? I've tried it both with R14B02 and R14B03 and it keeps the same behaviour there. Reltool seems to break init:stop for my release. -- Fred H?bert http://www.erlang-solutions.com From pablo.platt@REDACTED Tue Jun 28 14:20:07 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 28 Jun 2011 05:20:07 -0700 (PDT) Subject: [erlang-questions] bug in binary:split with trim? In-Reply-To: References: <1309255126.30623.YahooMailNeo@web112610.mail.gq1.yahoo.com> Message-ID: <1309263607.91250.YahooMailNeo@web112601.mail.gq1.yahoo.com> Is there a reason to make it work only for trailing empty parts but not for all empty parts? ________________________________ From: Ahmed Omar To: Pablo Platt Cc: "erlang-questions@REDACTED" Sent: Tuesday, June 28, 2011 1:43 PM Subject: Re: [erlang-questions] bug in binary:split with trim? According to documentation it's not trim Removes trailingempty parts of the result (as does trim in?re:split/3) http://www.erlang.org/doc/man/binary.html#split-2 On Tue, Jun 28, 2011 at 11:58 AM, Pablo Platt wrote: binary:split with the trim option doesn't remove empty results from the beginning of the binary: > > > >binary:split(<<"/a/b/">>, <<"/">>, [global, trim]). >[<<>>,<<"a">>,<<"b">>] > > >binary:split(<<"//a/b///">>, <<"/">>, [trim, global]). >[<<>>,<<>>,<<"a">>,<<"b">>] > > >Is this a bug? > > >Thanks >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin@REDACTED Tue Jun 28 14:48:03 2011 From: kevin@REDACTED (Kevin Smith) Date: Tue, 28 Jun 2011 08:48:03 -0400 Subject: [erlang-questions] recommended redis client? In-Reply-To: References: Message-ID: I'll also throw my client, reddy, in the ring as one to consider: https://github.com/kevsmith/reddy I developed it to support connection pools, async and sync commands, and better integration w/Erlang types (Redis hashes as proplists, for example). All commands are pipelined. It doesn't quite have 100% API coverage yet but I'm working on it, as time permits. --Kevin On Jun 27, 2011, at 11:11 PM, Wilson MacGyver wrote: > there are a few redis clients for erlang. > > is there one that stands out from the rest? What do you use/recommend? > > Thanks > > -- > Omnem crede diem tibi diluxisse supremum. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From alain.odea@REDACTED Tue Jun 28 15:13:39 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Tue, 28 Jun 2011 06:13:39 -0700 (PDT) Subject: [erlang-questions] Erlang emacs workflow question In-Reply-To: Message-ID: <8024573.83.1309266819614.JavaMail.geo-discussion-forums@yqcj24> Thank you Klas! This is great documentation. This should be part of http://www.erlang.org/doc/man/erlang.el.html. -------------- next part -------------- An HTML attachment was scrubbed... URL: From banibrata.dutta@REDACTED Tue Jun 28 17:10:22 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Tue, 28 Jun 2011 20:40:22 +0530 Subject: [erlang-questions] Erlang emacs workflow question In-Reply-To: <8024573.83.1309266819614.JavaMail.geo-discussion-forums@yqcj24> References: <8024573.83.1309266819614.JavaMail.geo-discussion-forums@yqcj24> Message-ID: On Tue, Jun 28, 2011 at 6:43 PM, Alain O'Dea wrote: > Thank you Klas! This is great documentation. This should be part of > http://www.erlang.org/doc/man/erlang.el.html. > +1 on that. Would be excellent if some generous Emacs+Emacs-Erlang+Distel+Etest expert, with some time in hand, could list down the commonly used key-strokes in some kind of a handy 1 pager. How most of these experts use Emacs is enviously slick, and would be great to get better. Being a predominantly vi/vim user, Emacs doesn't come naturally. -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Tue Jun 28 17:11:38 2011 From: comptekki@REDACTED (Wes James) Date: Tue, 28 Jun 2011 09:11:38 -0600 Subject: [erlang-questions] bug in binary:split with trim? In-Reply-To: <1309255126.30623.YahooMailNeo@web112610.mail.gq1.yahoo.com> References: <1309255126.30623.YahooMailNeo@web112610.mail.gq1.yahoo.com> Message-ID: On Tue, Jun 28, 2011 at 3:58 AM, Pablo Platt wrote: > binary:split with the trim option doesn't remove empty results from the > beginning of the binary: > > binary:split(<<"/a/b/">>, <<"/">>, [global, trim]). > [<<>>,<<"a">>,<<"b">>] > binary:split(<<"//a/b///">>, <<"/">>, [trim, global]). > [<<>>,<<>>,<<"a">>,<<"b">>] > Is this a bug? > Thanks Little more processing, but.... [list_to_binary(X) || X <- string:tokens(binary_to_list(<<"//a/b///">>),"/")]. [<<"a">>,<<"b">>] or maybe like this: [Result]=string:tokens(binary:split(<<"//a/b///">>, <<"/">>, [trim, global]), [<<>>]). [[<<"a">>,<<"b">>]] 11> Result. [<<"a">>,<<"b">>] -wes From g@REDACTED Tue Jun 28 17:33:30 2011 From: g@REDACTED (Garrett Smith) Date: Tue, 28 Jun 2011 09:33:30 -0600 Subject: [erlang-questions] Erlang emacs workflow question In-Reply-To: References: Message-ID: On Thu, May 12, 2011 at 12:39 PM, Mike Oxford wrote: > Write some code > C-c C-k to make sure it compiles > Spin a release (using rebar) in another window > Run the code in another window > Is there a better/more streamlined way to do this, all from within emacs (I > have erl running in another window.) It's a bit more elaborate than what you're asking for, but I use Emacs and an Erlang shell under tmux, which provides a pretty nice workflow. Emacs is in tmux window 0, use for editing and compiling. I have a key binding for 'compile' (C-x C-m) and use a makefile in the src directory. This gives me a quick way to compile the app and use C-x ~ to visit errors/warnings. I run erl in tmux window 1 and always keep it running. I have a 'shell' target in my makefile that handles setting ERL_LIBS (or explicit path settings) and starts any apps that I typically work with under dev. In particular, and very importantly, I always start Mochiweb's reloader module (which would be a great addition to core Erlang!). This will watch loaded modules for changes and automatically reload them. So when I compile from Emacs and switch over to the erl shell, all my latest changes are available. This "long running" shell lets me play around, test, whatever as I modify code in Emacs. It's a more ad hoc mode than re-running tests from a newly started Erlang VM. It also gives you a feel for how Erlang apps can indeed "run forever" with loosely coupled components that can be restarted, reloaded, corrupted, repaired, etc. While I know it's a common work flow, I'm not a fan of generating releases and restarting erl for routine dev changes. I actually can't imagine doing that, given how seamless the 'reloader' workflow is. Btw, tmux in this case is optional -- you could do the same thing in Emacs. I like (strike that, love) tmux because it does for shell sessions what Emacs does for text files. If you happen to do a lot of dev on remote servers over ssh, tmux is a life saver if you lose your connection. Garrett From fred.hebert@REDACTED Tue Jun 28 18:25:31 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Tue, 28 Jun 2011 12:25:31 -0400 Subject: [erlang-questions] Issues with reltool vs. init:stop() In-Reply-To: <541B4472-6E43-4F45-8F33-4233277977B0@erlang-solutions.com> References: <541B4472-6E43-4F45-8F33-4233277977B0@erlang-solutions.com> Message-ID: For anyone interested in playing with the source, I've uploaded it to http://ferd.ca/release.zip Just starting the release should be enough to get it going and printing some results. The init:stop/0 call takes place in erlcount-1.0/src/erlcount_dispatch.erl. The app itself is explained in details at http://learnyousomeerlang.com/the-count-of-applications I'm getting a bit lost trying to find causes and solutions for that issue. I haven't yet dug into reltool's code nor have I checked that the boot/rel scripts have similar instructions, but I guess this will be the next step if nobody's ever had that problem before me. Cheers, -- Fred H?bert http://www.erlang-solutions.com On 2011-06-28, at 07:56 AM, Fr?d?ric Trottier-H?bert wrote: > I'm having quite peculiar issues when it comes to shutting down the VM from within a release. > > I'm writing a little demo application to play with reltools. The application is using files from LYSE's last chapters on applications, but I believe my problem to be independent from there. I've modified my code a bit so that when it's done running, 'init:stop()' is called from within one of the gen_servers I have. Whenever I do "application:start(ppool), application:start(erlcount)." and wait, the erlcount application will run, call init:stop(), and everything will shut down fine. > > I then made a release (.rel file not included here) with systools: > > systools:make_script("erlcount-1.0", [local]). > systools:make_tar("erlcount-1.0", [{erts,"/local/lib/erlang"}]) > > And ran it with > > $ ./erts-5.8.4/bin/erl -boot ./releases/1.0.0/start > > from the right directory, which worked well. Init:stop() managed to take the VM down. > > Then I decided to switch to reltool. This is my current config file (not yet having all modules stripped down): > > {sys, [ > {lib_dirs, ["/home/ferd/code/learn-you-some-erlang/release/"]}, > {rel, "erlcount", "1.0.0", [kernel, stdlib, {ppool, permanent}, {erlcount, permanent}]}, > {boot_rel, "erlcount"}, > {relocatable, true}, > {profile, standalone}, > {app, ppool, [{vsn, "1.0.0"}, {app_file, all}, {debug_info, keep}]}, > {app, erlcount, [{vsn, "1.0.0"}, {incl_cond, include}, {app_file, strip}, {debug_info, strip}]} > ]}. > > I then run the following commands, without an error: > > 1> {ok, Conf} = file:consult("erlcount-1.0.config"). > 2> {ok, Spec} = reltool:get_target_spec(Conf). > 3> reltool:eval_target_spec(Spec, code:root_dir(), "./rel"). > > Then, when I call "./rel/bin/erl" from the shell, the application starts normally, does everything it needs to do, but init:stop() will hang forever and the VM will only stop if I start it with -shutdown_timeout SomeDelay that will kill everything. > > The same problem remains if I make a more stripped-down reltools release by using incl_cond on the apps I want to only keep the core of what's necessary. > > I've googled around a bit and have not found anything that might explain this behaviour. My application's code doesn't seem to be the culprit given it works normally from the shell, and also works normally with systools, but fails to quit with reltools. I've also tried calling './rel/bin/erl -run init stop' -- it seems that it kills part of the VM's apps (no output coming, or only partial output from the app), but will still fail to take the whole VM down. > > Anyone has an idea or has had similar problems? Pointers or idea of what to explore to resolve this? I've tried it both with R14B02 and R14B03 and it keeps the same behaviour there. Reltool seems to break init:stop for my release. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From moxford@REDACTED Tue Jun 28 20:38:52 2011 From: moxford@REDACTED (Mike Oxford) Date: Tue, 28 Jun 2011 11:38:52 -0700 Subject: [erlang-questions] Erlang emacs workflow question In-Reply-To: References: Message-ID: Can you post your makefile for example? I'm currently spinning releases for every dev change and if you've got a better way I'm dying to hear! -mox On Tue, Jun 28, 2011 at 8:33 AM, Garrett Smith wrote: > On Thu, May 12, 2011 at 12:39 PM, Mike Oxford wrote: >> Write some code >> C-c C-k to make sure it compiles >> Spin a release (using rebar) in another window >> Run the code in another window >> Is there a better/more streamlined way to do this, all from within emacs (I >> have erl running in another window.) > > It's a bit more elaborate than what you're asking for, but I use Emacs > and an Erlang shell under tmux, which provides a pretty nice workflow. > > Emacs is in tmux window 0, use for editing and compiling. I have a key > binding for 'compile' (C-x C-m) and use a makefile in the src > directory. This gives me a quick way to compile the app and use C-x ~ > to visit errors/warnings. > > I run erl in tmux window 1 and always keep it running. I have a > 'shell' target in my makefile that handles setting ERL_LIBS (or > explicit path settings) and starts any apps that I typically work with > under dev. > > In particular, and very importantly, I always start Mochiweb's > reloader module (which would be a great addition to core Erlang!). > This will watch loaded modules for changes and automatically reload > them. So when I compile from Emacs and switch over to the erl shell, > all my latest changes are available. > > This "long running" shell lets me play around, test, whatever as I > modify code in Emacs. It's a more ad hoc mode than re-running tests > from a newly started Erlang VM. It also gives you a feel for how > Erlang apps can indeed "run forever" with loosely coupled components > that can be restarted, reloaded, corrupted, repaired, etc. > > While I know it's a common work flow, I'm not a fan of generating > releases and restarting erl for routine dev changes. I actually can't > imagine doing that, given how seamless the 'reloader' workflow is. > > Btw, tmux in this case is optional -- you could do the same thing in > Emacs. I like (strike that, love) tmux because it does for shell > sessions what Emacs does for text files. If you happen to do a lot of > dev on remote servers over ssh, tmux is a life saver if you lose your > connection. > > Garrett > From dale@REDACTED Tue Jun 28 20:42:37 2011 From: dale@REDACTED (Dale Harvey) Date: Tue, 28 Jun 2011 19:42:37 +0100 Subject: [erlang-questions] Erlang emacs workflow question In-Reply-To: References: Message-ID: I wrote a little compile helper plugin for emacs, I am not an emacs guru but it helps me https://github.com/daleharvey/dotfiles/blob/master/emacs/compile-helper/compile-helper.el With the following config (setq build-helper-files '(("Makefile" . "make") (".couchapprc" . "couchapp push"))) Whenever I press M-x compile (I bind to M-1) it will take the current directory and search backwards until it finds some file that it recognizes, in this case it is Makefile, I have my Makefile kick off a rebar compile. and am using mochiweb:reloader when running a development to which will hot load any newly compiled beam files. If there are any errors during recompilation then an emacs buffer will popup with a link directory to the line the error was in, I also use distel + flymake mode to get syntax checking as I type so syntax errors are pretty rare. I havent actually added distel to the setup - but the compile helper and erlang mode is in - https://github.com/daleharvey/dotfiles/tree/master/emacs Hope its handy Dale On 28 June 2011 16:33, Garrett Smith wrote: > On Thu, May 12, 2011 at 12:39 PM, Mike Oxford wrote: > > Write some code > > C-c C-k to make sure it compiles > > Spin a release (using rebar) in another window > > Run the code in another window > > Is there a better/more streamlined way to do this, all from within emacs > (I > > have erl running in another window.) > > It's a bit more elaborate than what you're asking for, but I use Emacs > and an Erlang shell under tmux, which provides a pretty nice workflow. > > Emacs is in tmux window 0, use for editing and compiling. I have a key > binding for 'compile' (C-x C-m) and use a makefile in the src > directory. This gives me a quick way to compile the app and use C-x ~ > to visit errors/warnings. > > I run erl in tmux window 1 and always keep it running. I have a > 'shell' target in my makefile that handles setting ERL_LIBS (or > explicit path settings) and starts any apps that I typically work with > under dev. > > In particular, and very importantly, I always start Mochiweb's > reloader module (which would be a great addition to core Erlang!). > This will watch loaded modules for changes and automatically reload > them. So when I compile from Emacs and switch over to the erl shell, > all my latest changes are available. > > This "long running" shell lets me play around, test, whatever as I > modify code in Emacs. It's a more ad hoc mode than re-running tests > from a newly started Erlang VM. It also gives you a feel for how > Erlang apps can indeed "run forever" with loosely coupled components > that can be restarted, reloaded, corrupted, repaired, etc. > > While I know it's a common work flow, I'm not a fan of generating > releases and restarting erl for routine dev changes. I actually can't > imagine doing that, given how seamless the 'reloader' workflow is. > > Btw, tmux in this case is optional -- you could do the same thing in > Emacs. I like (strike that, love) tmux because it does for shell > sessions what Emacs does for text files. If you happen to do a lot of > dev on remote servers over ssh, tmux is a life saver if you lose your > connection. > > Garrett > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Tue Jun 28 20:55:17 2011 From: g@REDACTED (Garrett Smith) Date: Tue, 28 Jun 2011 12:55:17 -0600 Subject: [erlang-questions] Best Linux Distro for Erlang Devel In-Reply-To: References: Message-ID: On Sun, Jun 19, 2011 at 4:11 PM, Tom Murphy wrote: > ?If my choice of Lunix distro depended 100% on its solidness as an > Erlang devel platform (completeness of base, packages, stability, > support, robustness), what would you all recommend? I like Arch, which is a solid "rolling release" distro. I've found the ecosystem to be consistent with their religion: https://wiki.archlinux.org/index.php/The_Arch_Way Arch has a friendly packager that would let you build software (e.g. say you wanted a particular variant of Erlang) that fit cleanly into the system package management scheme. Garrett From g@REDACTED Tue Jun 28 23:12:06 2011 From: g@REDACTED (Garrett Smith) Date: Tue, 28 Jun 2011 15:12:06 -0600 Subject: [erlang-questions] Erlang emacs workflow question In-Reply-To: References: Message-ID: On Tue, Jun 28, 2011 at 12:38 PM, Mike Oxford wrote: > Can you post your makefile for example? > > I'm currently spinning releases for every dev change and if you've got > a better way I'm dying to hear! I posted a sample project in this repo: https://github.com/handsoncode/erlang See dev-workflow/makefile-reloader. Garrett From fred.hebert@REDACTED Wed Jun 29 01:50:04 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Tue, 28 Jun 2011 19:50:04 -0400 Subject: [erlang-questions] Issues with reltool vs. init:stop() In-Reply-To: References: <541B4472-6E43-4F45-8F33-4233277977B0@erlang-solutions.com> Message-ID: Let me close this by calling it 'RESOLVED'. I've found the time to analyse the scripts returned for both systools and reltool. It turns out that systools, no matter what dependencies you insert, will pretty much always start stdlib and kernel as part of the system (and primLoads them). Reltool will do that, but for ALL applications in your release. This means that if you forget to make your app depend on stdlib (which was my case, because I'm dumb), it's loaded alright, but I'm guessing that a node will be taken down by removing stdlib before your own app, and then everything stalls. As soon as I made my apps depend on stdlib, things went fine. The more you know. -- Fred H?bert http://www.erlang-solutions.com On 2011-06-28, at 12:25 PM, Fr?d?ric Trottier-H?bert wrote: > For anyone interested in playing with the source, I've uploaded it to http://ferd.ca/release.zip > > Just starting the release should be enough to get it going and printing some results. The init:stop/0 call takes place in erlcount-1.0/src/erlcount_dispatch.erl. The app itself is explained in details at http://learnyousomeerlang.com/the-count-of-applications > > I'm getting a bit lost trying to find causes and solutions for that issue. I haven't yet dug into reltool's code nor have I checked that the boot/rel scripts have similar instructions, but I guess this will be the next step if nobody's ever had that problem before me. > > Cheers, > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-06-28, at 07:56 AM, Fr?d?ric Trottier-H?bert wrote: > >> I'm having quite peculiar issues when it comes to shutting down the VM from within a release. >> >> I'm writing a little demo application to play with reltools. The application is using files from LYSE's last chapters on applications, but I believe my problem to be independent from there. I've modified my code a bit so that when it's done running, 'init:stop()' is called from within one of the gen_servers I have. Whenever I do "application:start(ppool), application:start(erlcount)." and wait, the erlcount application will run, call init:stop(), and everything will shut down fine. >> >> I then made a release (.rel file not included here) with systools: >> >> systools:make_script("erlcount-1.0", [local]). >> systools:make_tar("erlcount-1.0", [{erts,"/local/lib/erlang"}]) >> >> And ran it with >> >> $ ./erts-5.8.4/bin/erl -boot ./releases/1.0.0/start >> >> from the right directory, which worked well. Init:stop() managed to take the VM down. >> >> Then I decided to switch to reltool. This is my current config file (not yet having all modules stripped down): >> >> {sys, [ >> {lib_dirs, ["/home/ferd/code/learn-you-some-erlang/release/"]}, >> {rel, "erlcount", "1.0.0", [kernel, stdlib, {ppool, permanent}, {erlcount, permanent}]}, >> {boot_rel, "erlcount"}, >> {relocatable, true}, >> {profile, standalone}, >> {app, ppool, [{vsn, "1.0.0"}, {app_file, all}, {debug_info, keep}]}, >> {app, erlcount, [{vsn, "1.0.0"}, {incl_cond, include}, {app_file, strip}, {debug_info, strip}]} >> ]}. >> >> I then run the following commands, without an error: >> >> 1> {ok, Conf} = file:consult("erlcount-1.0.config"). >> 2> {ok, Spec} = reltool:get_target_spec(Conf). >> 3> reltool:eval_target_spec(Spec, code:root_dir(), "./rel"). >> >> Then, when I call "./rel/bin/erl" from the shell, the application starts normally, does everything it needs to do, but init:stop() will hang forever and the VM will only stop if I start it with -shutdown_timeout SomeDelay that will kill everything. >> >> The same problem remains if I make a more stripped-down reltools release by using incl_cond on the apps I want to only keep the core of what's necessary. >> >> I've googled around a bit and have not found anything that might explain this behaviour. My application's code doesn't seem to be the culprit given it works normally from the shell, and also works normally with systools, but fails to quit with reltools. I've also tried calling './rel/bin/erl -run init stop' -- it seems that it kills part of the VM's apps (no output coming, or only partial output from the app), but will still fail to take the whole VM down. >> >> Anyone has an idea or has had similar problems? Pointers or idea of what to explore to resolve this? I've tried it both with R14B02 and R14B03 and it keeps the same behaviour there. Reltool seems to break init:stop for my release. >> >> -- >> Fred H?bert >> http://www.erlang-solutions.com >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From bernie@REDACTED Wed Jun 29 02:32:30 2011 From: bernie@REDACTED (Bernie Duggan) Date: Wed, 29 Jun 2011 10:32:30 +1000 Subject: [erlang-questions] common_test and multiple nodes Message-ID: Hi list, I'm in the process of writing up some common_test based tests for a system we're developing. While common_test seems to basically be pretty cool, I have one thing that I can't quite see how to do (of course I may be looking at the problem entirely the wrong way, too). Our code can be looked at as behaving like a client-server system (actually it's more like a peer-peer, but it's easier to explain this way and doesn't really affect the question). What I'd like to be able to do is fire up a server on one node with some test hooks (so that we can watch what's happening), and a client on another node. Then, we have tests running on both of them to ensure that a) the client is seeing the responses it's expecting, and b) the server is behaving internally the way we expect for the given client input. It seems like common_test has /most/ of what I want for this as part of the "large scale testing" features (ability to start up remote nodes and run tests on them, having failures fed back to the master node etc). Except that I can't see a way to guarantee that, for example, client_test_1 runs on the client node at the same time as the corresponding server_test_1 runs on the server node. It looks like the large scale testing stuff is basically designed to run a whole pile of tests in parallel across multiple nodes, without any interaction between them. Is this correct? If not, what am I missing? If so, are there any suggestions for an alternative approach to the problem? Cheers, Bernard -------------- next part -------------- An HTML attachment was scrubbed... URL: From wmacgyver@REDACTED Wed Jun 29 08:44:45 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Wed, 29 Jun 2011 02:44:45 -0400 Subject: [erlang-questions] recommended redis client? In-Reply-To: References: Message-ID: <499120DA-42AC-4154-8323-CA844ECD7B15@gmail.com> Thanks everyone for the suggestions. Looks like I get to try at least a few of them. :) From lukas.larsson@REDACTED Wed Jun 29 10:01:28 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Wed, 29 Jun 2011 08:01:28 +0000 (GMT) Subject: [erlang-questions] common_test and multiple nodes In-Reply-To: <1102739946.25351309334441064.JavaMail.root@zimbra> Message-ID: <419954271.25381309334488550.JavaMail.root@zimbra> Hello! Large scale testing is indeed for running unrelated tests across several machines at the same time and then gathering the results. The tests for testing common_test sound very similar to what you want to do. What we do there is that we in the init_per_testcase startup X number of nodes and install our own event handlers on those nodes which register the events that happen and send them to the common_test node. Then we verify that the events which we received from the nodes is one of the possible (race conditions suck) outcomes of the test run which we are testing. You can find the test suite here: https://github.com/erlang/otp/blob/dev/lib/common_test/test/ct_hooks_SUITE.erl Keep in mind that the SUITE was originally written to run on only the test_server, so there are some strange things going on in there which could be a lot better done now that ct is used to run the tests. For instance you probably want to use ct_slave to start and monitor nodes which you start. Lukas ----- Original Message ----- From: "Bernie Duggan" To: "Erlang Questions" Sent: Wednesday, June 29, 2011 2:32:30 AM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: [erlang-questions] common_test and multiple nodes Hi list, I'm in the process of writing up some common_test based tests for a system we're developing. While common_test seems to basically be pretty cool, I have one thing that I can't quite see how to do (of course I may be looking at the problem entirely the wrong way, too). Our code can be looked at as behaving like a client-server system (actually it's more like a peer-peer, but it's easier to explain this way and doesn't really affect the question). What I'd like to be able to do is fire up a server on one node with some test hooks (so that we can watch what's happening), and a client on another node. Then, we have tests running on both of them to ensure that a) the client is seeing the responses it's expecting, and b) the server is behaving internally the way we expect for the given client input. It seems like common_test has /most/ of what I want for this as part of the "large scale testing" features (ability to start up remote nodes and run tests on them, having failures fed back to the master node etc). Except that I can't see a way to guarantee that, for example, client_test_1 runs on the client node at the same time as the corresponding server_test_1 runs on the server node. It looks like the large scale testing stuff is basically designed to run a whole pile of tests in parallel across multiple nodes, without any interaction between them. Is this correct? If not, what am I missing? If so, are there any suggestions for an alternative approach to the problem? Cheers, Bernard _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From hynek@REDACTED Wed Jun 29 10:28:13 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Wed, 29 Jun 2011 10:28:13 +0200 Subject: [erlang-questions] bug in binary:split with trim? In-Reply-To: References: <1309255126.30623.YahooMailNeo@web112610.mail.gq1.yahoo.com> Message-ID: Why not simply? [X || X<- binary:split(<<"//a/b///">>, <<"/">>, [trim, global]), X=/=<<>>] On Tue, Jun 28, 2011 at 5:11 PM, Wes James wrote: > On Tue, Jun 28, 2011 at 3:58 AM, Pablo Platt wrote: >> binary:split with the trim option doesn't remove empty results from the >> beginning of the binary: >> >> binary:split(<<"/a/b/">>, <<"/">>, [global, trim]). >> [<<>>,<<"a">>,<<"b">>] >> binary:split(<<"//a/b///">>, <<"/">>, [trim, global]). >> [<<>>,<<>>,<<"a">>,<<"b">>] >> Is this a bug? >> Thanks > > Little more processing, but.... > > > [list_to_binary(X) || X <- string:tokens(binary_to_list(<<"//a/b///">>),"/")]. > > [<<"a">>,<<"b">>] > > > or maybe like this: > > [Result]=string:tokens(binary:split(<<"//a/b///">>, <<"/">>, [trim, > global]), [<<>>]). > [[<<"a">>,<<"b">>]] > 11> Result. > [<<"a">>,<<"b">>] > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From Lionel.Cons@REDACTED Wed Jun 29 13:04:04 2011 From: Lionel.Cons@REDACTED (Lionel Cons) Date: Wed, 29 Jun 2011 11:04:04 +0000 Subject: [erlang-questions] Using a different cookie path Message-ID: <1D9CBD89533EE94D8295DFBFD923C23F2625EE59@PLOXCHG03.cern.ch> I would like to control which cookie Erlang will use. In my environment, it's not practical to let Erlang use ~/.erlang.cookie. I've found the -setcookie option for erl but this takes the cookie itself, not a file path. This is insecure since any user could list the Unix processes and see the cookie on the command line. Is there a way to tell Erlang to get its cookie from a file other than ~/.erlang.cookie? Thanks in adavnce for your advices, Lionel -------------- next part -------------- An HTML attachment was scrubbed... URL: From matti.oinas@REDACTED Wed Jun 29 13:22:55 2011 From: matti.oinas@REDACTED (Matti Oinas) Date: Wed, 29 Jun 2011 14:22:55 +0300 Subject: [erlang-questions] xmerl_xsd can't find schema Message-ID: Hi I'm trying to validate XML using XML schema but xmer_xsd:process_schema can't find schema in any other folder than the current pwd(). >xmerl_xsd:process_schema("../include/user.xsd"). {error,enoent} if I copy the user.xsd file into the folder returned by pwd() which happens to be src and modify the path to point to that copy the everything works just fine. >xmerl_xsd:process_schema("user.xsd"). {ok,{xsd_state,"user.xsd",...... xmerl_scan can find the file in both folders >xmerl_scan:file("../include/user.xsd"). {{xmlElement,'xs:schema','xs:schema',.... >xmerl_scan:file("user.xsd"). {{xmlElement,'xs:schema','xs:schema',.... It seems to be a bug or then I'm missing some point. I'm using erlang from arch(32 bit) linux repositories. Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] From michael.santos@REDACTED Wed Jun 29 13:53:53 2011 From: michael.santos@REDACTED (Michael Santos) Date: Wed, 29 Jun 2011 07:53:53 -0400 Subject: [erlang-questions] Using a different cookie path In-Reply-To: <1D9CBD89533EE94D8295DFBFD923C23F2625EE59@PLOXCHG03.cern.ch> References: <1D9CBD89533EE94D8295DFBFD923C23F2625EE59@PLOXCHG03.cern.ch> Message-ID: <20110629115353.GA4887@ecn.lan> On Wed, Jun 29, 2011 at 11:04:04AM +0000, Lionel Cons wrote: > I would like to control which cookie Erlang will use. In my > environment, it's not practical to let Erlang use ~/.erlang.cookie. > > I've found the -setcookie option for erl but this takes the cookie > itself, not a file path. This is insecure since any user could list > the Unix processes and see the cookie on the command line. Erlang cookie auth really isn't secure anyway. > Is there a way to tell Erlang to get its cookie from a file other than > ~/.erlang.cookie? {ok, Cookie} = file:read_file("/path/to/cookie"), erlang:set_cookie(binary_to_atom(Cookie, latin1)). > Thanks in adavnce for your advices, > > Lionel From spawn.think@REDACTED Wed Jun 29 13:57:18 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Wed, 29 Jun 2011 13:57:18 +0200 Subject: [erlang-questions] xmerl_xsd can't find schema In-Reply-To: References: Message-ID: It sounds like a big. xmerl_xsd for some reason will mess up your path. I think this patch should solve your problem index e56f147..3085bc7 100644 --- a/lib/xmerl/src/xmerl_xsd.erl +++ b/lib/xmerl/src/xmerl_xsd.erl @@ -288,9 +288,9 @@ process_schema(Schema) -> %% or a single error encountered during the processing. process_schema(Schema,Options) when is_list(Options) -> S = initiate_state(Options,Schema), - process_schema2(xmerl_scan:file(filename:join(S#xsd_state.xsd_base, Schema)),S,Schema); + process_schema2(xmerl_scan:file(filename:join(S#xsd_state.xsd_base, filename:basename(Schema))),S,Schema); process_schema(Schema,State) when is_record(State,xsd_state) -> - process_schema2(xmerl_scan:file(filename:join(State#xsd_state.xsd_base, Schema)),State,Schema). + process_schema2(xmerl_scan:file(filename:join(State#xsd_state.xsd_base, filename:basename(Schema))),State,Schema). process_schema2(Err={error,_},_,_) -> Err; On Wed, Jun 29, 2011 at 1:22 PM, Matti Oinas wrote: > Hi > > I'm trying to validate XML using XML schema but > xmer_xsd:process_schema can't find schema in any other folder than the > current pwd(). > > >xmerl_xsd:process_schema("../include/user.xsd"). > {error,enoent} > > if I copy the user.xsd file into the folder returned by pwd() which > happens to be src and modify the path to point to that copy the > everything works just fine. > > >xmerl_xsd:process_schema("user.xsd"). > {ok,{xsd_state,"user.xsd",...... > > xmerl_scan can find the file in both folders > > >xmerl_scan:file("../include/user.xsd"). > {{xmlElement,'xs:schema','xs:schema',.... > >xmerl_scan:file("user.xsd"). > {{xmlElement,'xs:schema','xs:schema',.... > > It seems to be a bug or then I'm missing some point. > > I'm using erlang from arch(32 bit) linux repositories. > > Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] > [hipe] [kernel-poll:false] > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think -------------- next part -------------- An HTML attachment was scrubbed... URL: From toby@REDACTED Wed Jun 29 13:57:34 2011 From: toby@REDACTED (Toby Thain) Date: Wed, 29 Jun 2011 07:57:34 -0400 Subject: [erlang-questions] Best Linux Distro for Erlang Devel In-Reply-To: References: Message-ID: <4E0B132E.8040302@telegraphics.com.au> On 28/06/11 2:55 PM, Garrett Smith wrote: > On Sun, Jun 19, 2011 at 4:11 PM, Tom Murphy wrote: > >> If my choice of Lunix distro depended 100% on its solidness as an >> Erlang devel platform (completeness of base, packages, stability, >> support, robustness), what would you all recommend? > > I like Arch, which is a solid "rolling release" distro. I've found the > ecosystem to be consistent with their religion: I would second the choice of a rolling release system, for flexibility. I choose Gentoo myself but I hear good things about Arch. --Toby > > https://wiki.archlinux.org/index.php/The_Arch_Way > > Arch has a friendly packager that would let you build software (e.g. > say you wanted a particular variant of Erlang) that fit cleanly into > the system package management scheme. > > Garrett > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From lars@REDACTED Wed Jun 29 14:12:19 2011 From: lars@REDACTED (Lars Thorsen) Date: Wed, 29 Jun 2011 14:12:19 +0200 Subject: [erlang-questions] xmerl_xsd can't find schema In-Reply-To: References: Message-ID: <4E0B16A3.6080800@erix.ericsson.se> Hi Matti, I looked through the xmerl_xsd code and it is a bug in process_schema. I must just check so the calls from xmerl_scan to xmerl_xsd when validating an xml file works ok with the change. A workaround while waiting for the fix is: xmerl_xsd:process_schema(, [{xsdbase, }]). There is just one thing that I wonder about, the only case that works for me when I use a path to the schema is ../dir/file.xsd because ../dir/../dir/file.xsd is the same place. Regards Lars On 06/29/2011 01:22 PM, Matti Oinas wrote: > Hi > > I'm trying to validate XML using XML schema but > xmer_xsd:process_schema can't find schema in any other folder than the > current pwd(). > >> xmerl_xsd:process_schema("../include/user.xsd"). > {error,enoent} > > if I copy the user.xsd file into the folder returned by pwd() which > happens to be src and modify the path to point to that copy the > everything works just fine. > >> xmerl_xsd:process_schema("user.xsd"). > {ok,{xsd_state,"user.xsd",...... > > xmerl_scan can find the file in both folders > >> xmerl_scan:file("../include/user.xsd"). > {{xmlElement,'xs:schema','xs:schema',.... >> xmerl_scan:file("user.xsd"). > {{xmlElement,'xs:schema','xs:schema',.... > > It seems to be a bug or then I'm missing some point. > > I'm using erlang from arch(32 bit) linux repositories. > > Erlang R14B (erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] > [hipe] [kernel-poll:false] > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From dang@REDACTED Wed Jun 29 16:19:46 2011 From: dang@REDACTED (Daniel Goertzen) Date: Wed, 29 Jun 2011 09:19:46 -0500 Subject: [erlang-questions] OTP roadmap slides from Erlang Factory? In-Reply-To: References: Message-ID: +1 There are a number of presentations from the Erlang Factory I would love to see. Dan. On Sun, Jun 26, 2011 at 1:20 PM, Paul Fisher wrote: > Kenneth, > > Can you publish your slides from your Erlang Factory talk on the OTP > Roadmap? I apologize if you have already done so, I was looking on the > Erlang Factory site and a link was not available. > > > -- > paul > > director, platform services > alertlogic, inc. > 713-484-8383 x2314 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Daniel Goertzen ----------------- dang@REDACTED (work) daniel.goertzen@REDACTED (home) ----------------- 1 204 272 6149 (home/office) 1 204 470 8360 (mobile) ----------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed Jun 29 17:05:24 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 29 Jun 2011 16:05:24 +0100 Subject: [erlang-questions] Using a different cookie path In-Reply-To: <20110629115353.GA4887@ecn.lan> References: <1D9CBD89533EE94D8295DFBFD923C23F2625EE59@PLOXCHG03.cern.ch> <20110629115353.GA4887@ecn.lan> Message-ID: On 29 June 2011 12:53, Michael Santos wrote: > On Wed, Jun 29, 2011 at 11:04:04AM +0000, Lionel Cons wrote: >> I would like to control which cookie Erlang will use. In my >> environment, it's not practical to let Erlang use ~/.erlang.cookie. >> >> I've found the -setcookie option for erl but this takes the cookie >> itself, not a file path. This is insecure since any user could list >> the Unix processes and see the cookie on the command line. > > Erlang cookie auth really isn't secure anyway. This is definitely true. > >> Is there a way to tell Erlang to get its cookie from a file other than >> ~/.erlang.cookie? > There is another option. Pass -vmargs pointing to a 'vmargs' file which contains the -setcookie option. This (cookie) won't appear on the command line, although the path to the vmargs file will, obviously. From daniel.goertzen@REDACTED Wed Jun 29 22:03:06 2011 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 29 Jun 2011 15:03:06 -0500 Subject: [erlang-questions] rebar: Unable to generate spec Message-ID: I can use rebar to compile an app and create a release template, but I am stuck on "generate": $ rebar.bat generate ==> rel (generate) ERROR: Unable to generate spec: read file info c:/erlang/bin/rebar.bat failed - I am running Windows 7. - Erlang is installed in C:\erlang - Rebar was built from today's git, and I copied rebar and rebar.bat into c:\erlang\bin - Rebar is invoked from cygwin Any ideas on what might be wrong? Thanks, Dan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.joseph.davis@REDACTED Thu Jun 30 00:11:06 2011 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Wed, 29 Jun 2011 18:11:06 -0400 Subject: [erlang-questions] Half word emulators and NIFs In-Reply-To: <4E085563.1050106@erix.ericsson.se> References: <4E085563.1050106@erix.ericsson.se> Message-ID: On Mon, Jun 27, 2011 at 6:03 AM, Sverker Eriksson wrote: > Have you tested with a debug built emulator. That may catch faulty NIF > behaviors earlier. Otherwise if a NIF builds broken terms on the process > heap it might not be discovered until later when those terms are matched or > maybe garbage collected. > > There is no difference to make NIF's for the halfword emulator. Just use the > NIF API as intended and don't cheat (make assumptions about > sizeof(ERL_NIF_TERM) for example). > > /Sverker, Erlang/OTP > I've reduced this to a failing test case in the NIF API [1]. In the end it boils down to these two functions (notice one is int64, the other not): ERL_NIF_TERM run_test1(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { return enif_make_int64(env, 1); } ERL_NIF_TERM run_test2(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { return enif_make_int(env, 1); } Test 1 returns a Value V where (V =:= 1) is false, but (V == 1) is true. The second function returns a value where both comparisons are true. Paolo also found that amusingly this bug disappears at 2^27 as shown by: ERL_NIF_TERM run_test3(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { return enif_make_int64(env, 134217727); } ERL_NIF_TERM run_test4(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { return enif_make_int64(env, 134217728); } run_test3 returns a value that can't be pattern matched against 134217727 but test 4 returns a value that can be matched against 134217728. Also, I built R14B03 on Ubuntu 10.04 with this configure: ./configure --enable-smp --enable-threads --enable-m64-build --without-javac --enable-halfword-emulator The processors are a pair of dual core Xeon's if that's important at all. [1] https://github.com/davisp/halfwordtest From emmiller@REDACTED Thu Jun 30 03:30:59 2011 From: emmiller@REDACTED (Evan Miller) Date: Wed, 29 Jun 2011 20:30:59 -0500 Subject: [erlang-questions] Two useful libraries (which now exist) Message-ID: A month ago I asked about the possible existence of three libraries I thought would be pretty useful. I've started to piece together two of them: 1. This fork of erl_img has JPEG read support. PNG write support, and crop/scale routines: https://github.com/evanmiller/erl_img Thanks to Tony Rogvall for sharing his JPEG code. The image scaling algorithm uses bicubic interpolation; it doesn't yet work on paletted images. 2. Here's a library I'm writing that can partially read and write RTF, HTML, BBCode, and Textile: https://github.com/evanmiller/jerome It is called Jerome in honor of the multi-lingual saint. Jerome is alpha quality but I've gotten some mileage out of it. It's nice being able to "choose your own adventure" with markup languages. Upload as RTF, edit as Textile, format as HTML, etc. It'd be nice to get more languages and syntax features supported; patches are most welcome. Regards, Evan From peralta.alejandro@REDACTED Thu Jun 30 04:06:54 2011 From: peralta.alejandro@REDACTED (Ale) Date: Wed, 29 Jun 2011 23:06:54 -0300 Subject: [erlang-questions] How to write a generic IRC client the OTP way? Message-ID: Hello All, As a learning experience I want to make a generic IRC client. My question is regarding the path to genericness, what would be the right way to do this, or the OTP way? At first I thought that I could use gen_server to hide logic there. A client would call gen_server:call(ircclient, {nick, ale}) to set the nickname for example. Since call is synchronous, and setting the nick in irc, might take a couple of (micro)seconds, the process calling will have to wait until it receives an answer and the client (calling gen_server) might loose some interactivity (or the ability to do something else while the response of the "NICK" irc command comes back) The next thought was to use gen_server:cast(ircclient, {nick, ale}), but then how would the client would get the response? As far as I understood there is no way to get a deferred reply from gen_server. The calling process has to wait, even if we use gen_server:call with norepley and then gen_server:reply (which as far as I understood it is useful, when making the gen_server container or process non blocking). I could pass a callback function to gen_server:call, like this gen_server:cast(ircclient, {nick, ale, fun CallbackFun/1}), and CallbackFun takes the result and does something with it. This migh work, but then that doesn't look very "OTP" to me. This is what I'm having trouble the most understanding if "gen_*" processes can call back in a deferred manner to the client using the "gen_*" process. "Hey gen_server, call me back when you get this done". Probably my design is wrong? Finally researching a bit, I thought that it might be a good idea to write a behaviour. But the same question comes up, how do I callback the client? (diagram: http://i.imgur.com/6N6Je.png) Hopefully I explained my problem, and questions. Thanks, Regards, -- Ale. From jwatte@REDACTED Thu Jun 30 05:08:55 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 29 Jun 2011 20:08:55 -0700 Subject: [erlang-questions] Half word emulators and NIFs In-Reply-To: References: <4E085563.1050106@erix.ericsson.se> Message-ID: FWIW, The implementation of Erlang treats integers larger than 2^27 as "bignums" which are separate from "regular integers." The reason it's not 2^31 or 2^32 is that Erlang uses a few bits for "tag bits," probably to optimize the storage of these numbers within the garbage collected heap. This also means that Erlang math arithmetic is not especially zippy compared to most other languages, because the tag bits must generally be removed for each operation. Chances are that making an int64 *always* generates a bignum, whereas making an int only makes a bignum if it's necessary. If the assumption is that integers < 2^27 in magnitude are always little-ints, then it seems like the behavior of making int64 is arguably broken for such numbers. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Wed, Jun 29, 2011 at 3:11 PM, Paul Davis wrote: > On Mon, Jun 27, 2011 at 6:03 AM, Sverker Eriksson > wrote: > > Have you tested with a debug built emulator. That may catch faulty NIF > > behaviors earlier. Otherwise if a NIF builds broken terms on the process > > heap it might not be discovered until later when those terms are matched > or > > maybe garbage collected. > > > > There is no difference to make NIF's for the halfword emulator. Just use > the > > NIF API as intended and don't cheat (make assumptions about > > sizeof(ERL_NIF_TERM) for example). > > > > /Sverker, Erlang/OTP > > > > I've reduced this to a failing test case in the NIF API [1]. > > In the end it boils down to these two functions (notice one is int64, > the other not): > > ERL_NIF_TERM > run_test1(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) > { > return enif_make_int64(env, 1); > } > > ERL_NIF_TERM > run_test2(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) > { > return enif_make_int(env, 1); > } > > Test 1 returns a Value V where (V =:= 1) is false, but (V == 1) is > true. The second function returns a value where both comparisons are > true. > > Paolo also found that amusingly this bug disappears at 2^27 as shown by: > > ERL_NIF_TERM > run_test3(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) > { > return enif_make_int64(env, 134217727); > } > > ERL_NIF_TERM > run_test4(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) > { > return enif_make_int64(env, 134217728); > } > > run_test3 returns a value that can't be pattern matched against > 134217727 but test 4 returns a value that can be matched against > 134217728. > > Also, I built R14B03 on Ubuntu 10.04 with this configure: > > ./configure --enable-smp --enable-threads --enable-m64-build > --without-javac --enable-halfword-emulator > > The processors are a pair of dual core Xeon's if that's important at all. > > [1] https://github.com/davisp/halfwordtest > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Thu Jun 30 05:12:20 2011 From: comptekki@REDACTED (Wes James) Date: Wed, 29 Jun 2011 21:12:20 -0600 Subject: [erlang-questions] json in erlang Message-ID: I see this: http://www.erlang.org/eeps/eep-0018.html but it doesn't exist in erlang proper (it seems). Does that mean the eep hasn't been accepted and won't be implemented? What does the S -> standards track eep mean? just wondering. thx, -wes From jwatte@REDACTED Thu Jun 30 05:12:27 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 29 Jun 2011 20:12:27 -0700 Subject: [erlang-questions] How to write a generic IRC client the OTP way? In-Reply-To: References: Message-ID: A client would call gen_server:call(ircclient, {nick, ale}) to set the nickname for example. The "Erlang Way" is said to be to export actual functions (setnick, getnick, ...) from the gen_server module public API, and have those functions do the gen_server calls. This means that you can change the gen_server message format without breaking compatibility. Also, for synchronous calls (where you want the status), waiting the necessary amount of time is pretty much what you expect to do. You *can* make arrangements to use cast, and pass in a respond-to-PID parameter as part of the request, but in most cases, that adds significant extra complexity for very little benefit. Note that gen_server:call also takes a timeout parameter -- it defaults to 5,000 milliseconds, so if the gen_server is "wedged" then the timeout will kick in. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jack@REDACTED Thu Jun 30 05:32:41 2011 From: jack@REDACTED (Jack Moffitt) Date: Wed, 29 Jun 2011 21:32:41 -0600 Subject: [erlang-questions] json in erlang In-Reply-To: References: Message-ID: > http://www.erlang.org/eeps/eep-0018.html > > but it doesn't exist in erlang proper (it seems). ?Does that mean the > eep hasn't been accepted and won't be implemented? ?What does the S -> > standards track eep mean? I don't know the status of the EEP, but pretty much everyone I know has standardized on usage of mochijson2. In fact, the only project that I've run across that uses something different is emongo, which does it's own thing. jack. From g9414002.pccu.edu.tw@REDACTED Thu Jun 30 06:21:22 2011 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Thu, 30 Jun 2011 12:21:22 +0800 Subject: [erlang-questions] json in erlang In-Reply-To: References: Message-ID: Hi, everyone. I'm stupid that I tried making a JSON parser when I knew mochhijson2 but did not know how to use. And I saw eep-018 too. Eventually I tried making the parser in combinator fashion, but I did not overcome lazy-evaluation part of combinators. For me, it's a dilemma between to make all lazy or to make few functions lazy. I need work more hard. :) /yhh On Thu, Jun 30, 2011 at 11:32 AM, Jack Moffitt wrote: > > http://www.erlang.org/eeps/eep-0018.html > > > > but it doesn't exist in erlang proper (it seems). Does that mean the > > eep hasn't been accepted and won't be implemented? What does the S -> > > standards track eep mean? > > I don't know the status of the EEP, but pretty much everyone I know > has standardized on usage of mochijson2. In fact, the only project > that I've run across that uses something different is emongo, which > does it's own thing. > > jack. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best Regards. --- Y-H. H. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Thu Jun 30 06:27:47 2011 From: mihai@REDACTED (Mihai Balea) Date: Thu, 30 Jun 2011 00:27:47 -0400 Subject: [erlang-questions] json in erlang In-Reply-To: References: Message-ID: <425BDED4-8DFD-475A-AD53-97A2F920CB45@hates.ms> On Jun 29, 2011, at 11:12 PM, Wes James wrote: > I see this: > > http://www.erlang.org/eeps/eep-0018.html > > but it doesn't exist in erlang proper (it seems). Does that mean the > eep hasn't been accepted and won't be implemented? What does the S -> > standards track eep mean? There is at least one implementation of eep0018 in Github. However people seem to prefer mochijson2. Mihai From neumark.peter@REDACTED Thu Jun 30 07:07:59 2011 From: neumark.peter@REDACTED (Peter Neumark) Date: Thu, 30 Jun 2011 07:07:59 +0200 Subject: [erlang-questions] json in erlang In-Reply-To: <425BDED4-8DFD-475A-AD53-97A2F920CB45@hates.ms> References: <425BDED4-8DFD-475A-AD53-97A2F920CB45@hates.ms> Message-ID: The EEP0018 implementation I found was here: https://github.com/talentdeficit/jsx The cool thing about JSX is that it also includes a stream-based parser, so you can even decode incomplete bits of JSON. Peter On Thu, Jun 30, 2011 at 6:27 AM, Mihai Balea wrote: > > On Jun 29, 2011, at 11:12 PM, Wes James wrote: > > > I see this: > > > > http://www.erlang.org/eeps/eep-0018.html > > > > but it doesn't exist in erlang proper (it seems). Does that mean the > > eep hasn't been accepted and won't be implemented? What does the S -> > > standards track eep mean? > > There is at least one implementation of eep0018 in Github. > However people seem to prefer mochijson2. > > Mihai > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Jun 30 07:11:46 2011 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 30 Jun 2011 17:11:46 +1200 Subject: [erlang-questions] json in erlang In-Reply-To: <425BDED4-8DFD-475A-AD53-97A2F920CB45@hates.ms> References: <425BDED4-8DFD-475A-AD53-97A2F920CB45@hates.ms> Message-ID: On 30/06/2011, at 4:27 PM, Mihai Balea wrote: > There is at least one implementation of eep0018 in Github. > However people seem to prefer mochijson2. That's "partial implementation". It doesn't support the option lists described in EEP 18. From mazen.harake@REDACTED Thu Jun 30 08:38:15 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Thu, 30 Jun 2011 08:38:15 +0200 Subject: [erlang-questions] How to write a generic IRC client the OTP way? In-Reply-To: References: Message-ID: Try eirc here: https://github.com/mazenharake/eirc Trying to do synchronous IRC is doomed to fail imho. eirc sends a message everytime you get an event/response and by keeping your own state you put context to these events to know what they mean. IRC wasn't really made for computer programs but rather for a telnet prompt so that is why this protocol is flawed in many ways (but very easy). eirc is under the 2-clause BSD license so do what ever you want with it. On 30 June 2011 04:06, Ale wrote: > Hello All, > > As a learning experience I want to make a generic IRC client. My > question is regarding the path to genericness, what would be the right > way to do this, or the OTP way? > > At first I thought that I could use gen_server to hide logic there. A > client would call gen_server:call(ircclient, {nick, ale}) to set the > nickname for example. Since call is synchronous, and setting the nick > in irc, might take a couple of (micro)seconds, the process calling > will have to wait until it receives an answer and the client (calling > gen_server) might loose some interactivity (or the ability to do > something else while the response of the "NICK" irc command comes > back) > > The next thought was to use gen_server:cast(ircclient, {nick, ale}), > but then how would the client would get the response? As far as I > understood there is no way to get a deferred reply from gen_server. > The calling process has to wait, even if we use gen_server:call with > norepley and then gen_server:reply (which as far as I understood it is > useful, when making the gen_server container or process non blocking). > > I could pass a callback function to gen_server:call, like this > gen_server:cast(ircclient, {nick, ale, fun CallbackFun/1}), and > CallbackFun takes the result and does something with it. This migh > work, but then that doesn't look very "OTP" to me. > > This is what I'm having trouble the most understanding if "gen_*" > processes can call back in a deferred manner to the client using the > "gen_*" process. ?"Hey gen_server, call me back when you get this > done". > > Probably my design is wrong? > > Finally researching a bit, I thought that it might be a good idea to > write a behaviour. But the same question comes up, how do I callback > the client? (diagram: http://i.imgur.com/6N6Je.png) > > Hopefully I explained my problem, and questions. > > Thanks, > > Regards, > -- > Ale. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mjtruog@REDACTED Thu Jun 30 08:48:54 2011 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 29 Jun 2011 23:48:54 -0700 Subject: [erlang-questions] json in erlang In-Reply-To: References: <425BDED4-8DFD-475A-AD53-97A2F920CB45@hates.ms> Message-ID: <4E0C1C56.6010708@gmail.com> Is there an effort to help make jsx (https://github.com/talentdeficit/jsx) satisfy the "option lists" requirement, or is a different implementation being pursued? On 06/29/2011 10:11 PM, Richard O'Keefe wrote: > On 30/06/2011, at 4:27 PM, Mihai Balea wrote: >> There is at least one implementation of eep0018 in Github. >> However people seem to prefer mochijson2. > That's "partial implementation". It doesn't support the > option lists described in EEP 18. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From paolo.negri@REDACTED Thu Jun 30 09:14:39 2011 From: paolo.negri@REDACTED (Paolo Negri) Date: Thu, 30 Jun 2011 09:14:39 +0200 Subject: [erlang-questions] json in erlang In-Reply-To: References: Message-ID: Not related to the eep but we're successfully using jiffy [1] which implements a json parser as a NIF, we preferred it to mochijson2 because of garbage production and cpu usage factors. We previously used a NIF related to the eep [2] but had some limitation we couldn't live with. In the couchdb master repo I've seen popping up ejson [3] another NIF implementation which has also been extracted in its own project [4], I never tried nor I know any detail about ejson but I'd like to have some more information about it and the plan for its inclusion in a stable couchdb release. Both jiffy and ejson use yajl [5] which is a json parser/generator written in ANSI C which is widely used. [1] https://github.com/davisp/jiffy [2] https://github.com/davisp/eep0018 [3] https://github.com/apache/couchdb/tree/trunk/src/ejson [4] https://github.com/benoitc/ejson [5] http://lloyd.github.com/yajl Paolo. On Thu, Jun 30, 2011 at 5:12 AM, Wes James wrote: > I see this: > > http://www.erlang.org/eeps/eep-0018.html > > but it doesn't exist in erlang proper (it seems). ?Does that mean the > eep hasn't been accepted and won't be implemented? ?What does the S -> > standards track eep mean? > > just wondering. > > thx, > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From adam@REDACTED Thu Jun 30 09:30:06 2011 From: adam@REDACTED (Adam Lindberg) Date: Thu, 30 Jun 2011 09:30:06 +0200 Subject: [erlang-questions] problems with PropEr includes from eunit In-Reply-To: References: Message-ID: <4E0C25FE.1080208@erlang-solutions.com> This is a combined bug in rebar and PropEr as far as I can see. See my entry on the rebar mailing list: http://lists.basho.com/pipermail/rebar_lists.basho.com/2011-June/000888.html PropEr relies on the source if the beam file cannot be found (it can't when it's cover compiled), and it doesn't seem to know about ERL_LIBS or the include paths used. Especially since rebar copies the source file to the .eunit folder, it's harder for PropEr to detect linked source files. You might want to reply on the rebar mailing list to get some traction for fixing the problem. Cheers, Adam Martynas Pumputis wrote: > Hello, > > I have problems running PropEr:check_spec/1 from Eunit. > With this line in module file: > -include("bp_app/include/log_utils.hrl"). > PropEr stops working. After removing it, PropEr starts working properly. > > Error message I get from PropEr: > > motiejus:~/mll/gateway_app$ make test_gateway_specs_tests > ERL_FLAGS="-sname eunit -config ../bp_app/config/default.config > +P1000000 -env ERL_MAX_ETS_TABLES 20000" ../rebar verbose=2 > skip_deps=true suite=gateway_specs_tests eunit > ==> gateway_app (eunit) > Compiled tests/gateway_specs_tests.erl > Compiled tests/gateway_app_tests.erl > ======================== EUnit ======================== > gateway_specs_tests: yadda_test (module 'gateway_specs_tests')... > =INFO REPORT==== 27-Jun-2011::17:03:39 === > pid:<0.475.0> > source: "tests/gateway_specs_tests.erl:29" > message: "checkspec says" > error: {typeserver, > {cant_load_code,gateway_helpers, > {cant_find_object_file, > {cant_compile_source_file, > > [{"/home/motiejus/code/mll/gateway_app/.eunit/gateway_helpers.erl", > [{16,epp, > {include,file, > "bp_app/include/log_utils.hrl"}}]}]}}}} > [0.029 s] ok > ======================================================= > Test passed. > > I suspect PropEr recompiles those modules and does not add include paths > properly. > > > Relevant Code: > ===== gateway_helpers.erl ===== > -module(gateway_helpers). > -include("bp_app/include/log_utils.hrl"). > -export([demo/1]). > -spec demo(integer()) -> integer(). > demo(Item) -> > case Item of > 1 -> undefined; > R when is_integer(R) -> > Item > end. > > ===== gateway_specs_tests.erl ===== > -module(gateway_specs_tests). > -include_lib("proper/include/proper.hrl"). > -include_lib("eunit/include/eunit.hrl"). > -include("bp_app/include/log_utils.hrl"). > > yadda_test() -> > ?info("checkspec says", [proper:check_spec({gateway_helpers, demo, 1})]). > > bp_app > ??? include > ? ??? log_utils.hrl > gateway_app > ??? include > ? ??? consts.hrl > ? ??? types.hrl > ??? src > ? ??? gateway_helpers.erl > ??? tests > ??? gateway_specs_tests.erl > > Versions: > Erlang R14A (from Squeeze apt) > PropEr v1.0-1-g5fe56e9 > > Please let me know if you need any more information. > > Motiejus > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From luismarianoguerra@REDACTED Thu Jun 30 11:37:21 2011 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Thu, 30 Jun 2011 10:37:21 +0100 Subject: [erlang-questions] [ANN] efene 0.9 - a language for the erlang vm - released Message-ID: announcement url: http://www.efenelang.org/2011/06/efene-09-language-for-erlang-vm.html efene 0.9 Release Notes ----------------------- About efene ~~~~~~~~~~~ efene is a programming language that runs on the erlang virtual machine. The idea is to provide an alternative syntax to erlang that is most suitable for people coming from languages like Java, C, C++, C#, Javascript. The language is almost 100% compatible with erlang (and will be), the compiler allows to translate an efene source file into a readable erlang one or compile it directly to bytecode. It also adds some syntactic sugar in some places to make some tasks easier. Participate ~~~~~~~~~~~ a mailing list is available at librelist just send a mail to efene@REDACTED to subscribe as first mail you may send a hello world program in efene and present yourself by saying your name, from where you are, how did you heard about efene and anything else you would like to say. you can also participate by helping promote efene by trying it and posting your thoughts on your blog/twitter/facebook/google+ of course, help on documentation or code is more than welcome New Language Syntax ~~~~~~~~~~~~~~~~~~~ * struct expressions * fat arrow expressions * allow new lines after opening list * allow variables in a binary item like: <[Val:Var/binary]> * source code parsing works with windows line break CR+LF * allow lambdas defined as function parameters * allow lightweight struct syntax in ifene * add support for binary generators * add support for "receive after " expression (receive without a pattern) Syntax Changes ~~~~~~~~~~~~~~ * changed syntax to access modules from variables * pretty print structs in the shell License Changes ~~~~~~~~~~~~~~~ efene project is now licensed under the new BSD license, see LICENSE file for details. Tools ~~~~~ * fnc reimplementation in C * fnc builds on windows * -a and -p command line arguments added to fnc (-pa -pz in erl) * unified fnc and fn in a single binary (fnc -r to run programs) * allow empty lines and comment lines on efene shell Extras ~~~~~~ * new domain at http://efenelang.org * first simple version of emacs mode for ifene * basic TextMate support by creating a TextMate bundle Internal Changes ~~~~~~~~~~~~~~~~ * build scripts for windows * Add global build script to build tools, src, and libs * check that all function clauses have the same number of arguments, print a nice error if not * the else branch in the if expression translate to match _ instead of false to avoid an exception when the expression in the if doesn't evaluate to a boolean * pretty print the results in the shell with efene syntax New Modules or Functions ~~~~~~~~~~~~~~~~~~~~~~~~ * make rest parse simplified attributes, just one attribute required to describe a REST API for a function * new struct module * new schema module that allows to define a schema for validation using struct syntax and json schema format to validate structs * removed obj code, libraries and examples * started adding tests to efene libs * third party libs bundled with efene * mochiweb * ibrowser * SimpleBridge Documentation ~~~~~~~~~~~~~ * New features documented. * fnc man pages * documentation on how to build fnc * Remove Go programming language dependency from README Also mention what OSX users have to do * New efene quickstart guide More Information ~~~~~~~~~~~~~~~~ * Issue Tracker: http://github.com/marianoguerra/efene/issues * Central repo: http://github.com/marianoguerra/efene * Documentation: http://marianoguerra.com.ar/efene/docs * Quickstart Guide: http://marianoguerra.com.ar/efene/docs/quickstart.html * Website: http://efenelang.org * Download the latest snapshot: http://github.com/marianoguerra/efene/tarball/master * Rosetta code page: http://rosettacode.org/wiki/Efene From sverker@REDACTED Thu Jun 30 11:59:51 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 30 Jun 2011 11:59:51 +0200 Subject: [erlang-questions] Half word emulators and NIFs In-Reply-To: References: <4E085563.1050106@erix.ericsson.se> Message-ID: <4E0C4917.6060700@erix.ericsson.se> Paul Davis wrote: > On Mon, Jun 27, 2011 at 6:03 AM, Sverker Eriksson > wrote: > >> Have you tested with a debug built emulator. That may catch faulty NIF >> behaviors earlier. Otherwise if a NIF builds broken terms on the process >> heap it might not be discovered until later when those terms are matched or >> maybe garbage collected. >> >> There is no difference to make NIF's for the halfword emulator. Just use the >> NIF API as intended and don't cheat (make assumptions about >> sizeof(ERL_NIF_TERM) for example). >> >> /Sverker, Erlang/OTP >> >> > > I've reduced this to a failing test case in the NIF API [1]. > > In the end it boils down to these two functions (notice one is int64, > the other not): > > ERL_NIF_TERM > run_test1(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) > { > return enif_make_int64(env, 1); > } > > ERL_NIF_TERM > run_test2(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) > { > return enif_make_int(env, 1); > } > > Test 1 returns a Value V where (V =:= 1) is false, but (V == 1) is > true. The second function returns a value where both comparisons are > true. > > Paolo also found that amusingly this bug disappears at 2^27 as shown by: > > ERL_NIF_TERM > run_test3(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) > { > return enif_make_int64(env, 134217727); > } > > ERL_NIF_TERM > run_test4(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) > { > return enif_make_int64(env, 134217728); > } > > run_test3 returns a value that can't be pattern matched against > 134217727 but test 4 returns a value that can be matched against > 134217728. > > Also, I built R14B03 on Ubuntu 10.04 with this configure: > > ./configure --enable-smp --enable-threads --enable-m64-build > --without-javac --enable-halfword-emulator > > The processors are a pair of dual core Xeon's if that's important at all. > > [1] https://github.com/davisp/halfwordtest > > Very nice trouble shooting. Thank you Paul. It's a halfword-bug affecting: enif_make_int64 enif_make_long enif_make_uint64 enif_make_ulong for integer values that can fit into 28 bits. What happens is that it makes a bignum even though the value could have fit into a small integer. Small bignums like that are not allowed internally and that's the reason why it behaves so strangely when comparing. I'll be back with a source patch. /Sverker, Erlang/OTP From carlo.bertoldi@REDACTED Thu Jun 30 16:00:55 2011 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Thu, 30 Jun 2011 16:00:55 +0200 Subject: [erlang-questions] SASL problem Message-ID: <4E0C8197.6070507@ubiquity.it> Hello everybody, I'm trying to launch my application using heart. This is the script I'm using: #!/bin/bash EBIN_PATH=/home/itadm/prototipo_mps/ebin LOG_FILE=/home/itadm/erlang/log/sonda.log ERL=/usr/bin/erl NODE_NAME=sonda $ERL -sname $NODE_NAME -heart -pa $EBIN_PATH -boot start_sasl -sasl sasl_error_logger "{file, \"/home/itadm/erlang/log/prototipo_mps_sasl.log\"}" -noshell -eval "application:start(prototipo_mps)." 2>&1 > $LOG_FILE & The application starts, but in the log I've got this error, and I can see that the sasl keeps restarting. =ERROR REPORT==== 30-Jun-2011::15:36:46 === application_controller: syntax error before: '/': {file, /home/itadm/erlang/log/prototipo_mps_sasl.log} {"init terminating in do_boot",{error,'Bad environment variable: "{file, /home/itadm/erlang/log/prototipo_mps_sasl.log}" Application: sasl'}} =erl_crash_dump:0.1 Thu Jun 30 15:52:32 2011 Slogan: Kernel pid terminated (application_controller) ({application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}}) System version: Erlang R14A (erts-5.8) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] If I launch the application using the shell, it works as expected. Any suggestions? Thanks, Carlo From erlangsiri@REDACTED Thu Jun 30 16:26:35 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Thu, 30 Jun 2011 16:26:35 +0200 Subject: [erlang-questions] SASL problem In-Reply-To: <4E0C8197.6070507@ubiquity.it> References: <4E0C8197.6070507@ubiquity.it> Message-ID: Hi Carlo! Your script works for me, but it seems from the error report that the double quotes around the log file path have disappeared in your case. You could try to increase the number of backslashes - e.g. to three: ... -sasl sasl_error_logger "{file, \\\"/home/itadm/erlang/log/prototipo_mps_sasl.log\\\"}" ... Regards /siri 2011/6/30 Carlo Bertoldi > Hello everybody, > I'm trying to launch my application using heart. This is the script I'm > using: > > #!/bin/bash > > EBIN_PATH=/home/itadm/**prototipo_mps/ebin > LOG_FILE=/home/itadm/erlang/**log/sonda.log > ERL=/usr/bin/erl > NODE_NAME=sonda > > $ERL -sname $NODE_NAME -heart -pa $EBIN_PATH -boot start_sasl -sasl > sasl_error_logger "{file, \"/home/itadm/erlang/log/**prototipo_mps_sasl.log\"}" > -noshell -eval "application:start(prototipo_**mps)." 2>&1 > $LOG_FILE & > > > The application starts, but in the log I've got this error, and I can see > that the sasl keeps restarting. > > > =ERROR REPORT==== 30-Jun-2011::15:36:46 === > application_controller: syntax error before: '/': {file, > /home/itadm/erlang/log/**prototipo_mps_sasl.log} > {"init terminating in do_boot",{error,'Bad environment variable: "{file, > /home/itadm/erlang/log/**prototipo_mps_sasl.log}" Application: sasl'}} > > > =erl_crash_dump:0.1 > Thu Jun 30 15:52:32 2011 > Slogan: Kernel pid terminated (application_controller) > ({application_start_failure,**kernel,{shutdown,{kernel,** > start,[normal,[]]}}}) > System version: Erlang R14A (erts-5.8) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:0] [kernel-poll:false] > > If I launch the application using the shell, it works as expected. Any > suggestions? > Thanks, > Carlo > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker@REDACTED Thu Jun 30 16:27:38 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 30 Jun 2011 16:27:38 +0200 Subject: [erlang-questions] Half word emulators and NIFs In-Reply-To: <4E0C4917.6060700@erix.ericsson.se> References: <4E085563.1050106@erix.ericsson.se> <4E0C4917.6060700@erix.ericsson.se> Message-ID: <4E0C87DA.1090707@erix.ericsson.se> Sverker Eriksson wrote: > Paul Davis wrote: >> On Mon, Jun 27, 2011 at 6:03 AM, Sverker Eriksson >> wrote: >> >>> Have you tested with a debug built emulator. That may catch faulty NIF >>> behaviors earlier. Otherwise if a NIF builds broken terms on the >>> process >>> heap it might not be discovered until later when those terms are >>> matched or >>> maybe garbage collected. >>> >>> There is no difference to make NIF's for the halfword emulator. Just >>> use the >>> NIF API as intended and don't cheat (make assumptions about >>> sizeof(ERL_NIF_TERM) for example). >>> >>> /Sverker, Erlang/OTP >>> >>> >> >> I've reduced this to a failing test case in the NIF API [1]. >> >> In the end it boils down to these two functions (notice one is int64, >> the other not): >> >> ERL_NIF_TERM >> run_test1(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) >> { >> return enif_make_int64(env, 1); >> } >> >> ERL_NIF_TERM >> run_test2(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) >> { >> return enif_make_int(env, 1); >> } >> >> Test 1 returns a Value V where (V =:= 1) is false, but (V == 1) is >> true. The second function returns a value where both comparisons are >> true. >> >> Paolo also found that amusingly this bug disappears at 2^27 as shown by: >> >> ERL_NIF_TERM >> run_test3(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) >> { >> return enif_make_int64(env, 134217727); >> } >> >> ERL_NIF_TERM >> run_test4(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) >> { >> return enif_make_int64(env, 134217728); >> } >> >> run_test3 returns a value that can't be pattern matched against >> 134217727 but test 4 returns a value that can be matched against >> 134217728. >> >> Also, I built R14B03 on Ubuntu 10.04 with this configure: >> >> ./configure --enable-smp --enable-threads --enable-m64-build >> --without-javac --enable-halfword-emulator >> >> The processors are a pair of dual core Xeon's if that's important at >> all. >> >> [1] https://github.com/davisp/halfwordtest >> >> > Very nice trouble shooting. Thank you Paul. > > It's a halfword-bug affecting: > enif_make_int64 > enif_make_long > enif_make_uint64 > enif_make_ulong > > for integer values that can fit into 28 bits. > > What happens is that it makes a bignum even though the value could have > fit into a small integer. Small bignums like that are not allowed > internally > and that's the reason why it behaves so strangely when comparing. > > I'll be back with a source patch. > > /Sverker, Erlang/OTP > Here is a source patch. Only tested on halfword emulator. diff --git a/erts/emulator/beam/erl_nif.c b/erts/emulator/beam/erl_nif.c index 68421b4..d9b1a8e 100644 --- a/erts/emulator/beam/erl_nif.c +++ b/erts/emulator/beam/erl_nif.c @@ -833,8 +833,11 @@ ERL_NIF_TERM enif_make_uint(ErlNifEnv* env, unsigned i) ERL_NIF_TERM enif_make_long(ErlNifEnv* env, long i) { + if (IS_SSMALL(i)) { + return make_small(i); + } #if SIZEOF_LONG == ERTS_SIZEOF_ETERM - return IS_SSMALL(i) ? make_small(i) : small_to_big(i, alloc_heap(env,2)); + return small_to_big(i, alloc_heap(env,2)); #elif SIZEOF_LONG == 8 ensure_heap(env,3); return erts_sint64_to_big(i, &env->hp); @@ -843,8 +846,11 @@ ERL_NIF_TERM enif_make_long(ErlNifEnv* env, long i) ERL_NIF_TERM enif_make_ulong(ErlNifEnv* env, unsigned long i) { + if (IS_USMALL(0,i)) { + return make_small(i); + } #if SIZEOF_LONG == ERTS_SIZEOF_ETERM - return IS_USMALL(0,i) ? make_small(i) : uint_to_big(i,alloc_heap(env,2)); + return uint_to_big(i,alloc_heap(env,2)); #elif SIZEOF_LONG == 8 ensure_heap(env,3); return erts_uint64_to_big(i, &env->hp); /Sverker, Erlang/OTP From carlo.bertoldi@REDACTED Thu Jun 30 16:34:23 2011 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Thu, 30 Jun 2011 16:34:23 +0200 Subject: [erlang-questions] SASL problem In-Reply-To: References: <4E0C8197.6070507@ubiquity.it> Message-ID: <4E0C896F.4010202@ubiquity.it> Thanks, but that didn't solved the problem, and the double quotes are still missing in the log file, even with the 3 backslashes. On 30/06/2011 16:26, Siri Hansen wrote: > Hi Carlo! > > Your script works for me, but it seems from the error report that the > double quotes around the log file path have disappeared in your case. > You could try to increase the number of backslashes - e.g. to three: > > ... -sasl sasl_error_logger "{file, > \\\"/home/itadm/erlang/log/prototipo_mps_sasl.log\\\"}" ... > > Regards > /siri > > > 2011/6/30 Carlo Bertoldi > > > Hello everybody, > I'm trying to launch my application using heart. This is the > script I'm using: > > #!/bin/bash > > EBIN_PATH=/home/itadm/prototipo_mps/ebin > LOG_FILE=/home/itadm/erlang/log/sonda.log > ERL=/usr/bin/erl > NODE_NAME=sonda > > $ERL -sname $NODE_NAME -heart -pa $EBIN_PATH -boot start_sasl > -sasl sasl_error_logger "{file, > \"/home/itadm/erlang/log/prototipo_mps_sasl.log\"}" -noshell -eval > "application:start(prototipo_mps)." 2>&1 > $LOG_FILE & > > > The application starts, but in the log I've got this error, and I > can see that the sasl keeps restarting. > > > =ERROR REPORT==== 30-Jun-2011::15:36:46 === > application_controller: syntax error before: '/': {file, > /home/itadm/erlang/log/prototipo_mps_sasl.log} > {"init terminating in do_boot",{error,'Bad environment variable: > "{file, /home/itadm/erlang/log/prototipo_mps_sasl.log}" > Application: sasl'}} > > > =erl_crash_dump:0.1 > Thu Jun 30 15:52:32 2011 > Slogan: Kernel pid terminated (application_controller) > ({application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}}) > System version: Erlang R14A (erts-5.8) [source] [64-bit] [smp:4:4] > [rq:4] [async-threads:0] [kernel-poll:false] > > If I launch the application using the shell, it works as expected. > Any suggestions? > Thanks, > Carlo > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Ubiquity - A little bit forward *Carlo Bertoldi* Ubiquity Via Teodosio 65 -20131, Milano Fisso: +39 02 288584.32 - Fax: +39 02 2829795 Email: carlo.bertoldi@REDACTED Website: www.ubiquity.it From paul.joseph.davis@REDACTED Thu Jun 30 16:37:35 2011 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Thu, 30 Jun 2011 10:37:35 -0400 Subject: [erlang-questions] Half word emulators and NIFs In-Reply-To: <4E0C87DA.1090707@erix.ericsson.se> References: <4E085563.1050106@erix.ericsson.se> <4E0C4917.6060700@erix.ericsson.se> <4E0C87DA.1090707@erix.ericsson.se> Message-ID: On Thu, Jun 30, 2011 at 10:27 AM, Sverker Eriksson wrote: > Sverker Eriksson wrote: >> >> Paul Davis wrote: >>> >>> On Mon, Jun 27, 2011 at 6:03 AM, Sverker Eriksson >>> wrote: >>> >>>> >>>> Have you tested with a debug built emulator. That may catch faulty NIF >>>> behaviors earlier. Otherwise if a NIF builds broken terms on the process >>>> heap it might not be discovered until later when those terms are matched >>>> or >>>> maybe garbage collected. >>>> >>>> There is no difference to make NIF's for the halfword emulator. Just use >>>> the >>>> NIF API as intended and don't cheat (make assumptions about >>>> sizeof(ERL_NIF_TERM) for example). >>>> >>>> /Sverker, Erlang/OTP >>>> >>>> >>> >>> I've reduced this to a failing test case in the NIF API [1]. >>> >>> In the end it boils down to these two functions (notice one is int64, >>> the other not): >>> >>> ERL_NIF_TERM >>> run_test1(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) >>> { >>> ? ?return enif_make_int64(env, 1); >>> } >>> >>> ERL_NIF_TERM >>> run_test2(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) >>> { >>> ? ?return enif_make_int(env, 1); >>> } >>> >>> Test 1 returns a Value V where (V =:= 1) is false, but (V == 1) is >>> true. The second function returns a value where both comparisons are >>> true. >>> >>> Paolo also found that amusingly this bug disappears at 2^27 as shown by: >>> >>> ERL_NIF_TERM >>> run_test3(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) >>> { >>> ? ?return enif_make_int64(env, 134217727); >>> } >>> >>> ERL_NIF_TERM >>> run_test4(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) >>> { >>> ? ?return enif_make_int64(env, 134217728); >>> } >>> >>> run_test3 returns a value that can't be pattern matched against >>> 134217727 but test 4 returns a value that can be matched against >>> 134217728. >>> >>> Also, I built R14B03 on Ubuntu 10.04 with this configure: >>> >>> ./configure --enable-smp --enable-threads --enable-m64-build >>> --without-javac --enable-halfword-emulator >>> >>> The processors are a pair of dual core Xeon's if that's important at all. >>> >>> [1] https://github.com/davisp/halfwordtest >>> >>> >> >> Very nice trouble shooting. Thank you Paul. >> >> It's a halfword-bug affecting: >> enif_make_int64 >> enif_make_long >> enif_make_uint64 >> enif_make_ulong >> >> for integer values that can fit into 28 bits. >> >> What happens is that it makes a bignum even though the value could have >> fit into a small integer. Small bignums like that are not allowed >> internally >> and that's the reason why it behaves so strangely when comparing. >> >> I'll be back with a source patch. >> >> /Sverker, Erlang/OTP >> > Here is a source patch. Only tested on halfword emulator. > > diff --git a/erts/emulator/beam/erl_nif.c b/erts/emulator/beam/erl_nif.c > index 68421b4..d9b1a8e 100644 > --- a/erts/emulator/beam/erl_nif.c > +++ b/erts/emulator/beam/erl_nif.c > @@ -833,8 +833,11 @@ ERL_NIF_TERM enif_make_uint(ErlNifEnv* env, unsigned i) > > ERL_NIF_TERM enif_make_long(ErlNifEnv* env, long i) > { > + ? ?if (IS_SSMALL(i)) { > + ? ? ? return make_small(i); > + ? ?} > #if SIZEOF_LONG == ERTS_SIZEOF_ETERM > - ? ?return IS_SSMALL(i) ? make_small(i) : small_to_big(i, > alloc_heap(env,2)); > + ? ?return small_to_big(i, alloc_heap(env,2)); > #elif SIZEOF_LONG == 8 > ? ?ensure_heap(env,3); > ? ?return erts_sint64_to_big(i, &env->hp); > @@ -843,8 +846,11 @@ ERL_NIF_TERM enif_make_long(ErlNifEnv* env, long i) > > ERL_NIF_TERM enif_make_ulong(ErlNifEnv* env, unsigned long i) > { > + ? ?if (IS_USMALL(0,i)) { > + ? ? ? return make_small(i); > + ? ?} > #if SIZEOF_LONG == ERTS_SIZEOF_ETERM > - ? ?return IS_USMALL(0,i) ? make_small(i) : > uint_to_big(i,alloc_heap(env,2)); > + ? ?return uint_to_big(i,alloc_heap(env,2)); > #elif SIZEOF_LONG == 8 > ? ?ensure_heap(env,3); > ? ?return erts_uint64_to_big(i, &env->hp); > > > > /Sverker, Erlang/OTP > > Verified that this makes my earlier tests pass. Thanks for the quick response! Paul Davis From erlangsiri@REDACTED Thu Jun 30 16:54:55 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Thu, 30 Jun 2011 16:54:55 +0200 Subject: [erlang-questions] SASL problem In-Reply-To: <4E0C896F.4010202@ubiquity.it> References: <4E0C8197.6070507@ubiquity.it> <4E0C896F.4010202@ubiquity.it> Message-ID: Ok, one more shot then - this is pure guessing of course, since I can not reproduce the problem... You could try to use single quotes around the tuple and double quotes with no backslash around the string: ... -sasl sasl_error_logger '{file, "/home/itadm/erlang/log/prototipo_mps_sasl.log"}' ... I'm not at all an expert on shell scripts and the secrets related to quotes - there are probably others on this list who are... I'm just giving some ideas since I have had similar problems myself... hope it helps /siri 2011/6/30 Carlo Bertoldi > Thanks, > but that didn't solved the problem, and the double quotes are still > missing in the log file, even with the 3 backslashes. > > > On 30/06/2011 16:26, Siri Hansen wrote: > >> Hi Carlo! >> >> Your script works for me, but it seems from the error report that the >> double quotes around the log file path have disappeared in your case. You >> could try to increase the number of backslashes - e.g. to three: >> >> ... -sasl sasl_error_logger "{file, \\\"/home/itadm/erlang/log/**prototipo_mps_sasl.log\\\"}" >> ... >> >> Regards >> /siri >> >> >> 2011/6/30 Carlo Bertoldi > carlo.bertoldi@**ubiquity.it >> >> >> >> Hello everybody, >> I'm trying to launch my application using heart. This is the >> script I'm using: >> >> #!/bin/bash >> >> EBIN_PATH=/home/itadm/**prototipo_mps/ebin >> LOG_FILE=/home/itadm/erlang/**log/sonda.log >> ERL=/usr/bin/erl >> NODE_NAME=sonda >> >> $ERL -sname $NODE_NAME -heart -pa $EBIN_PATH -boot start_sasl >> -sasl sasl_error_logger "{file, >> \"/home/itadm/erlang/log/**prototipo_mps_sasl.log\"}" -noshell -eval >> "application:start(prototipo_**mps)." 2>&1 > $LOG_FILE & >> >> >> The application starts, but in the log I've got this error, and I >> can see that the sasl keeps restarting. >> >> >> =ERROR REPORT==== 30-Jun-2011::15:36:46 === >> application_controller: syntax error before: '/': {file, >> /home/itadm/erlang/log/**prototipo_mps_sasl.log} >> {"init terminating in do_boot",{error,'Bad environment variable: >> "{file, /home/itadm/erlang/log/**prototipo_mps_sasl.log}" >> Application: sasl'}} >> >> >> =erl_crash_dump:0.1 >> Thu Jun 30 15:52:32 2011 >> Slogan: Kernel pid terminated (application_controller) >> ({application_start_failure,**kernel,{shutdown,{kernel,** >> start,[normal,[]]}}}) >> System version: Erlang R14A (erts-5.8) [source] [64-bit] [smp:4:4] >> [rq:4] [async-threads:0] [kernel-poll:false] >> >> If I launch the application using the shell, it works as expected. >> Any suggestions? >> Thanks, >> Carlo >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > >> >> http://erlang.org/mailman/**listinfo/erlang-questions >> >> >> > > -- > Ubiquity - A little bit forward > *Carlo Bertoldi* > Ubiquity > Via Teodosio 65 -20131, Milano > Fisso: +39 02 288584.32 - Fax: +39 02 2829795 > Email: carlo.bertoldi@REDACTED > > > Website: www.ubiquity.it > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlo.bertoldi@REDACTED Thu Jun 30 16:56:06 2011 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Thu, 30 Jun 2011 16:56:06 +0200 Subject: [erlang-questions] SASL problem In-Reply-To: References: <4E0C8197.6070507@ubiquity.it> <4E0C896F.4010202@ubiquity.it> Message-ID: <4E0C8E86.9060107@ubiquity.it> I've just managed to resolve the problem: I changed -sname $NODE_NAME to -sname sonda, and it's working. Though I wonder why this can cause such a problem. Thank you Siri for your support. Carlo On 30/06/2011 16:54, Siri Hansen wrote: > Ok, one more shot then - this is pure guessing of course, since I can > not reproduce the problem... You could try to use single quotes around > the tuple and double quotes with no backslash around the string: > > ... -sasl sasl_error_logger '{file, > "/home/itadm/erlang/log/prototipo_mps_sasl.log"}' ... > > I'm not at all an expert on shell scripts and the secrets related to > quotes - there are probably others on this list who are... I'm just > giving some ideas since I have had similar problems myself... hope it > helps > > /siri > > 2011/6/30 Carlo Bertoldi > > > Thanks, > but that didn't solved the problem, and the double quotes are > still missing in the log file, even with the 3 backslashes. > > > On 30/06/2011 16:26, Siri Hansen wrote: > > Hi Carlo! > > Your script works for me, but it seems from the error report > that the double quotes around the log file path have > disappeared in your case. You could try to increase the number > of backslashes - e.g. to three: > > ... -sasl sasl_error_logger "{file, > \\\"/home/itadm/erlang/log/prototipo_mps_sasl.log\\\"}" ... > > Regards > /siri > > > 2011/6/30 Carlo Bertoldi > >> > > > Hello everybody, > I'm trying to launch my application using heart. This is the > script I'm using: > > #!/bin/bash > > EBIN_PATH=/home/itadm/prototipo_mps/ebin > LOG_FILE=/home/itadm/erlang/log/sonda.log > ERL=/usr/bin/erl > NODE_NAME=sonda > > $ERL -sname $NODE_NAME -heart -pa $EBIN_PATH -boot start_sasl > -sasl sasl_error_logger "{file, > \"/home/itadm/erlang/log/prototipo_mps_sasl.log\"}" > -noshell -eval > "application:start(prototipo_mps)." 2>&1 > $LOG_FILE & > > > The application starts, but in the log I've got this error, > and I > can see that the sasl keeps restarting. > > > =ERROR REPORT==== 30-Jun-2011::15:36:46 === > application_controller: syntax error before: '/': {file, > /home/itadm/erlang/log/prototipo_mps_sasl.log} > {"init terminating in do_boot",{error,'Bad environment > variable: > "{file, /home/itadm/erlang/log/prototipo_mps_sasl.log}" > Application: sasl'}} > > > =erl_crash_dump:0.1 > Thu Jun 30 15:52:32 2011 > Slogan: Kernel pid terminated (application_controller) > > ({application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}}) > System version: Erlang R14A (erts-5.8) [source] [64-bit] > [smp:4:4] > [rq:4] [async-threads:0] [kernel-poll:false] > > If I launch the application using the shell, it works as > expected. > Any suggestions? > Thanks, > Carlo > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Ubiquity - A little bit forward > *Carlo Bertoldi* > Ubiquity > Via Teodosio 65 -20131, Milano > Fisso: +39 02 288584.32 - Fax: +39 02 2829795 > Email: carlo.bertoldi@REDACTED > > > > Website: www.ubiquity.it > > > -- Ubiquity - A little bit forward *Carlo Bertoldi* Ubiquity Via Teodosio 65 -20131, Milano Fisso: +39 02 288584.32 - Fax: +39 02 2829795 Email: carlo.bertoldi@REDACTED Website: www.ubiquity.it From phil.toland@REDACTED Thu Jun 30 17:03:44 2011 From: phil.toland@REDACTED (Phillip Toland) Date: Thu, 30 Jun 2011 10:03:44 -0500 Subject: [erlang-questions] json in erlang In-Reply-To: References: Message-ID: On Wed, Jun 29, 2011 at 10:32 PM, Jack Moffitt wrote: >> http://www.erlang.org/eeps/eep-0018.html >> >> but it doesn't exist in erlang proper (it seems). ?Does that mean the >> eep hasn't been accepted and won't be implemented? ?What does the S -> >> standards track eep mean? > > I don't know the status of the EEP, but pretty much everyone I know > has standardized on usage of mochijson2. In fact, the only project > that I've run across that uses something different is emongo, which > does it's own thing. The thing about mochijson2 is that it is sloooooow. We have found that it quickly becomes a bottleneck if you are trying to implement a high-throughput API. Sorry I don't have any hard numbers handy. If you decide to go with mochijson2 then I strongly recommend that you do some performance testing. We currently use https://github.com/benoitc/eep0018 which leverages YAJL via a NIF and is pretty speedy. It has a similar (though not identical) API to mochijson2. There are a number of forks of this project on github and we tried a few before we found one that we liked. ~p From comptekki@REDACTED Thu Jun 30 17:44:24 2011 From: comptekki@REDACTED (Wes James) Date: Thu, 30 Jun 2011 09:44:24 -0600 Subject: [erlang-questions] json in erlang In-Reply-To: References: Message-ID: On Thu, Jun 30, 2011 at 9:03 AM, Phillip Toland wrote: > We currently use https://github.com/benoitc/eep0018 which leverages > YAJL via a NIF and is pretty speedy. It has a similar (though not > identical) API to mochijson2. There are a number of forks of this > project on github and we tried a few before we found one that we > liked. > > ~p > Thanks all for responding. -wes From matti.oinas@REDACTED Thu Jun 30 20:14:33 2011 From: matti.oinas@REDACTED (Matti Oinas) Date: Thu, 30 Jun 2011 21:14:33 +0300 Subject: [erlang-questions] SASL problem In-Reply-To: <4E0C8E86.9060107@ubiquity.it> References: <4E0C8197.6070507@ubiquity.it> <4E0C896F.4010202@ubiquity.it> <4E0C8E86.9060107@ubiquity.it> Message-ID: <4E0CBD09.2020302@gmail.com> One possible cause for these problems are characters in file that arent visible but are there and they mess up the actual command. I have experienced similar problems with shell scripts and usually there is \r\n line change (someone edited the file using some windows edirot and uploaded it into server) or some UTF-8 character that isn't visible in editor. You can check if these are the problem by checking the file with hexdump or similar tool. Matti On 06/30/2011 05:56 PM, Carlo Bertoldi wrote: > I've just managed to resolve the problem: I changed -sname $NODE_NAME > to -sname sonda, and it's working. > Though I wonder why this can cause such a problem. > Thank you Siri for your support. > > Carlo > > > On 30/06/2011 16:54, Siri Hansen wrote: >> Ok, one more shot then - this is pure guessing of course, since I can >> not reproduce the problem... You could try to use single quotes >> around the tuple and double quotes with no backslash around the string: >> >> ... -sasl sasl_error_logger '{file, >> "/home/itadm/erlang/log/prototipo_mps_sasl.log"}' ... >> >> I'm not at all an expert on shell scripts and the secrets related to >> quotes - there are probably others on this list who are... I'm just >> giving some ideas since I have had similar problems myself... hope it >> helps >> >> /siri >> >> 2011/6/30 Carlo Bertoldi > > >> >> Thanks, >> but that didn't solved the problem, and the double quotes are >> still missing in the log file, even with the 3 backslashes. >> >> >> On 30/06/2011 16:26, Siri Hansen wrote: >> >> Hi Carlo! >> >> Your script works for me, but it seems from the error report >> that the double quotes around the log file path have >> disappeared in your case. You could try to increase the number >> of backslashes - e.g. to three: >> >> ... -sasl sasl_error_logger "{file, >> \\\"/home/itadm/erlang/log/prototipo_mps_sasl.log\\\"}" ... >> >> Regards >> /siri >> >> >> 2011/6/30 Carlo Bertoldi > >> > >> >> >> >> Hello everybody, >> I'm trying to launch my application using heart. This is the >> script I'm using: >> >> #!/bin/bash >> >> EBIN_PATH=/home/itadm/prototipo_mps/ebin >> LOG_FILE=/home/itadm/erlang/log/sonda.log >> ERL=/usr/bin/erl >> NODE_NAME=sonda >> >> $ERL -sname $NODE_NAME -heart -pa $EBIN_PATH -boot start_sasl >> -sasl sasl_error_logger "{file, >> \"/home/itadm/erlang/log/prototipo_mps_sasl.log\"}" >> -noshell -eval >> "application:start(prototipo_mps)." 2>&1 > $LOG_FILE & >> >> >> The application starts, but in the log I've got this error, >> and I >> can see that the sasl keeps restarting. >> >> >> =ERROR REPORT==== 30-Jun-2011::15:36:46 === >> application_controller: syntax error before: '/': {file, >> /home/itadm/erlang/log/prototipo_mps_sasl.log} >> {"init terminating in do_boot",{error,'Bad environment >> variable: >> "{file, /home/itadm/erlang/log/prototipo_mps_sasl.log}" >> Application: sasl'}} >> >> >> =erl_crash_dump:0.1 >> Thu Jun 30 15:52:32 2011 >> Slogan: Kernel pid terminated (application_controller) >> >> ({application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}}) >> System version: Erlang R14A (erts-5.8) [source] [64-bit] >> [smp:4:4] >> [rq:4] [async-threads:0] [kernel-poll:false] >> >> If I launch the application using the shell, it works as >> expected. >> Any suggestions? >> Thanks, >> Carlo >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> >> > > >> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> -- Ubiquity - A little bit forward >> *Carlo Bertoldi* >> Ubiquity >> Via Teodosio 65 -20131, Milano >> Fisso: +39 02 288584.32 - Fax: +39 02 2829795 >> Email: carlo.bertoldi@REDACTED >> >> > > >> Website: www.ubiquity.it >> >> >> > > From paul.joseph.davis@REDACTED Thu Jun 30 21:18:42 2011 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Thu, 30 Jun 2011 15:18:42 -0400 Subject: [erlang-questions] json in erlang In-Reply-To: References: Message-ID: On Wed, Jun 29, 2011 at 11:12 PM, Wes James wrote: > I see this: > > http://www.erlang.org/eeps/eep-0018.html > > but it doesn't exist in erlang proper (it seems). ?Does that mean the > eep hasn't been accepted and won't be implemented? ?What does the S -> > standards track eep mean? > > just wondering. > > thx, > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Apologies for missing this for so long. But a brief history of JSON as I've seen it over the last couple years. Originally in CouchDB we used mochijson2 and that was fine. There were also a few other pure erlang modules around this time as well, specifically I think RabbitMQ still uses something that isn't mochijson2. Though I haven't checked lately. Through various benchmarks of CouchDB we started seeing it popping up as a place in the code that was taking a significant amount of time. Around the end of 2008 me and another guy (who's name escapes me at the moment) started working on an implementation of eep0018 as a linked in port driver. Shortly-ish after we had a mostly working version of this, NIF's came along and I switched to using those for eep0018. There have been a number of people using it and I've fixed a couple bugs but its been mostly stable since that update. Recently in CouchDB land there was a big push to finally move to a NIF based JSON parser. To make this happen we had to go through and make sure that the behavior was much closer to mochijson2 than eep0018 was originally. Specifically large number support and some UTF-8 behaviors needed to be changed. Filipe Manana and Damien Katz did this update to minimize behavior changes. The fork on Beno?t's GitHub account (referenced in this thread) appears to have two line patch applied to this second version. I make a note of the second version because in order to match the mochijson2 semantics they ended making a more hybrid solution for the parser which slows it down a bit. When I was looking at this I decided that I'd spent enough time around the JSON spec to have a decent chance at writing a parser that is much quicker that can minimize the amount of effort needed in Erlang to make things as fast as possible. The new project is called Jiffy [1] and should be a completely (with caveats [2]) working parser. In my tests its also noticeably faster than the newer eep0018 and mochijson2, but only slightly faster or tied with the older eep0018 (though still matches mochijson2 better). Also I should point out that when I say "implementation of eep0018," a more apt description would be "an implementation of mochijson2 in C". For the most part I think most people have abandoned most of the points in eep0018 that aren't captured in mochijson2. For the stuff in eep0018 that isn't in one of the current parsers I'd suggest just adding them as a library on top of a JSON encoder/decoder. Also, my focus for JSON encoding/decoding has always been on making it very fast to process smallish bits of data. If you need a library that will allow you to stream parsing events for a large JSON stream I would highly recommend Alisdair Sullivan's jsx library [3]. This got a bit longer than I expected so I'm gonna cut myself off before I ramble even further. If you have any questions feel free to ask. [1] https://github.com/davisp/jiffy [2] I only support a strict implementation of UTF-8. [3] https://github.com/talentdeficit/jsx From moxford@REDACTED Thu Jun 30 22:03:09 2011 From: moxford@REDACTED (Mike Oxford) Date: Thu, 30 Jun 2011 13:03:09 -0700 Subject: [erlang-questions] json in erlang In-Reply-To: References: Message-ID: I can build it, but trying to use it in my application it complains about jiffy/ebin/jiffy.app {"init terminating in do_boot",{function_clause,[{reltool_server,parse_app_info,["/deps/jiffy/ebin/jiffy.app",[{mod,reloading,[]},{applications,[kernel]},{modules,[jiffy]}],{app_info,"JSON Decoder/Encoder.",[],"0.1.0-6-ga8d15f4",[],infinity,infinity,[],[],[],[],undefined,undefined},{ok,[]}]},{reltool_server,refresh_app,3},{reltool_server,refresh_apps,5},{reltool_server,merge_config,4},{reltool_server,refresh,3},{reltool_server,do_init,1},{reltool_server,init,1},{proc_lib,init_p_do_apply,3}]}} Here is the generated ebin/jiffy.app file ///////// {application,jiffy, [{description,"JSON Decoder/Encoder."}, {vsn,"0.1.0-6-ga8d15f4"}, {registered,[]}, {mod,reloading,[]}, {applications,[kernel]}, {modules,[jiffy]}]}. ///////////// This is happening for me with both {tag, "0.1.0"} and "master" rebar configs. Other github modules in the same rebar.config work normally. Thanks, -mox On Thu, Jun 30, 2011 at 12:18 PM, Paul Davis wrote: > On Wed, Jun 29, 2011 at 11:12 PM, Wes James wrote: >> I see this: >> >> http://www.erlang.org/eeps/eep-0018.html >> >> but it doesn't exist in erlang proper (it seems). ?Does that mean the >> eep hasn't been accepted and won't be implemented? ?What does the S -> >> standards track eep mean? >> >> just wondering. >> >> thx, >> >> -wes >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > Apologies for missing this for so long. But a brief history of JSON as > I've seen it over the last couple years. > > Originally in CouchDB we used mochijson2 and that was fine. There were > also a few other pure erlang modules around this time as well, > specifically I think RabbitMQ still uses something that isn't > mochijson2. Though I haven't checked lately. > > Through various benchmarks of CouchDB we started seeing it popping up > as a place in the code that was taking a significant amount of time. > Around the end of 2008 me and another guy (who's name escapes me at > the moment) started working on an implementation of eep0018 as a > linked in port driver. Shortly-ish after we had a mostly working > version of this, NIF's came along and I switched to using those for > eep0018. There have been a number of people using it and I've fixed a > couple bugs but its been mostly stable since that update. > > Recently in CouchDB land there was a big push to finally move to a NIF > based JSON parser. To make this happen we had to go through and make > sure that the behavior was much closer to mochijson2 than eep0018 was > originally. Specifically large number support and some UTF-8 behaviors > needed to be changed. Filipe Manana and Damien Katz did this update to > minimize behavior changes. The fork on Beno?t's GitHub account > (referenced in this thread) appears to have two line patch applied to > this second version. > > I make a note of the second version because in order to match the > mochijson2 semantics they ended making a more hybrid solution for the > parser which slows it down a bit. When I was looking at this I decided > that I'd spent enough time around the JSON spec to have a decent > chance at writing a parser that is much quicker that can minimize the > amount of effort needed in Erlang to make things as fast as possible. > The new project is called Jiffy [1] and should be a completely (with > caveats [2]) working parser. In my tests its also noticeably faster > than the newer eep0018 and mochijson2, but only slightly faster or > tied with the older eep0018 (though still matches mochijson2 better). > > Also I should point out that when I say "implementation of eep0018," a > more apt description would be "an implementation of mochijson2 in C". > For the most part I think most people have abandoned most of the > points in eep0018 that aren't captured in mochijson2. For the stuff in > eep0018 that isn't in one of the current parsers I'd suggest just > adding them as a library on top of a JSON encoder/decoder. > > Also, my focus for JSON encoding/decoding has always been on making it > very fast to process smallish bits of data. If you need a library that > will allow you to stream parsing events for a large JSON stream I > would highly recommend Alisdair Sullivan's jsx library [3]. > > This got a bit longer than I expected so I'm gonna cut myself off > before I ramble even further. If you have any questions feel free to > ask. > > [1] https://github.com/davisp/jiffy > [2] I only support a strict implementation of UTF-8. > [3] https://github.com/talentdeficit/jsx > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions >