From mfidelman@REDACTED Thu Mar 1 00:04:03 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Wed, 29 Feb 2012 18:04:03 -0500 Subject: [erlang-questions] Massive Numbers of Actors vs. Massive Numbers of Objects vs. ???? In-Reply-To: References: <4F4D08E9.4010705@meetinghouse.net> <4F4E9A47.9060407@meetinghouse.net> Message-ID: <4F4EAEE3.801@meetinghouse.net> Robert Melton wrote: > Hey Miles! Interesting topic. Hey Rob, Thanks! It's fun to bat around (and useful). > > On Tue, Feb 28, 2012 at 12:03 PM, Miles Fidelman > wrote: >> Think of something like massive numbers of stored email messages, where each >> message is addressable, can respond to events, and in some cases can >> initiate events - > I think this is a poor way to structure the problem. An "email" is a > static lifeless thing that can be acted on by processes. I would > consider an email-message something that can be serialized down to the > disk, it is data. What you need is a number of email_wrangler > processes, each one tied to a single piece of data, your "email". I guess I'm thinking that today's emails often contain HTML and JavaScript - i.e., they're no longer "lifeless." Concatenating them into mbox files no longer seems like a good to deal with them. Seems like each message is better modeled as either an object or an actor, or some hybrid. >> for example, on receiving an email message, a reader can >> fill in a form, and have that update every copy of the message spread across >> dozens (or hundreds, or thousands) of mailboxes/folders distributed across >> the Internet. > Please explain the use case, I can't think of any use case that would > demand this structure. If the idea is to have a public-state (shared > among everyone who got the email) and a private state (unique to that > user) -- I think it would make way more sense to separate the storage > of these two concerns and apply updates respectively (parent-child > style). If the idea is that all state is shared, why copy? If the > idea is that all state is private, why message? Simple case: I distribute a document (say an article). I then want to distribute an update that gets auto-applied, rather than leaving it to the recipient to "replace paragraph x with ". Particularly useful when dealing with formal documents - e.g. project plans - that have a formal update process associated with them. Email copies of documents that contain their own update rules (who can send updates, how they get applied, conflict resolution rules). Then email updates that get passed to the original document, which updates itself. (Not that different than emailing software patches.) >> Or where an email message, stored in some folder, can wake up >> and send a reminder. > The email_wrangler could do this, or a email_scheduler could start the > email_wrangler when it is time to do that work... which is how I would > probably structure it, not reason to leave email_wranglers running. The wrangler concept seems conceptually clumsy - compared to, say, objects and triggers stored in an object database. What's the actor-oriented equivalent? (Actually, there is a middle ground - document databases - and CouchDB comes to mind.) > I am currently building a system with millions of concurrent processes > across a cluster of Erlang nodes. But, these processes aren't left up > just rotting, they are spawned in reaction to events, and live for a > responsible and useful time period in which they do work. I can not > fathom why you would DESIGN a system to have huge numbers of > persistent inactive actors. > > >> In some sense, I'm describing an "actor-oriented database" - a place to park >> large numbers of persistent actors, surrounded by mechanisms to deliver >> messages, and allow them to wake up after timeouts. >> >> I'm kind of surprised somebody hasn't built such a beast - at least as a >> research experiment. > I would never see a use for such a beast when everything it > accomplishes can be done easier using existing idioms with less memory > and CPU usage... unless I entirely missed something. See, I'm coming to the opposite conclusion. Folks in the object-oriented world came up with object-oriented databases as a way to persist huge numbers of objects. But my problem with object-oriented models is that they really don't deal with flow-of-control (particularly when dealing with massively concurrent problems). And lots of things are better modeled as long-lived actors, rather than objects. I come at this after spending some years in the simulation world. I arrived at a company that made, among other things, "computer generate forces" simulators - doing things like simulating 10,000 people, vehicles, and weapons moving around a battlespace (think massively multi-player game). My first guess was that each entity was modeled as a light-weight thread, but all of our c++ programmers assured me that, no, you can't do that, context switching overhead kills you. Instead everything was modeled as an object, and four main threads would touch every object once per simulation cycle (40 times a second or so). Conceptually ugly, and lots of spaghetti code. I discovered Erlang when looking for other ways to do things. Seems to me that lots of things (e.g., tanks) are better modeled as actors than as objects - particularly if you have an environment like Erlang that supports massive concurrency. But... that leads to the question of what to do with those entities when they're not doing anything - where do you "park" a tank? If I'm parking objects, an object database is the obvious answer. If I'm parking an actor, the answer is less obvious - I can have it hibernate, but that doesn't persist across crashes, reboots, etc. - and I eventually run out of PIDs. Which leads me to the thought that an actor-oriented database would be very useful for large simulations, gaming platforms, .. and the kind of "active emails" that I'm thinking about. CouchDB might be a good platform for things that are document-like, but I'm getting intrigued by the more general case (a "parking lot" for actors, if you will). Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From jakob@REDACTED Thu Mar 1 00:04:58 2012 From: jakob@REDACTED (Jakob Praher) Date: Thu, 01 Mar 2012 00:04:58 +0100 Subject: [erlang-questions] object vs. actor (Re: Massive Numbers of Actors vs. Massive Numbers of Objects vs. ????) In-Reply-To: References: Message-ID: <4F4EAF1A.3090500@praher.info> Am 29.02.12 23:26, schrieb Raoul Duke: > ...ah so one way of looking at it is: when sending a message to an > actor, the actor is going to get thread on which to process the > message, whereas an object either needs you to call into it so it can > get the thread, or you have to do some other more manual work to give > it a thread. Yes that is how most object (oriented) systems are implemented today. But it is more an implementation detail: e.g Simula the grand mother of object orientation was especially used for simulation an objects could also be active entities by using coroutines (were control can be interrupted and later continued thus enabling multi tasking). Jakob > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mfidelman@REDACTED Thu Mar 1 00:11:55 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Wed, 29 Feb 2012 18:11:55 -0500 Subject: [erlang-questions] object vs. actor (Re: Massive Numbers of Actors vs. Massive Numbers of Objects vs. ????) In-Reply-To: References: Message-ID: <4F4EB0BB.8000201@meetinghouse.net> Raoul Duke wrote: > hi, > > question: what is the difference between an actor and an object? i > have seen so many different definitions of the things! the words seem > to be heavily context-dependent. > > My understanding (and how I differentiate): - an object encapsulates state and methods, but is not "active" - depending on the environment, methods can be invoked by procedure call or message passing (it's a chunk of data bound to methods for manipulating that data) - an actor also encapsulates state and methods, but is "active" - it's a running process that's listening for and responding to messages (it's a process) There are, of course, more formal definitions. Hewitt is sort of the definitive source for defining the actor formalism. Not sure who's his counterpart for objects. What it always comes down to, for me, is flow of control. Objects just sit there, until some thread winds through them, or until triggered by an event. Actor based systems consist of massive numbers of concurrently executing processes. Very different ways of thinking about things at a systemic level. Of course these are only my views - your mileage may vary. Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From eppa.ramu@REDACTED Thu Mar 1 06:04:12 2012 From: eppa.ramu@REDACTED (raamu eppa) Date: Thu, 1 Mar 2012 10:34:12 +0530 Subject: [erlang-questions] Erlang Run Time Target System. Message-ID: Hi, How to create erlang first target system.Please explain with example. Thanks, Ramu -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Thu Mar 1 07:33:11 2012 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 29 Feb 2012 22:33:11 -0800 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: References: <4F4A9BDE.1080408@gmail.com> Message-ID: <4F4F1827.70800@gmail.com> On 02/29/2012 01:09 PM, Per Andersson wrote: > >> Please also remember that the function to retrieve the MAC address from within Erlang is undocumented, > The function to retrieve MAC address from Erlang is not > undocumented anymore. > > [0] http://www.erlang.org/doc/man/inet.html#getifaddrs-0 > Thanks for mentioning that. >> and "okeuday/uuid" v1 does not depend on an interface being available (i.e., for the MAC address) while >> "avtobiff/erlang-uuid" v1 will attempt to use random:uniform/1 to create more than 44bits of randomness if no interface is available. >> >> 2) There is an important problem within "avtobiff/erlang-uuid" for v4, since random:uniform/1 only provides 44 bits of randomness, if you are lucky (that is probably pushing it, since the algorithm may have been meant to only provide 32 bits), however, the v4 function is attempting to use random:uniform/1 to provide 48 bits and 62 bits, where the rest should become 0 because of the algorithm used by random:uniform/1. The other problem with v4 is that it reseeds random:uniform/1 each time the function is called with erlang:now(), which creates additional latency and possibly makes the v4 uuid less-random (depending on how it is called, e.g., on a timer). To see a better implementation for v4, using random:uniform/1, there is get_v4_urandom_bigint/0 and get_v4_urandom_native/0 in "okeuday/uuid", however, both solutions become slower in testing when compared with get_v4/0 which uses crypto:rand_bytes/1 instead of multiple calls to random:uniform/1. > Thanks for the comments on this. Erlang UUID no longer uses > random:uniform/1 to create anymore than 32 bits of random > data. > > However, how do you deduce that random:uniform/1 only provides > 44 bits of random data? Since a float is returned from random:random/0 > which is used by random:uniform/1, isn't the total random bits 16 or 24 > (4 and 3 words on 32-bit and 64-bit architectures respectively)? > > [1] http://www.erlang.org/doc/efficiency_guide/advanced.html#id68291 > I have not tried to track down the paper connected with the algorithm implemented within the random module (B.A. Wichmann and I.D.Hill, in 'An efficient and portable pseudo-random number generator', Journal of Applied Statistics. AS183. 1982, or Byte March 1987), so I am not sure if they were limited to 32 bit integers, but I assume they were. When I looked at this awhile ago, I forgot to add 1 to the result, so it is actually 45 bits, not 44 bits. Before looking at the algorithm, I knew empirically that the random module had some limitation that causes a lack of variation in the output I saw, and the way I consider it to be 45 bits is through looking at the input to the floating point number that is used (part of the random:uniform/0 code): B1 = (A1*171) rem 30269, B2 = (A2*172) rem 30307, B3 = (A3*170) rem 30323, put(random_seed, {B1,B2,B3}), R = A1/30269 + A2/30307 + A3/30323, R - trunc(R). So, {B1, B2, B3} becomes the next seed value {A1, A2, A3}. Simplifying this, gives: R = (918999161 * A1 + 917846887 * A2 + 917362583 * A3) / 27817185604309 Whatever the values for A1, A2, and A3, (918999161 * A1 + 917846887 * A2 + 917362583 * A3) can not exceed 27817185604309 (30269 * 30307 * 30323) because of the previous modulus. So, that means the algorithm breaks down, being unable to uniformly sample numbers beyond an N of 27817185604309, when considering the N provided to random:uniform/1. When determining how many bits it takes to represent 27817185604309, you get: 1> (math:log(27817185604309) / math:log(2)) + 1. 45.6610416965467 If you are concerned about the floating point part: 2> 1 / 27817185604309. 3.59489998098548e-14 As shown here http://en.wikipedia.org/wiki/Double-precision_floating-point_format , 15.955 decimal digits can be represented within a double precision floating point number. This is enough to contain the fraction, since Erlang is using double precision for the calculation, so it should not reduce the maximum number of bits that can be provided for uniformly random numbers. >> 3) The other important difference is the license. "avtobiff/erlang-uuid" is under a GPLv3 license, while "okeuday/uuid" is under a BSD license. So, there is nothing blocking the commercial use of "okeuday/uuid" (please remember I am not a lawyer, nor am I giving legal advice). > It is of course entirely possible to develop commercial applications with > GPL'd software. What you can't do is to transform free software into > proprietary software. > This is a little misleading. You can never sell anything that contains GPL software, so many people consider GPL software to poison a code-base, since it can limit a full stack from becoming a product (this is the reason why some BSD distributions want a compiler that is not gcc, etc.). So, the most you can do with GPL software is sell the service that is written with it, assuming you always submit your proprietary code changes to the developer of the GPL project (disclaimer: please remember I am not a lawyer, nor am I giving legal advice). -- Michael From avtobiff@REDACTED Thu Mar 1 10:58:21 2012 From: avtobiff@REDACTED (Per Andersson) Date: Thu, 1 Mar 2012 10:58:21 +0100 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: <4F4F1827.70800@gmail.com> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> Message-ID: On Thu, Mar 1, 2012 at 7:33 AM, Michael Truog wrote: > On 02/29/2012 01:09 PM, Per Andersson wrote: >> >>> Please also remember that the function to retrieve the MAC address from within Erlang is undocumented, >> The function to retrieve MAC address from Erlang is not >> undocumented anymore. >> >> ? ? [0] http://www.erlang.org/doc/man/inet.html#getifaddrs-0 >> > > Thanks for mentioning that. No problem. I think it was documented in R15. >>> and "okeuday/uuid" v1 does not depend on an interface being available (i.e., for the MAC address) while >>> "avtobiff/erlang-uuid" v1 will attempt to use random:uniform/1 to create more than 44bits of randomness if no interface is available. >>> >>> 2) There is an important problem within "avtobiff/erlang-uuid" for v4, since random:uniform/1 only provides 44 bits of randomness, if you are lucky (that is probably pushing it, since the algorithm may have been meant to only provide 32 bits), however, the v4 function is attempting to use random:uniform/1 to provide 48 bits and 62 bits, where the rest should become 0 because of the algorithm used by random:uniform/1. ?The other problem with v4 is that it reseeds random:uniform/1 each time the function is called with erlang:now(), which creates additional latency and possibly makes the v4 uuid less-random (depending on how it is called, e.g., on a timer). ?To see a better implementation for v4, using random:uniform/1, there is get_v4_urandom_bigint/0 and get_v4_urandom_native/0 in "okeuday/uuid", however, both solutions become slower in testing when compared with get_v4/0 which uses crypto:rand_bytes/1 instead of multiple calls to random:uniform/1. >> Thanks for the comments on this. Erlang UUID no longer uses >> random:uniform/1 to create anymore than 32 bits of random >> data. >> >> However, how do you deduce that random:uniform/1 only provides >> 44 bits of random data? Since a float is returned from random:random/0 >> which is used by random:uniform/1, isn't the total random bits 16 or 24 >> (4 and 3 words on 32-bit and 64-bit architectures respectively)? >> >> ? ? [1] http://www.erlang.org/doc/efficiency_guide/advanced.html#id68291 >> > I have not tried to track down the paper connected with the algorithm implemented within the random module (B.A. Wichmann and I.D.Hill, in 'An efficient and portable pseudo-random number generator', Journal of Applied Statistics. AS183. 1982, or Byte March 1987), so I am not sure if they were limited to 32 bit integers, but I assume they were. ?When I looked at this awhile ago, I forgot to add 1 to the result, so it is actually 45 bits, not 44 bits. ?Before looking at the algorithm, I knew empirically that the random module had some limitation that causes a lack of variation in the output I saw, and the way I consider it to be 45 bits is through looking at the input to the floating point number that is used (part of the random:uniform/0 code): > ? ?B1 = (A1*171) rem 30269, > ? ?B2 = (A2*172) rem 30307, > ? ?B3 = (A3*170) rem 30323, > ? ?put(random_seed, {B1,B2,B3}), > ? ?R = A1/30269 + A2/30307 + A3/30323, > ? ?R - trunc(R). > So, {B1, B2, B3} becomes the next seed value {A1, A2, A3}. ?Simplifying this, gives: > ? ?R = (918999161 * A1 + 917846887 * A2 + 917362583 * A3) / 27817185604309 > Whatever the values for A1, A2, and A3, (918999161 * A1 + 917846887 * A2 + 917362583 * A3) can not exceed 27817185604309 (30269 * 30307 * 30323) because of the previous modulus. ?So, that means the algorithm breaks down, being unable to uniformly sample numbers beyond an N of 27817185604309, when considering the N provided to random:uniform/1. ?When determining how many bits it takes to represent 27817185604309, you get: > 1> (math:log(27817185604309) / math:log(2)) + 1. > 45.6610416965467 > > If you are concerned about the floating point part: > 2> 1 / 27817185604309. > 3.59489998098548e-14 > > As shown here http://en.wikipedia.org/wiki/Double-precision_floating-point_format , 15.955 decimal digits can be represented within a double precision floating point number. ?This is enough to contain the fraction, since Erlang is using double precision for the calculation, so it should not reduce the maximum number of bits that can be provided for uniformly random numbers. Thanks for the run down on random bits! I actually tracked down the paper and skimmed it. AFAIK It is possible (and even recommended IIRC) to combine several generators for exceeding the 32-bit limit. Since they when the paper was written used 16-bit arches also, but wanted a portable PRNG, combined generators mean same on both 16-bit and 32-bit arches. >>> 3) The other important difference is the license. ?"avtobiff/erlang-uuid" is under a GPLv3 license, while "okeuday/uuid" is under a BSD license. ?So, there is nothing blocking the commercial use of "okeuday/uuid" (please remember I am not a lawyer, nor am I giving legal advice). >> It is of course entirely possible to develop commercial applications with >> GPL'd software. What you can't do is to transform free software into >> proprietary software. >> > This is a little misleading. ?You can never sell anything that contains GPL software, so many people consider GPL software to poison a code-base, since it can limit a full stack from becoming a product (this is the reason why some BSD distributions want a compiler that is not gcc, etc.). ?So, the most you can do with GPL software is sell the service that is written with it, assuming you always submit your proprietary code changes to the developer of the GPL project (disclaimer: please remember I am not a lawyer, nor am I giving legal advice). Sorry but you are wrong. You can of course sell GPL'd software! Remember RedHat sells RHEL of which great parts are GPL'd. What you can't do is to distribute only binaries, you have to supply (buildable!) source code upon demand. See the Free Software Foundations FAQ on the subject. [0] http://www.gnu.org/licenses/gpl-faq.html#DoesTheGPLAllowMoney So as long as you don't sell only binaries but also include the source code (upon demand) you are complying with the GNU GPL. If you provide a service (say on the world wide web) you don't have to provide source code at all, so it is fully compliant with GPL to use GPL'd software for your proprietary web service. That is what the Affero GPL is for, to ensure that services also distribute source code. E. g. of AGPL users are gitorious.org and jabber.se. -- Per > -- Michael > From aronisstav@REDACTED Thu Mar 1 11:07:51 2012 From: aronisstav@REDACTED (Stavros Aronis) Date: Thu, 1 Mar 2012 02:07:51 -0800 (PST) Subject: [erlang-questions] dialyzer and supervisor init/1 callback spec In-Reply-To: <4F4E928B.90907@gmail.com> References: <4F4E3CB3.3000304@gmail.com> <4F4E46AA.2020300@cs.ntua.gr> <4F4E48F7.1050100@gmail.com> <4F4E522D.1070904@cs.ntua.gr> <4F4E928B.90907@gmail.com> Message-ID: <21743433.920.1330596471612.JavaMail.geo-discussion-forums@vbux23> We have confirmed this bug and I will update my recent dialyzer-fixes patch with a fix for it. Thanks for the report! Stavros > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jimenezrick@REDACTED Thu Mar 1 11:42:20 2012 From: jimenezrick@REDACTED (Ricardo Catalinas =?iso-8859-1?Q?Jim=E9nez?=) Date: Thu, 1 Mar 2012 11:42:20 +0100 Subject: [erlang-questions] Erlang Run Time Target System. In-Reply-To: References: Message-ID: <20120301104220.GC847@viper.local> On Thu, Mar 01, 2012 at 10:34:12AM +0530, raamu eppa wrote: > How to create erlang first target system.Please explain with example. The Rebar tool makes the task far more easier: http://www.metabrew.com/article/erlangotp-releases-rebar-release_handler-appup-etc http://www.metabrew.com/article/erlang-rebar-tutorial-generating-releases-upgrades Regards -- Ricardo (http://r.untroubled.be/) From erlangsiri@REDACTED Thu Mar 1 12:00:58 2012 From: erlangsiri@REDACTED (Siri Hansen) Date: Thu, 1 Mar 2012 12:00:58 +0100 Subject: [erlang-questions] Erlang Run Time Target System. In-Reply-To: References: Message-ID: Under System Documentation - > System Principles there is a chapter named "Creating a First Target System" http://www.erlang.org/doc/system_principles/create_target.html /siri Den 06:04 1. mars 2012 skrev raamu eppa f?lgende: > Hi, > > How to create erlang first target system.Please explain with example. > > Thanks, > Ramu > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Thu Mar 1 14:22:11 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 1 Mar 2012 08:22:11 -0500 Subject: [erlang-questions] Common Test & skip_cases Message-ID: Hi there mailing list, I have a very simple question. I was revisiting Common Test for LYSE's next chapter, and I remembered that Common Test has the following skip options in its test specifications: {skip_suites, [NodeRefs,] DirRef, Suites, Comment} {skip_cases, [NodeRefs,] DirRef, Suite, Cases, Comment} These two options have their equivalent in tuples keyed by 'suites' and 'cases'. However, there is no documented option for 'skip_groups' even though there is a 'groups' tuple. Is there any special reason for that? Regards, Fred. -------------- next part -------------- An HTML attachment was scrubbed... URL: From juk80x@REDACTED Thu Mar 1 15:06:24 2012 From: juk80x@REDACTED (Jason Rogers) Date: Thu, 1 Mar 2012 09:06:24 -0500 Subject: [erlang-questions] Types and Specs support in erl_syntax or erl_prettypr Message-ID: I've noticed a lack of support for types and specs in erl_syntax or erl_prettypr. If you try: {ok,{_,[{abstract_code,{_,AC}}]}} = beam_lib:chunks(Beam,[abstract_code]). io:fwrite("~s~n", [erl_prettypr:format(erl_syntax:form_list(AC))]). on a beam file with type or spec declarations, the resulting code will not compile. Any idea if official support is forthcoming? I tried to modify both libraries to provide some minimal support for my own use, but ran into an issue with typed records. A typed record shows up in the syntax tree twice: as a un-typed record and as a typed record. And the typed version, which includes all the information contained in the un-typed declaration, appears after the un-typed declaration. So if you write code to process the typed version, you will end up with source code that has both a typed and an un-typed declaration. Is there anyway to get to a single record declaration? Thanks, Jason From watson.timothy@REDACTED Thu Mar 1 16:43:33 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 1 Mar 2012 15:43:33 +0000 Subject: [erlang-questions] Type Server Message-ID: <2F85B636-4713-4B77-927D-8F669BE0B0F5@gmail.com> Hi all, Both PropEr and Dialyzer appear to have a 'type server' that provides functions for working with type specs. What are the chances of this being available as a separate unit? There is currently very little support for working with types, as the previous post about lack of support in erl_syntax pointed out. It would be nice to see type specs being better supported. Cheers, Tim From watson.timothy@REDACTED Thu Mar 1 16:46:01 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 1 Mar 2012 15:46:01 +0000 Subject: [erlang-questions] Type Specs Preserved in Beam Code Message-ID: <395A7BEA-5244-4735-818A-AFA781A4A081@gmail.com> Current type specs are only available in beam code if you compile it with debug_info or use the 'abstract_code' parse transform that's floating around on the internet. What are the chances of getting the type spec information preserved in the beam (in a separate chunk) without having to do this? Is this a change to the parser, the compiler, or to the modules dealing with Core - good question there: are type specs supported in Core? - or all of these? If this is low priority I'll happily take it away and try to write a patch, but I'd need a hint about where to start. Cheers, Tim From mfidelman@REDACTED Thu Mar 1 16:53:01 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Thu, 01 Mar 2012 10:53:01 -0500 Subject: [erlang-questions] Massive Numbers of Actors vs. Massive Numbers of Objects vs. ???? In-Reply-To: References: <4F4D08E9.4010705@meetinghouse.net> <4F4E9A47.9060407@meetinghouse.net> <4F4EAEE3.801@meetinghouse.net> Message-ID: <4F4F9B5D.8070706@meetinghouse.net> Robert Melton wrote: > On Wed, Feb 29, 2012 at 6:04 PM, Miles Fidelman > wrote: >> I guess I'm thinking that today's emails often contain HTML and JavaScript - >> i.e., they're no longer "lifeless." Concatenating them into mbox files no >> longer seems like a good to deal with them. Seems like each message is >> better modeled as either an object or an actor, or some hybrid. > Wait, is the plan to actually execute the javascript in the context of > Erlang (like Spidermonkey in Riak?)... if so, then I guess you could > treat them as active things -- else, I need to go with "lifeless" -- > they are opaque text data that has no active component, regardless if > javascript, python, perl, php or X in that text document. Short answer is yes. Longer answer: I'm thinking through the details of a mail database to replace mbox files or mh-style directories - where, for example, a message (say a meeting agenda) - buried away in the database, can wake up and remind me that my slides are overdue for distribution, or that the meeting is in two hours. Yes, there are lots of ways to do this, but one that strikes me as conceptually nice is simply to drop that message into with the following characteristics: - allows for storing/browsing/retrieving messages through a classic directory/folder style hierarchy through a mail client - allows stored messages to wake up and do things - allows "follow-up" messages to be passed directly to stored items One obvious model is to mediate everything through a database engine of some sort. Another is to treat each message as a first-class, addressable, process that's "parked." I find this model somewhat interesting - so I figure it's worth playing out. The conceptual model I keep coming back to is a bookshelf, where each book is active - able to flash a light to tell me to go look at it, able to receive update messages, able to propagate notes that I make in the margins, ..... The question becomes, what does the underlying infrastructure look like. >> Simple case: I distribute a document (say an article). I then want to >> distribute an update that gets auto-applied, rather than leaving it to the >> recipient to "replace paragraph x with". > Owner control would be much easier with a reference to a root document > than to distribute updates to thousands of copies. I'm a big fan of the LOCKSS model (lots of copies keeps stuff safe). Particularly important in situations where connectivity is not guaranteed, or where where individual copies can get destroyed (say military operations). >> Particularly useful when dealing with formal documents - e.g. project plans >> - that have a formal update process associated with them. Email copies of >> documents that contain their own update rules (who can send updates, how >> they get applied, conflict resolution rules). Then email updates that get >> passed to the original document, which updates itself. (Not that different >> than emailing software patches.) > My team is currently working on a similar problem (but part of the > core domain-space, not accidental complexity created by us)... if you > ever design a rules system that can do this well, release it and I > will buy you a beer the next time you are in DC. Seriously, > distributed conflict resolution and surfacing is hard stuff (tm). > Obviously, all this complexity would because you want the clients to > support local customizations of the document, and merge / control the > parts they want. Again, if you can create a decent meta-language for > this type of stuff, I am seriously interested. We should talk offline about this - how always looking for collaborators. >> The wrangler concept seems conceptually clumsy - compared to, say, objects >> and triggers stored in an object database. What's the actor-oriented >> equivalent? (Actually, there is a middle ground - document databases - and >> CouchDB comes to mind.) > I don't see why it is clumsy, it is the standard process / data > pairing that we know and love. In actual use, I think it is very > elegant, because you only ever deal with is the wranglers, all the > magic of storing the documents behind them is hidden by them -- you > get a simple understandable "living" interface. Even the complexity > of scheduling events happens behind the wranglers ... the wranglers > are your API touch point. Behind a system of wranglers, I would > probably use a document store of some sort (insert your favorite > flavor of K/V store). I'm a protocol guy by background. Put the intelligence at the endpoints, let them talk to each other. Minimize the role of everything in the middle. (If I were designing Facebook, I'd start with NNTP, not a central server. Neglecting business reasons for a centralized model, of course.). I'm trying to keep my focus on the objects involved, and their behaviors - and keep coming back to actors as the fundamental construct, as opposed to classic objects (though classic might not be the right word, I believe the actor formalism was coined before that of object orientation). >> See, I'm coming to the opposite conclusion. Folks in the object-oriented >> world came up with object-oriented databases as a way to persist huge >> numbers of objects. But my problem with object-oriented models is that they >> really don't deal with flow-of-control (particularly when dealing with >> massively concurrent problems). And lots of things are better modeled as >> long-lived actors, rather than objects. > You can persist massive numbers of "things" in any standard database > (see: Mysql / Facebook). OO DBs were not created to solve a "scale" > problem with storing objects, they were created to solve the ORIM > (object-relational impedance mismatch). They created as many problems > as they solved in most cases, and every team I know of that used one, > ended up regretting it in due time. I think that is part of the > reason you hear more about document& k/v stores than object > persistence frameworks. I consider OO DBs to be one of the incredible > failures of the OO community... even worse than the nasty ORMs they > were created to replace, at least the ORMs didn't have horrific > lock-in, just bitter painful annoying lock-in. Well yeah, there is that. But the question of long-term persistence remains, and I can see a need for long-term persistence of large numbers of both object-like "things" and actor-like "things" (as well as key-value pairs and other tupple-like "things"). >> I come at this after spending some years in the simulation world. I arrived >> at a company that made, among other things, "computer generate forces" >> simulators - doing things like simulating 10,000 people, vehicles, and >> weapons moving around a battlespace... >> >> Seems to me that lots of things (e.g., tanks) are better modeled as actors >> than as objects - particularly if you have an environment like Erlang that >> supports massive concurrency. But... that leads to the question of what to >> do with those entities when they're not doing anything - where do you "park" >> a tank? > These simulations run for massive amounts of time? It seems that in > most simulations you are looking for unexpected emergent behaviors -- > don't the actors have to be alive and responding the events happening > in there reactive space to do this? You know what, there are enough > implementation details in how to handle this that I am going to skip > ahead... The ones I've been involved with are more for training purposes - networked wargames that contain a mix of real people/equipment and simulated ones. They can run for several weeks, and very often you want to re-run a scenario under different conditions. And then there are MMORPGs - where virtual worlds persist for very long periods of time. >> If I'm parking objects, an object database is the obvious answer. > ... Seems obvious, but often isn't. OO DBs have lots of issues (as > listed before) and even with OQL still end up being more trouble than > they are worth. Lots of people still store billions of objects to > (favorite RDBMS here) via ORMs -- as the tooling support, 3rd party > integration, etc, etc, etc is better. There is a reason in the last > 20 years OO DBs didn't take over the world. They are still niche. > > >> If I'm parking an actor, the answer is less obvious - I can have it >> hibernate, but that doesn't persist across crashes, reboots, etc. - and I >> eventually run out of PIDs. > Seems fairly obvious to me, have the actor persist its state somewhere > (K/V store, document store, even RDBMS or heck, an OODB if you really > want to), and have it load it back when it is looked up... end up with > a 1:1 actor (process) to data system. Seems like an unnecessary step that adds confusion to one's conceptual model. For example: Conceptually, actors seem like the right framework modeling a tank (with either a human or software in the driver's seat) - and there are times when you want to "park tank no. 5 in the motor pool." I really don't want to think about things like "change the tank from an actor to a database record when it's not doing anything - I just want it to sit there, "parked" if you will. Erlang and OTP go way beyond anything out there when it comes to managing huge numbers of actors, when they're doing something. And hibernation is a reasonable way to view an actor that's inactive for a long period of time - but current mechanisms don't seem to scale all that well. I'm raising the question of what to do if you want to persist inactive actors over extended periods of time. >> Which leads me to the thought that an actor-oriented database would be very >> useful for large simulations, gaming platforms, .. and the kind of "active >> emails" that I'm thinking about. CouchDB might be a good platform for >> things that are document-like, but I'm getting intrigued by the more general >> case (a "parking lot" for actors, if you will). > Once again, seems fairly "solved"... almost every Erlang system I have > built had this setup in some form or another ... process persists > itself via some mechanism. It sounds like you want like a > super-hibernate that moves state to some persistence layer and keeps > the process running in an ultra minimal state... should probably be > fairly easy to build what you want, you should give it a go and report > back. Exactly the conclusion I've come to. Before building it, I figure it's worth seeing what other people think about the need, approach, and whether anybody else has built (pieces of) such a beast already. "Super-hibernate" is a great way of putting it. Not so sure it would be all that easy to build without digging into the VM. The other choice is to decouple PID from the identity of super-hibernating processes - which would probably require recreating a whole bunch of functionality that Erlang already provides for handling processes and messaging. (Now if someone has already done some of that.....) Cheers, Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From g@REDACTED Thu Mar 1 17:16:48 2012 From: g@REDACTED (Garrett Smith) Date: Thu, 1 Mar 2012 10:16:48 -0600 Subject: [erlang-questions] object vs. actor (Re: Massive Numbers of Actors vs. Massive Numbers of Objects vs. ????) In-Reply-To: <4F4EB0BB.8000201@meetinghouse.net> References: <4F4EB0BB.8000201@meetinghouse.net> Message-ID: On Wed, Feb 29, 2012 at 5:11 PM, Miles Fidelman wrote: > Raoul Duke wrote: >> >> hi, >> >> question: what is the difference between an actor and an object? i >> have seen so many different definitions of the things! the words seem >> to be heavily context-dependent. >> >> > My understanding (and how I differentiate): > > - an object encapsulates state and methods, but is not "active" - depending > on the environment, methods can be invoked by procedure call or message > passing (it's a chunk of data bound to methods for manipulating that data) > > - an actor also encapsulates state and methods, but is "active" - it's a > running process that's listening for and responding to messages (it's a > process) > > There are, of course, more formal definitions. ?Hewitt is sort of the > definitive source for defining the actor formalism. ?Not sure who's his > counterpart for objects. > > What it always comes down to, for me, is flow of control. ?Objects just sit > there, until some thread winds through them, or until triggered by an event. > ?Actor based systems consist of massive numbers of concurrently executing > processes. ?Very different ways of thinking about things at a systemic > level. > > Of course these are only my views - your mileage may vary. Just my two cents, but I'd take the energy spent here trying to define "object" and "actor" and start thinking about how you're going to build the system: - How do you store the data? (MySQL, MongoDB, Riak, Mnesia, multiple stores -- the list of options is *huge* and ) - What are you going to write your "business logic" in (Java, Python, Erlang, all of the above -- I assume your questions here are related to how suitable Erlang is for the job -- I'd say pretty good based on what you're looking for) - How are you going to communicate between nodes / servers / programs? Looking for a conceptual model is probably not a total waste of time, but at some point, you'll have to evaluate your options on those three points based on technical merit. As far as the "actor" model and massive concurrency, as as been said, Erlang will give you options that other language environments won't. You can even call them "objects" and the VM will work just the same :) Garrett From comptekki@REDACTED Thu Mar 1 17:23:05 2012 From: comptekki@REDACTED (Wes James) Date: Thu, 1 Mar 2012 09:23:05 -0700 Subject: [erlang-questions] trapexit.org error Message-ID: I just went to trapexit and got: ------- phpBB : Critical Error Could not connect to the database ------- Just an FYI - but maybe someone already knows. -wes From mfidelman@REDACTED Thu Mar 1 17:53:33 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Thu, 01 Mar 2012 11:53:33 -0500 Subject: [erlang-questions] object vs. actor (Re: Massive Numbers of Actors vs. Massive Numbers of Objects vs. ????) In-Reply-To: References: <4F4EB0BB.8000201@meetinghouse.net> Message-ID: <4F4FA98D.20503@meetinghouse.net> Garrett Smith wrote: > Just my two cents, but I'd take the energy spent here trying to define > "object" and "actor" and start thinking about how you're going to > build the system: well duh.. :-) questions/discussion here are about trying to bounce concepts off people, and try to uncover technologies that are not all that visible > > - How do you store the data? (MySQL, MongoDB, Riak, Mnesia, multiple > stores -- the list of options is *huge* and ) would sort of be nice to store data as simply a large cloud of Erlang processes - hence inquiries and discussion here but, at least in the short term, most likely is CouchDB for early versions (has most of the characteristics I'm looking for), though some possibility of using eXist (XML document store, with APP/Atom interfaces already available) probably leverage an erlang web framework (Yaws, Mochiweb, Nitrogen) > > - What are you going to write your "business logic" in (Java, Python, > Erlang, all of the above -- I assume your questions here are related > to how suitable Erlang is for the job -- I'd say pretty good based on > what you're looking for) embedded in documents, i.e., Javascript embedded in HTML/XML; possibly embedded erlang as well (I'm pretty sure Couch supports embedded erlang scripts) > - How are you going to communicate between nodes / servers / programs? protocol based - XMPP or Atom/APP, with support for message based protocols as well (SMTP, NNTP, possibly IMAP for access) > > Looking for a conceptual model is probably not a total waste of time, > but at some point, you'll have to evaluate your options on those three > points based on technical merit. I've generally found it useful as a way to organize thinking. > As far as the "actor" model and massive concurrency, as as been said, > Erlang will give you options that other language environments won't. Absolulutely, but it doesn't quite go as far as I'd like. Erlang is great for massive concurrency, not so great if you want huge numbers of processes to persist, perhaps permanently. Our application will be distributing HTML+Javascript documents by email - message like, but also object-like and actor like when live in a client or browser. The question is how to maintain the actor-like behavior when filing away large numbers of emails. It would be interesting to simply store each message as an erlang thread, accessed through message passing. But that doesn't scale with current technology. The short term answer is clearly to use a database of some sort, but a longer solution would be to find ways to support large numbers of hibernating processes. > You can even call them "objects" and the VM will work just the same :) > I could call them that, but erlang does NOT support object-oriented programming in the sense of inheritance and so forth. Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From mjtruog@REDACTED Thu Mar 1 17:53:56 2012 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 01 Mar 2012 08:53:56 -0800 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> Message-ID: <4F4FA9A4.9050909@gmail.com> On 03/01/2012 01:58 AM, Per Andersson wrote: > On Thu, Mar 1, 2012 at 7:33 AM, Michael Truog wrote: >> On 02/29/2012 01:09 PM, Per Andersson wrote: >>>> and "okeuday/uuid" v1 does not depend on an interface being available (i.e., for the MAC address) while >>>> "avtobiff/erlang-uuid" v1 will attempt to use random:uniform/1 to create more than 44bits of randomness if no interface is available. >>>> >>>> 2) There is an important problem within "avtobiff/erlang-uuid" for v4, since random:uniform/1 only provides 44 bits of randomness, if you are lucky (that is probably pushing it, since the algorithm may have been meant to only provide 32 bits), however, the v4 function is attempting to use random:uniform/1 to provide 48 bits and 62 bits, where the rest should become 0 because of the algorithm used by random:uniform/1. The other problem with v4 is that it reseeds random:uniform/1 each time the function is called with erlang:now(), which creates additional latency and possibly makes the v4 uuid less-random (depending on how it is called, e.g., on a timer). To see a better implementation for v4, using random:uniform/1, there is get_v4_urandom_bigint/0 and get_v4_urandom_native/0 in "okeuday/uuid", however, both solutions become slower in testing when compared with get_v4/0 which uses crypto:rand_bytes/1 instead of multiple calls to random:uniform/1. >>> Thanks for the comments on this. Erlang UUID no longer uses >>> random:uniform/1 to create anymore than 32 bits of random >>> data. >>> >>> However, how do you deduce that random:uniform/1 only provides >>> 44 bits of random data? Since a float is returned from random:random/0 >>> which is used by random:uniform/1, isn't the total random bits 16 or 24 >>> (4 and 3 words on 32-bit and 64-bit architectures respectively)? >>> >>> [1] http://www.erlang.org/doc/efficiency_guide/advanced.html#id68291 >>> >> I have not tried to track down the paper connected with the algorithm implemented within the random module (B.A. Wichmann and I.D.Hill, in 'An efficient and portable pseudo-random number generator', Journal of Applied Statistics. AS183. 1982, or Byte March 1987), so I am not sure if they were limited to 32 bit integers, but I assume they were. When I looked at this awhile ago, I forgot to add 1 to the result, so it is actually 45 bits, not 44 bits. Before looking at the algorithm, I knew empirically that the random module had some limitation that causes a lack of variation in the output I saw, and the way I consider it to be 45 bits is through looking at the input to the floating point number that is used (part of the random:uniform/0 code): >> B1 = (A1*171) rem 30269, >> B2 = (A2*172) rem 30307, >> B3 = (A3*170) rem 30323, >> put(random_seed, {B1,B2,B3}), >> R = A1/30269 + A2/30307 + A3/30323, >> R - trunc(R). >> So, {B1, B2, B3} becomes the next seed value {A1, A2, A3}. Simplifying this, gives: >> R = (918999161 * A1 + 917846887 * A2 + 917362583 * A3) / 27817185604309 >> Whatever the values for A1, A2, and A3, (918999161 * A1 + 917846887 * A2 + 917362583 * A3) can not exceed 27817185604309 (30269 * 30307 * 30323) because of the previous modulus. So, that means the algorithm breaks down, being unable to uniformly sample numbers beyond an N of 27817185604309, when considering the N provided to random:uniform/1. When determining how many bits it takes to represent 27817185604309, you get: >> 1> (math:log(27817185604309) / math:log(2)) + 1. >> 45.6610416965467 >> >> If you are concerned about the floating point part: >> 2> 1 / 27817185604309. >> 3.59489998098548e-14 >> >> As shown here http://en.wikipedia.org/wiki/Double-precision_floating-point_format , 15.955 decimal digits can be represented within a double precision floating point number. This is enough to contain the fraction, since Erlang is using double precision for the calculation, so it should not reduce the maximum number of bits that can be provided for uniformly random numbers. > Thanks for the run down on random bits! > > I actually tracked down the paper and skimmed it. AFAIK It is possible > (and even recommended IIRC) to combine several generators for > exceeding the 32-bit limit. Since they when the paper was written used > 16-bit arches also, but wanted a portable PRNG, combined generators > mean same on both 16-bit and 32-bit arches. > Sounds good, then just keep it to 45 bits and under. >>>> 3) The other important difference is the license. "avtobiff/erlang-uuid" is under a GPLv3 license, while "okeuday/uuid" is under a BSD license. So, there is nothing blocking the commercial use of "okeuday/uuid" (please remember I am not a lawyer, nor am I giving legal advice). >>> It is of course entirely possible to develop commercial applications with >>> GPL'd software. What you can't do is to transform free software into >>> proprietary software. >>> >> This is a little misleading. You can never sell anything that contains GPL software, so many people consider GPL software to poison a code-base, since it can limit a full stack from becoming a product (this is the reason why some BSD distributions want a compiler that is not gcc, etc.). So, the most you can do with GPL software is sell the service that is written with it, assuming you always submit your proprietary code changes to the developer of the GPL project (disclaimer: please remember I am not a lawyer, nor am I giving legal advice). > Sorry but you are wrong. You can of course sell GPL'd software! > Remember RedHat sells RHEL of which great parts are GPL'd. > What you can't do is to distribute only binaries, you have to supply > (buildable!) source code upon demand. See the Free Software > Foundations FAQ on the subject. > > [0] http://www.gnu.org/licenses/gpl-faq.html#DoesTheGPLAllowMoney > > So as long as you don't sell only binaries but also include the source > code (upon demand) you are complying with the GNU GPL. > > If you provide a service (say on the world wide web) you don't have to > provide source code at all, so it is fully compliant with GPL to use GPL'd > software for your proprietary web service. That is what the Affero GPL is for, > to ensure that services also distribute source code. E. g. of AGPL users > are gitorious.org and jabber.se. > I guess we will just agree to disagree on this. -- Michael From g@REDACTED Thu Mar 1 19:18:06 2012 From: g@REDACTED (Garrett Smith) Date: Thu, 1 Mar 2012 12:18:06 -0600 Subject: [erlang-questions] object vs. actor (Re: Massive Numbers of Actors vs. Massive Numbers of Objects vs. ????) In-Reply-To: <4F4FA98D.20503@meetinghouse.net> References: <4F4EB0BB.8000201@meetinghouse.net> <4F4FA98D.20503@meetinghouse.net> Message-ID: On Thu, Mar 1, 2012 at 10:53 AM, Miles Fidelman wrote: > Garrett Smith wrote: > >> - How do you store the data? (MySQL, MongoDB, Riak, Mnesia, multiple >> stores -- the list of options is *huge* and ) > > would sort of be nice to store data as simply a large cloud of Erlang > processes - hence inquiries and discussion here > > but, at least in the short term, most likely is CouchDB for early versions > (has most of the characteristics I'm looking for), though some possibility > of using eXist (XML document store, with APP/Atom interfaces already > available) > > probably leverage an erlang web framework (Yaws, Mochiweb, Nitrogen) > >> - What are you going to write your "business logic" in (Java, Python, >> Erlang, all of the above -- I assume your questions here are related >> to how suitable Erlang is for the job -- I'd say pretty good based on >> what you're looking for) > > embedded in documents, i.e., Javascript embedded in HTML/XML; possibly > embedded erlang as well (I'm pretty sure Couch supports embedded erlang > scripts) Ah! See comment below. >> - How are you going to communicate between nodes / servers / programs? > > protocol based - XMPP or Atom/APP, with support for message based protocols > as well (SMTP, NNTP, possibly IMAP for access) > >> Looking for a conceptual model is probably not a total waste of time, >> but at some point, you'll have to evaluate your options on those three >> points based on technical merit. > > I've generally found it useful as a way to organize thinking. > >> As far as the "actor" model and massive concurrency, as as been said, >> Erlang will give you options that other language environments won't. > > Absolulutely, but it doesn't quite go as far as I'd like. ?Erlang is great > for massive concurrency, not so great if you want huge numbers of processes > to persist, perhaps permanently. Erlang processes aren't persisted, but their state can be, quite easily. If the "business logic" associated with your entities/documents is expected to change so often that you want to store it as a part of the document itself, Erlang offers some excellent options. A trivial example, in an Erlang shell: 1> Add = fun(X, Y) -> X + Y end. 2> Add(10, 10). 100 3> file:write_file("addfun", term_to_binary(Add)). 4> init:stop(). And restart Erlang shell: 1> {ok, Bin} = file:read_file("addfun"). 2> Add = binary_to_term(Bin). 3> Add(10, 10). 100 You don't need CouchDB here -- just store your "logic" off in the database of your choice -- or as a base64 encoded blob in your document! > Our application will be distributing HTML+Javascript documents by email - > message like, but also object-like and actor like when live in a client or > browser. ?The question is how to maintain the actor-like behavior when > filing away large numbers of emails. > > It would be interesting to simply store each message as an erlang thread, > accessed through message passing. ?But that doesn't scale with current > technology. ?The short term answer is clearly to use a database of some > sort, but a longer solution would be to find ways to support large numbers > of hibernating processes. > >> You can even call them "objects" and the VM will work just the same :) >> > I could call them that, but erlang does NOT support object-oriented > programming in the sense of inheritance and so forth. If you're planning to store "code" with these documents, using side-effect free functions is a Very Good Idea, IMO. Btw, off topic, but this is an interesting take on OO (note the author): http://lists.squeakfoundation.org/pipermail/squeak-dev/1998-October/017019.html Garrett From erlang@REDACTED Thu Mar 1 19:25:16 2012 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 1 Mar 2012 19:25:16 +0100 Subject: [erlang-questions] erl2 tutorial Message-ID: Tutorials 1 and 2 ================= Download from https://github.com/joearms/erl2 Tutorial 1 ========== Here is a 10 line erl2 program: (see if you can guess what it does before reading the explanation) (( and tell me why if you guessed wrong)) $ cat example1.erl2 defMods mod1 shell end. addMod shell. def fac = fun(0) -> 1; (N) -> N*fac(N-1) end. 120 = fac(5). F25 = fac(25). addMod mod1. def a(N) -> F25 + N end. addMod shell. io:format("mod1:a(10)=~p~n",[mod1:a(10)]). erl2:make_mods(). To run the program we say: $./erl2 example1.erl2 Created:"gen/exprs.tmp" mod1:a(10)=15511210043330985984000010 make_mods saving generated code... Created:"gen/all.gen" Created:gen/mod1.erl Created:gen/shell.erl Success this generates two files $ cat gen/shell.erl -module(shell). -compile(export_all). fac(0) -> 1; fac(N) -> N * fac(N - 1). $cat gen/mod1.erl -module(mod1). -compile(export_all). a(N) -> F25 = 15511210043330985984000000, F25 + N. What do the commands do? 1) Start by defining which modules we will be creating > defMods mod1 shell end. 2) addMod X - means "we are adding code to the module X" so > addMod shell means we are adding some code to the module shell 3) Define a function > def fac = fun(0) -> 1; (N) -> N*fac(N-1) end. Defines a function fac (in shell since we add adding code to shell) Note: we can write self-referential functions here. 4) test the code > 120 = fac(5). 5) Run the code > F25 = fac(25). 6) Now add some code to mod1 > addMod mod1. > def a(N) -> F25 + N end. 7) Go back to adding code in the shell > addMod shell. 8) Test the function works > io:format("mod1:a(10)=~p~n",[mod1:a(10)]). 9) Generate the code > erl2:make_mods(). That's it Now we can look at the generated code Look at the code in mod1 - The shell binding for F25 was propagated into the shell. $ cat gen/shell.erl -module(shell). -compile(export_all). fac(0) -> 1; fac(N) -> N * fac(N - 1). $cat gen/mod1.erl -module(mod1). -compile(export_all). a(N) -> F25 = 15511210043330985984000000, F25 + N. Tutorial 2 ========== Now we change the code to the following: $ cat example1_fail.erl2 defMods mod1 shell end. addMod shell. def fac = fun(0) -> 1; (N) -> N*fac(N-1) end. 100 = fac(5). F25 = fac(25). addMod mod1. def a(N) -> F25 + N end. addMod shell. io:format("mod1:a(10)=~p~n",[mod1:a(10)]). erl2:make_mods(). This is *identical* to example1.erl2 BUT the unit test will fail since fac(5) is NOT 100 Now we run the program: $ ./erl2 example1_fail.erl2 Created:"gen/exprs.tmp" Error:{{badmatch,120},[{erl_eval,expr,3,[]}]} Compile failed Now NO CODE is generated. So 1) If the test fails the code is not compiled. 2) The unit tests are NOT run after the code is compiled but *before* 3) Compile modules failing the unit tests are *impossible* to create Cheers /Joe From mfidelman@REDACTED Thu Mar 1 20:42:34 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Thu, 01 Mar 2012 14:42:34 -0500 Subject: [erlang-questions] object vs. actor (Re: Massive Numbers of Actors vs. Massive Numbers of Objects vs. ????) In-Reply-To: References: <4F4EB0BB.8000201@meetinghouse.net> <4F4FA98D.20503@meetinghouse.net> Message-ID: <4F4FD12A.6000202@meetinghouse.net> Garrett, Garrett Smith wrote: > Erlang processes aren't persisted, but their state can be, quite easily. > > If the "business logic" associated with your entities/documents is > expected to change so often that you want to store it as a part of the > document itself, Erlang offers some excellent options. > > A trivial example, in an Erlang shell: > > 1> Add = fun(X, Y) -> X + Y end. > 2> Add(10, 10). > 100 > 3> file:write_file("addfun", term_to_binary(Add)). > 4> init:stop(). > > And restart Erlang shell: > > 1> {ok, Bin} = file:read_file("addfun"). > 2> Add = binary_to_term(Bin). > 3> Add(10, 10). > 100 That's not persisting a process' state, though. It's not so easy to say: 1. spawn, say, 10,000 gen_fsms, 2. run for a while 3. save the internal state of those 10,000 state machines 4. stop the shell (really the VM) 5. restart 6. restore those state machines, complete with their original PIDs Fairly straightforward to: 1. spawn 10,000 gen_fsms 2. add their PIDs to a name-to-PID lookup table 3. include logic such that an inactive fsm - times out - writes it's state to a file - updates the name-to-PID table to point the file - kill itself 4. write a listener that takes incoming messages, identifies the associated fsm by name and either - passes the message to the file, or - loads and restarts the process, then passes the messages What I've really done is rewrite erlang's hibernation functionality. > You don't need CouchDB here -- just store your "logic" off in the > database of your choice -- or as a base64 encoded blob in your > document! CouchDB provides a lot of baseline functionality that I'd just as soon not re-write. >> Our application will be distributing HTML+Javascript documents by email - >> message like, but also object-like and actor like when live in a client or >> browser. The question is how to maintain the actor-like behavior when >> filing away large numbers of emails. >> >> It would be interesting to simply store each message as an erlang thread, >> accessed through message passing. But that doesn't scale with current >> technology. The short term answer is clearly to use a database of some >> sort, but a longer solution would be to find ways to support large numbers >> of hibernating processes. >> >>> You can even call them "objects" and the VM will work just the same :) >>> >> I could call them that, but erlang does NOT support object-oriented >> programming in the sense of inheritance and so forth. > If you're planning to store "code" with these documents, using > side-effect free functions is a Very Good Idea, IMO. Well sure! > Btw, off topic, but this is an interesting take on OO (note the author): > > http://lists.squeakfoundation.org/pipermail/squeak-dev/1998-October/017019.html It is. As I understand it, Kay originally envisioned Smalltalk objects as more "actor-like" (i.e., each running as an independent process or thread), and then backed away from that. It sure would be nice if someone really did implement erlang-style concurrency in a smalltalk varient. (Flow of control always strikes me as the real shortcoming of smalltalk.) Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From raould@REDACTED Thu Mar 1 21:35:58 2012 From: raould@REDACTED (Raoul Duke) Date: Thu, 1 Mar 2012 12:35:58 -0800 Subject: [erlang-questions] other actory things (Re: object vs. actor (Re: Massive Numbers of Actors vs. Massive Numbers of Objects vs. ????)) Message-ID: On Thu, Mar 1, 2012 at 11:42 AM, Miles Fidelman wrote: > implement erlang-style concurrency in a smalltalk varient. ?(Flow of control > always strikes me as the real shortcoming of smalltalk.) maybe see flow based programming. https://en.wikipedia.org/wiki/Flow-based_programming From aronisstav@REDACTED Thu Mar 1 22:03:15 2012 From: aronisstav@REDACTED (Stavros Aronis) Date: Thu, 1 Mar 2012 13:03:15 -0800 (PST) Subject: [erlang-questions] Type Server In-Reply-To: <2F85B636-4713-4B77-927D-8F669BE0B0F5@gmail.com> References: <2F85B636-4713-4B77-927D-8F669BE0B0F5@gmail.com> Message-ID: <26560912.2245.1330635795455.JavaMail.geo-discussion-forums@vbai14> Hi Tim! I am not sure what you mean by a "type server", as Dialyzer does not really have a "component" that I could imagine being separated and labeled so. Dialyzer has the ability to store the specs and exported types in the PLT and from there use them to increase the precision of the analysis it performs. What do you mean by "working with types"? Regards, Stavros On Thursday, March 1, 2012 4:43:33 PM UTC+1, Tim Watson wrote: > > Hi all, > > Both PropEr and Dialyzer appear to have a 'type server' that provides > functions for working with type specs. What are the chances of this being > available as a separate unit? There is currently very little support for > working with types, as the previous post about lack of support in > erl_syntax pointed out. It would be nice to see type specs being better > supported. > > Cheers, > > Tim > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Thu Mar 1 22:11:39 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Thu, 01 Mar 2012 16:11:39 -0500 Subject: [erlang-questions] other actory things (Re: object vs. actor (Re: Massive Numbers of Actors vs. Massive Numbers of Objects vs. ????)) In-Reply-To: References: Message-ID: <4F4FE60B.7040603@meetinghouse.net> Raoul Duke wrote: > On Thu, Mar 1, 2012 at 11:42 AM, Miles Fidelman > wrote: >> implement erlang-style concurrency in a smalltalk varient. (Flow of control >> always strikes me as the real shortcoming of smalltalk.) > maybe see flow based programming. > https://en.wikipedia.org/wiki/Flow-based_programming > Thanks! There are some very interesting items that link off that page. Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From kostis@REDACTED Thu Mar 1 22:16:11 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 01 Mar 2012 23:16:11 +0200 Subject: [erlang-questions] Type Specs Preserved in Beam Code In-Reply-To: <395A7BEA-5244-4735-818A-AFA781A4A081@gmail.com> References: <395A7BEA-5244-4735-818A-AFA781A4A081@gmail.com> Message-ID: <4F4FE71B.4060204@cs.ntua.gr> On 03/01/12 17:46, Tim Watson wrote: > Current type specs are only available in beam code if you compile it with debug_info or use the 'abstract_code' parse transform that's floating around on the internet. What are the chances of getting the type spec information preserved in the beam (in a separate chunk) without having to do this? It would help if you provided some reasons why compiling with +debug_info is inconvenient for what you would really like to achieve. Recall that the original purpose of BEAM files has been to contain VM byte code and the default setting for them has been to be as compact as possible, so, naturally, chunks other than the one containing byte code have been frowned upon.(*) However, compiling with +debug_info essentially retains all source in the form of abstract code. Having special chunks with portions of the information which exists in this abstract code (and I am assuming here you want these chunks present by default) does not seem very appealing to me. If there are good reasons to be able to manipulate (the information in) the source, a better alternative would be to ask for abstract code to always be present in .beam files. (I.e., +debug_info by default and a new option like +min_size for environments where the size of beam files is a concern.) Kostis (*) IMO, this made perfect sense 10+ years ago; not sure this is so relevant a reason nowadays. From aronisstav@REDACTED Thu Mar 1 22:17:57 2012 From: aronisstav@REDACTED (Stavros Aronis) Date: Thu, 1 Mar 2012 13:17:57 -0800 (PST) Subject: [erlang-questions] Types and Specs support in erl_syntax or erl_prettypr In-Reply-To: References: Message-ID: <21685696.34.1330636677219.JavaMail.geo-discussion-forums@vbhm18> Hi Jason, The lack of support for types in erl_syntax is something that has bothered me as well and I think that I ran into the same issue with the erlang parser when I was trying to do some work there. If this is indeed the case, there should definitely be a way to make the representation of types in the abstract syntax better and uniform (for the records case). Keep in mind however that at least two tools (Dialyzer and EDoc) have been designed to use the current representation, so any change will need to be compatible with those. I can definitely help with the Dialyzer front. Regards, Stavros On Thursday, March 1, 2012 3:06:24 PM UTC+1, Jason Rogers wrote: > > I've noticed a lack of support for types and specs in erl_syntax or > erl_prettypr. > > If you try: > > {ok,{_,[{abstract_code,{_,AC}}]}} = beam_lib:chunks(Beam,[abstract_code]). > io:fwrite("~s~n", [erl_prettypr:format(erl_syntax:form_list(AC))]). > > on a beam file with type or spec declarations, the resulting code will > not compile. > > Any idea if official support is forthcoming? > > I tried to modify both libraries to provide some minimal support for > my own use, but ran into an issue with typed records. A typed record > shows up in the syntax tree twice: as a un-typed record and as a typed > record. And the typed version, which includes all the information > contained in the un-typed declaration, appears after the un-typed > declaration. > > So if you write code to process the typed version, you will end up > with source code that has both a typed and an un-typed declaration. > > Is there anyway to get to a single record declaration? > > Thanks, > Jason > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > On Thursday, March 1, 2012 3:06:24 PM UTC+1, Jason Rogers wrote: > > I've noticed a lack of support for types and specs in erl_syntax or > erl_prettypr. > > If you try: > > {ok,{_,[{abstract_code,{_,AC}}]}} = beam_lib:chunks(Beam,[abstract_code]). > io:fwrite("~s~n", [erl_prettypr:format(erl_syntax:form_list(AC))]). > > on a beam file with type or spec declarations, the resulting code will > not compile. > > Any idea if official support is forthcoming? > > I tried to modify both libraries to provide some minimal support for > my own use, but ran into an issue with typed records. A typed record > shows up in the syntax tree twice: as a un-typed record and as a typed > record. And the typed version, which includes all the information > contained in the un-typed declaration, appears after the un-typed > declaration. > > So if you write code to process the typed version, you will end up > with source code that has both a typed and an un-typed declaration. > > Is there anyway to get to a single record declaration? > > Thanks, > Jason > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Thu Mar 1 22:53:14 2012 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 1 Mar 2012 22:53:14 +0100 Subject: [erlang-questions] Types and Specs support in erl_syntax or erl_prettypr In-Reply-To: <21685696.34.1330636677219.JavaMail.geo-discussion-forums@vbhm18> References: <21685696.34.1330636677219.JavaMail.geo-discussion-forums@vbhm18> Message-ID: <4EDD8D82-272E-4420-B2B8-375373D5B178@feuerlabs.com> Here is how it's done in parse_trans_pp.erl (http://github.com/uwiger/parse_trans) pp_beam_to_str(F) -> case beam_lib:chunks(F, [abstract_code]) of {ok, {_, [{abstract_code,{_,AC0}}]}} -> AC = epp:restore_typed_record_fields(AC0), {ok, lists:flatten( io_lib:fwrite("~s~n", [lists:flatten( [erl_pp:form(Form) || Form <- AC])]) )}; Other -> {error, Other} end. BR, Ulf W On 1 Mar 2012, at 22:17, Stavros Aronis wrote: > Hi Jason, > > The lack of support for types in erl_syntax is something that has bothered me as well and I think that I ran into the same issue with the erlang parser when I was trying to do some work there. If this is indeed the case, there should definitely be a way to make the representation of types in the abstract syntax better and uniform (for the records case). > > Keep in mind however that at least two tools (Dialyzer and EDoc) have been designed to use the current representation, so any change will need to be compatible with those. I can definitely help with the Dialyzer front. > > Regards, > Stavros > > On Thursday, March 1, 2012 3:06:24 PM UTC+1, Jason Rogers wrote: > I've noticed a lack of support for types and specs in erl_syntax or > erl_prettypr. > If you try: > > {ok,{_,[{abstract_code,{_,AC}}]}} = beam_lib:chunks(Beam,[abstract_code]). > io:fwrite("~s~n", [erl_prettypr:format(erl_syntax:form_list(AC))]). > > on a beam file with type or spec declarations, the resulting code will > not compile. > > Any idea if official support is forthcoming? > > I tried to modify both libraries to provide some minimal support for > my own use, but ran into an issue with typed records. A typed record > shows up in the syntax tree twice: as a un-typed record and as a typed > record. And the typed version, which includes all the information > contained in the un-typed declaration, appears after the un-typed > declaration. > > So if you write code to process the typed version, you will end up > with source code that has both a typed and an un-typed declaration. > > Is there anyway to get to a single record declaration? > > Thanks, > Jason > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > On Thursday, March 1, 2012 3:06:24 PM UTC+1, Jason Rogers wrote: > I've noticed a lack of support for types and specs in erl_syntax or > erl_prettypr. > If you try: > > {ok,{_,[{abstract_code,{_,AC}}]}} = beam_lib:chunks(Beam,[abstract_code]). > io:fwrite("~s~n", [erl_prettypr:format(erl_syntax:form_list(AC))]). > > on a beam file with type or spec declarations, the resulting code will > not compile. > > Any idea if official support is forthcoming? > > I tried to modify both libraries to provide some minimal support for > my own use, but ran into an issue with typed records. A typed record > shows up in the syntax tree twice: as a un-typed record and as a typed > record. And the typed version, which includes all the information > contained in the un-typed declaration, appears after the un-typed > declaration. > > So if you write code to process the typed version, you will end up > with source code that has both a typed and an un-typed declaration. > > Is there anyway to get to a single record declaration? > > Thanks, > Jason > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Thu Mar 1 22:53:48 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 1 Mar 2012 21:53:48 +0000 Subject: [erlang-questions] Type Server In-Reply-To: <26560912.2245.1330635795455.JavaMail.geo-discussion-forums@vbai14> References: <2F85B636-4713-4B77-927D-8F669BE0B0F5@gmail.com> <26560912.2245.1330635795455.JavaMail.geo-discussion-forums@vbai14> Message-ID: <28C6337A-CC4F-4D68-B6D9-E263368BC105@gmail.com> Hi Stavros, Ok sorry for not stating my aims so clearly. I am looking at various ways to formalise my API declarations and make dynamic binding to implementations cleaner than having code that does `Mod = orddict, ...'. So for example, to go back to the recent discussions about having a unified API for dict. Instead of Mod = orddict, %% some more code... Mod:fetch(.....) I would like to be able to bind to an implementation once, and then after that use the generic module/api in my code: true = dictionary:is_key(a, dict:from_list([{a, 1}])), true = dictionary:is_key(a, orddict:from_list([{a, 1}])), true = dictionary:is_key(a, gb_trees:from_orddict(orddict:from_list([{a, 1}]))). So I was thinking about various ways to bind these, both at compile time and at runtime based on the conversation about Clojure's 'prototypes'. My general thought is that an implementation of the 'prototype' behaviour defined by the 'dictionary' module will basically be resolved based on the type(s) of the 1..* arguments the various behaviour callback functions are bound over. So if you've defined is_key like so: -spec is_key(term(), dictionary:instance(dictionary:unbound_type_parameter())) -> boolean(). then you will create many instances, one for dict, one for orddict, one for gb_trees, etc. Where the module 'natively' implements the interface, there's little work to do other than 'stating' (or registering somehow) that you consider it an implementation/instance of the prototype - forgive me I don't have a better term for this. For gb_trees, you'd basically write an implementation for each of the prototype's functions that need 'rewiring', such that you'd get -module(gb_trees_dictionary). -implements(dictionary, gb_trees:gb_tree()). -spec is_key(term(), dictionary:instance(gb_trees:gb_tree())) -> boolean(). is_key(K, D) -> gb_trees:is_defined(K, D). At compile time (or at runtime, possibly though beam code rewriting), I want to use the type specs for the various -implements modules (which could equally be declared anonymously using funs) to resolve the callee. Currently, there's very little to help me work with type specs. I can pull them out of the abstract code, but they're pretty 'low level' structures and AFAIK aren't documented anywhere. Now I'm simply assuming that there is loads of useful code in PropEr and Dialyzer that 'understands' these spec related data structures and was hoping I could make use of it, rather than learn my way around by fumbling in the dark. Hope that makes sense of how I'm trying to save myself a bit of time. Cheers, Tim On 1 Mar 2012, at 21:03, Stavros Aronis wrote: > Hi Tim! > > I am not sure what you mean by a "type server", as Dialyzer does not really have a "component" that I could imagine being separated and labeled so. Dialyzer has the ability to store the specs and exported types in the PLT and from there use them to increase the precision of the analysis it performs. > > What do you mean by "working with types"? > > Regards, > Stavros > > On Thursday, March 1, 2012 4:43:33 PM UTC+1, Tim Watson wrote: > Hi all, > Both PropEr and Dialyzer appear to have a 'type server' that provides functions for working with type specs. What are the chances of this being available as a separate unit? There is currently very little support for working with types, as the previous post about lack of support in erl_syntax pointed out. It would be nice to see type specs being better supported. > > Cheers, > > Tim > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Thu Mar 1 23:03:07 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 1 Mar 2012 22:03:07 +0000 Subject: [erlang-questions] Type Server In-Reply-To: <28C6337A-CC4F-4D68-B6D9-E263368BC105@gmail.com> References: <2F85B636-4713-4B77-927D-8F669BE0B0F5@gmail.com> <26560912.2245.1330635795455.JavaMail.geo-discussion-forums@vbai14> <28C6337A-CC4F-4D68-B6D9-E263368BC105@gmail.com> Message-ID: <2F4F0087-2D9A-4D16-BA11-DD34E764C397@gmail.com> On 1 Mar 2012, at 21:53, Tim Watson wrote: > > At compile time (or at runtime, possibly though beam code rewriting), I want to use the type specs for the various -implements modules (which could equally be declared anonymously using funs) to resolve the callee. Currently, there's very little to help me work with type specs. I can pull them out of the abstract code, but they're pretty 'low level' structures and AFAIK aren't documented anywhere. Now I'm simply assuming that there is loads of useful code in PropEr and Dialyzer that 'understands' these spec related data structures and was hoping I could make use of it, rather than learn my way around by fumbling in the dark. > Concrete example of this would be, how can I determine if a type is a generalisation/specialisation of another type? For example the relationship between list(term()) and list(atom()) or between tuple(term(), term()) and tuple(atom(), term()) where the type restriction is either narrowing or widening. I am fully aware there is no 'sub-typing' and that's not what I'm trying to do - I'm interested in using the type spec to identify, from a compatibility perspective, how the dispatch should work, so that I can rewrite either the caller or the callee to make the dispatch work properly - I'd prefer the latter, but that requires a bit more support from the -implements collaborator around identifying whether an arbitrary term is of a suitable type. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayson.barley@REDACTED Fri Mar 2 00:24:31 2012 From: jayson.barley@REDACTED (Jayson Barley) Date: Thu, 1 Mar 2012 15:24:31 -0800 Subject: [erlang-questions] TDD, Erlang and Emacs Message-ID: I am wondering if anyone has any recommendations for a smoother TDD experience with Erlang and Emacs. I am finding the process to be less than optimal. The way I am doing it now is that I have 3 windows open. 1. In my test module I write my test 2. Switch to the Erlang buffer 3. Run my tests, see that it fails 4. Switch to the module under test buffer, write code to get the test to pass 5. Switch back to the Erlang buffer, run my tests again 6. Switch back to the test buffer and repeat. Ideally I would like to have 2 windows open, 1 for the module under test and the other for the test window. From the module under test buffer or the test module I should be able to run the tests, see that they pass or fail and then move on.I have been considering writing an plugin to handle this functionality for me. Before I go that route I want to make sure I am not duplicating what someone else has already done. Does anyone have any recommendations for how to do this better? -------------- next part -------------- An HTML attachment was scrubbed... URL: From gbulmer@REDACTED Fri Mar 2 00:48:45 2012 From: gbulmer@REDACTED (G Bulmer) Date: Thu, 1 Mar 2012 23:48:45 +0000 Subject: [erlang-questions] Raspberry-Pi Message-ID: <064E04AC-7408-4934-91A6-BA0D6D495592@gmail.com> The Raspberry-Pi went on sale for $35. They sold the first batch of 10,000 very, very quickly: http://www.raspberrypi.org/ IMHO, a $35 Erlang machine with Ethernet, and no moving parts (boots from SD) is very interesting. It has some digital I/O available, and low-level interfaces like SPI and I2C, so it should talk to accelerometers, and gyro's, making it good fun for robots, or maybe home automation. Farnell quoted 23rd April for mine. From watson.timothy@REDACTED Fri Mar 2 01:20:44 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 2 Mar 2012 00:20:44 +0000 Subject: [erlang-questions] Type Specs Preserved in Beam Code In-Reply-To: <4F4FE71B.4060204@cs.ntua.gr> References: <395A7BEA-5244-4735-818A-AFA781A4A081@gmail.com> <4F4FE71B.4060204@cs.ntua.gr> Message-ID: <97B5CBDC-2193-4AE9-952F-CF7D5AEFF0F1@gmail.com> On 1 Mar 2012, at 21:16, Kostis Sagonas wrote: > On 03/01/12 17:46, Tim Watson wrote: >> Current type specs are only available in beam code if you compile it with debug_info or use the 'abstract_code' parse transform that's floating around on the internet. What are the chances of getting the type spec information preserved in the beam (in a separate chunk) without having to do this? > > It would help if you provided some reasons why compiling with +debug_info is inconvenient for what you would really like to achieve. Recall that the original purpose of BEAM files has been to contain VM byte code and the default setting for them has been to be as compact as possible, so, naturally, chunks other than the one containing byte code have been frowned upon.(*) > Maybe I'm wrong in this, but to me, the type spec is as much a useful part of the function signature as the name and arity. I would like to be able to call Mod:module_info and see type spec information, or maybe 1> erlang:fun_info(fun lists:map/2). [{module,lists}, {name,map}, {arity,2}, {env,[]}, {type,external}, {spec, ...}] > However, compiling with +debug_info essentially retains all source in the form of abstract code. Having special chunks with portions of the information which exists in this abstract code (and I am assuming here you want these chunks present by default) does not seem very appealing to me. Well to be honest I'm probably wrong about special chunks and the fact is I just want the information. > > If there are good reasons to be able to manipulate (the information in) the source, a better alternative would be to ask for abstract code to always be present in .beam files. (I.e., +debug_info by default and a new option like +min_size for environments where the size of beam files is a concern.) > Maybe that's the right thing to do, but I'm not entirely sure there's enough demand. Anyway the use case I have in mind is that a. I need to introspect the types/specs defined in someone else's module b. I have no control over how their module was compiled So just like module attributes are available at runtime regardless of the compiler flags given, I'd like the same for exported types (does this already work? - I haven't checked actually) and for type specs on functions. > Kostis > > (*) IMO, this made perfect sense 10+ years ago; not sure this is so relevant a reason nowadays. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Fri Mar 2 01:25:11 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 2 Mar 2012 00:25:11 +0000 Subject: [erlang-questions] Type Server In-Reply-To: <2F4F0087-2D9A-4D16-BA11-DD34E764C397@gmail.com> References: <2F85B636-4713-4B77-927D-8F669BE0B0F5@gmail.com> <26560912.2245.1330635795455.JavaMail.geo-discussion-forums@vbai14> <28C6337A-CC4F-4D68-B6D9-E263368BC105@gmail.com> <2F4F0087-2D9A-4D16-BA11-DD34E764C397@gmail.com> Message-ID: And what I'm really after figuring out, is whether dialyzer_codeserver is a useful interface to the type information. After poking around a little bit, it looks like Richard C's erl_types module is the API that I'm looking for. That appears to be part of hipe, which isn't always present in a release, so I'm not sure if I should be relying on it. On 1 Mar 2012, at 22:03, Tim Watson wrote: > On 1 Mar 2012, at 21:53, Tim Watson wrote: >> >> At compile time (or at runtime, possibly though beam code rewriting), I want to use the type specs for the various -implements modules (which could equally be declared anonymously using funs) to resolve the callee. Currently, there's very little to help me work with type specs. I can pull them out of the abstract code, but they're pretty 'low level' structures and AFAIK aren't documented anywhere. Now I'm simply assuming that there is loads of useful code in PropEr and Dialyzer that 'understands' these spec related data structures and was hoping I could make use of it, rather than learn my way around by fumbling in the dark. >> > > Concrete example of this would be, how can I determine if a type is a generalisation/specialisation of another type? For example the relationship between list(term()) and list(atom()) or between tuple(term(), term()) and tuple(atom(), term()) where the type restriction is either narrowing or widening. I am fully aware there is no 'sub-typing' and that's not what I'm trying to do - I'm interested in using the type spec to identify, from a compatibility perspective, how the dispatch should work, so that I can rewrite either the caller or the callee to make the dispatch work properly - I'd prefer the latter, but that requires a bit more support from the -implements collaborator around identifying whether an arbitrary term is of a suitable type. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Fri Mar 2 01:28:32 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 2 Mar 2012 00:28:32 +0000 Subject: [erlang-questions] Type Server In-Reply-To: References: <2F85B636-4713-4B77-927D-8F669BE0B0F5@gmail.com> <26560912.2245.1330635795455.JavaMail.geo-discussion-forums@vbai14> <28C6337A-CC4F-4D68-B6D9-E263368BC105@gmail.com> <2F4F0087-2D9A-4D16-BA11-DD34E764C397@gmail.com> Message-ID: <44DC0A67-7DD9-43A0-BCA9-78550A1F265F@gmail.com> I just found this on line 3179 of erl_types %% Abstract records. Used for comparing contracts. %% -spec t_abstract_records(erl_type(), dict()) -> erl_type(). And a bit later on.... %% Map over types. Depth first. Used by the contract checker. ?list is This sounds like almost 'exactly' what I'm after as a standalone thing. The contract checker is presumably looking at the compatibility of function type signatures. That's what I want to do - preferably without making a pigs ear of it by trying to write it myself. Cheers, Tim On 2 Mar 2012, at 00:25, Tim Watson wrote: > And what I'm really after figuring out, is whether dialyzer_codeserver is a useful interface to the type information. After poking around a little bit, it looks like Richard C's erl_types module is the API that I'm looking for. That appears to be part of hipe, which isn't always present in a release, so I'm not sure if I should be relying on it. > > On 1 Mar 2012, at 22:03, Tim Watson wrote: > >> On 1 Mar 2012, at 21:53, Tim Watson wrote: >>> >>> At compile time (or at runtime, possibly though beam code rewriting), I want to use the type specs for the various -implements modules (which could equally be declared anonymously using funs) to resolve the callee. Currently, there's very little to help me work with type specs. I can pull them out of the abstract code, but they're pretty 'low level' structures and AFAIK aren't documented anywhere. Now I'm simply assuming that there is loads of useful code in PropEr and Dialyzer that 'understands' these spec related data structures and was hoping I could make use of it, rather than learn my way around by fumbling in the dark. >>> >> >> Concrete example of this would be, how can I determine if a type is a generalisation/specialisation of another type? For example the relationship between list(term()) and list(atom()) or between tuple(term(), term()) and tuple(atom(), term()) where the type restriction is either narrowing or widening. I am fully aware there is no 'sub-typing' and that's not what I'm trying to do - I'm interested in using the type spec to identify, from a compatibility perspective, how the dispatch should work, so that I can rewrite either the caller or the callee to make the dispatch work properly - I'd prefer the latter, but that requires a bit more support from the -implements collaborator around identifying whether an arbitrary term is of a suitable type. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Fri Mar 2 01:42:26 2012 From: g@REDACTED (Garrett Smith) Date: Thu, 1 Mar 2012 18:42:26 -0600 Subject: [erlang-questions] TDD, Erlang and Emacs In-Reply-To: References: Message-ID: On Thu, Mar 1, 2012 at 5:24 PM, Jayson Barley wrote: > I am wondering if anyone has any recommendations for a smoother TDD > experience with Erlang and Emacs. I am finding the process to be less than > optimal. The way I am doing it now is that I have 3 windows open. > > In my test module I write my test > Switch to the Erlang buffer > Run my tests, see that it fails > Switch to the module under test buffer, write code to get the test to pass > Switch back to the Erlang buffer, run my tests again > Switch back to the test buffer and repeat. > > Ideally I would like to have 2 windows open, 1 for the module under test and > the other for the test window. From the module under test buffer or the test > module I should be able to run the tests, see that they pass or fail and > then move on.I have been considering writing an plugin to handle this > functionality for me. Before I go that route I want to make sure I am not > duplicating what someone else has already done. Take a look at the 'reloader' module in Mochiweb. It watches for changes to a module (recompile), reloads the module, and runs any tests, if exported. Garrett From ok@REDACTED Fri Mar 2 03:59:58 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 2 Mar 2012 15:59:58 +1300 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: <4F4FA9A4.9050909@gmail.com> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> Message-ID: >>> I have not tried to track down the paper connected with the algorithm implemented within the random module (B.A. Wichmann and I.D.Hill, in 'An efficient and portable pseudo-random number generator', Journal of Applied Statistics. AS183. 1982, or Byte March 1987), so I am not sure if they were limited to 32 bit integers, but I assume they were. In fact they were very explicitly concerned with generating good quality random numbers on machines with 16-bit integers but able to do 16 * 16 -> 32 32 mod 16 -> 16 using register pairs (as you could on a PDP-11, for example). AS183 basically uses three independent linear congruential generators with 15 bits of state each, giving the equivalent of one 45-bit linear congruential generator. At the time, people normally expected to work mostly with single-precision floats and this was ample to give you a 32-bit float result. This was really exciting to me in DEC-10 Prolog, where integers were held to just eighteen bits, although we did have the intermediate 36-bit results we needed to make AS183 work, and which didn't actually have floating point anyway. Here was something I could use on a PDP-11, a VAX, or a DEC-10, and get the same answers on each. As has been explained before in this mailing list, and also in a couple of Prolog mailing lists, is that AS183 was a fine algorithm in its day, but that day has long passed. Wichmann & Hill have since published a four-generator algorithm, see Wichmann and Hill have a new generator with a longer period. [2] B. A. Wichmann, I. D. Hill, Generating good pseudo-random numbers, Computational Statistics & Data Analysis, 51, 1614-1622 (2006). In fact generating full-precision random doubles is something RNG writers often neglect. As for generating random integers in the range [0,N) where N is a bignum, I would love to be pointed to a paper on the subject. From gianfranco.alongi@REDACTED Fri Mar 2 05:53:54 2012 From: gianfranco.alongi@REDACTED (Gianfranco Alongi) Date: Fri, 2 Mar 2012 05:53:54 +0100 Subject: [erlang-questions] TDD, Erlang and Emacs In-Reply-To: References: Message-ID: You could let the compile command (M - compile) do compilation and test execution in one? Maybe just call the compile target && test target? Just a very pragmatic idea. I work with a very quick cycle using tmux (thanks Magnus), emacs and make. On Mar 2, 2012 12:24 AM, "Jayson Barley" wrote: > I am wondering if anyone has any recommendations for a smoother TDD > experience with Erlang and Emacs. I am finding the process to be less than > optimal. The way I am doing it now is that I have 3 windows open. > > 1. In my test module I write my test > 2. Switch to the Erlang buffer > 3. Run my tests, see that it fails > 4. Switch to the module under test buffer, write code to get the test > to pass > 5. Switch back to the Erlang buffer, run my tests again > 6. Switch back to the test buffer and repeat. > > Ideally I would like to have 2 windows open, 1 for the module under test > and the other for the test window. From the module under test buffer or the > test module I should be able to run the tests, see that they pass or fail > and then move on.I have been considering writing an plugin to handle this > functionality for me. Before I go that route I want to make sure I am not > duplicating what someone else has already done. > > Does anyone have any recommendations for how to do this better? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Fri Mar 2 07:11:43 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 2 Mar 2012 07:11:43 +0100 Subject: [erlang-questions] TDD, Erlang and Emacs In-Reply-To: References: Message-ID: On Fri, Mar 2, 2012 at 05:53, Gianfranco Alongi wrote: > You could let the compile command (M - compile) do compilation and test > execution in one? Maybe just call the compile target && test target? Just yesterday I have pushed a change to Emacs-prelude for better support of rebar and Makefiles. It rebinds C-c C-k to function which finds root of the project (based on existence one of files: rebar.config, .git, .svn, .projectile) and runs rebar if available, otherwise checks for Makefile and runs 'make' if Makefile was found. 'C-x ' works well in that setup. It would be super easy to extend it to run tests. Best, Gleb Peregud From kenji.rikitake@REDACTED Fri Mar 2 07:48:01 2012 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Fri, 2 Mar 2012 15:48:01 +0900 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> Message-ID: <20120302064801.GA3620@k2r.org> FYI: A crude implementation in pure Erlang is available at https://github.com/jj1bdx/sfmt-erlang/blob/master/src/random_wh06.erl For the 2006 Wichmann-Hill RNG which Richard mentioned below. ++> Richard O'Keefe [2012-03-02 15:59:58 +1300]: [snip] > Wichmann & Hill have since published a four-generator algorithm, see > Wichmann and Hill have a new generator with a longer period. > [2] B. A. Wichmann, I. D. Hill, > Generating good pseudo-random numbers, > Computational Statistics & Data Analysis, 51, 1614-1622 (2006). BTW > In fact generating full-precision random doubles is something RNG writers often neglect. Indeed true. Providing enough bits for the mantissa digits is required. > As for generating random integers in the range [0,N) where N is a bignum, I would love > to be pointed to a paper on the subject. Me too - so far all I see is for fixed-length integers. Kenji Rikitake From Tobias.Schlager@REDACTED Fri Mar 2 08:54:31 2012 From: Tobias.Schlager@REDACTED (Tobias Schlager) Date: Fri, 2 Mar 2012 07:54:31 +0000 Subject: [erlang-questions] TDD, Erlang and Emacs In-Reply-To: References: Message-ID: <12F2115FD1CCEE4294943B2608A18FA3EB1B66@MAIL01.win.lbaum.eu> Hi, in case you decide to write your own emacs plugin I suggest you'll have a look at the erl-mvn-distel emacs mode. It's an emacs integration for projects built with the maven-erlang-plugin. You could port it to rebar projects (or whatever you're currently using to build and run your project). Regarding the features, I think its pretty close to what you're looking for: * JIT compiling of code/test code * toggling between code/test code (using F5) * running a single test case (F6) or all test cases of a test module (F7) (afaik this even works from the code under test) * test success or failure is indicated by coloring the test function header * tracing a single test case (SHIFT-F6) Link * http://github.com/sheyll/erl-mvn-distel/ Regards Tobias Von: erlang-questions-bounces@REDACTED [erlang-questions-bounces@REDACTED]" im Auftrag von "Jayson Barley [jayson.barley@REDACTED] Gesendet: Freitag, 2. M?rz 2012 00:24 An: erlang-questions@REDACTED Betreff: [erlang-questions] TDD, Erlang and Emacs I am wondering if anyone has any recommendations for a smoother TDD experience with Erlang and Emacs. I am finding the process to be less than optimal. The way I am doing it now is that I have 3 windows open. In my test module I write my test Switch to the Erlang bufferRun my tests, see that it failsSwitch to the module under test buffer, write code to get the test to passSwitch back to the Erlang buffer, run my tests againSwitch back to the test buffer and repeat. Ideally I would like to have 2 windows open, 1 for the module under test and the other for the test window. From the module under test buffer or the test module I should be able to run the tests, see that they pass or fail and then move on.I have been considering writing an plugin to handle this functionality for me. Before I go that route I want to make sure I am not duplicating what someone else has already done. Does anyone have any recommendations for how to do this better? From s.j.thompson@REDACTED Fri Mar 2 10:02:20 2012 From: s.j.thompson@REDACTED (Simon Thompson) Date: Fri, 2 Mar 2012 09:02:20 +0000 Subject: [erlang-questions] Researcher / Lecturer position at Univ. of Kent Message-ID: <3530C838-C344-43F7-A352-BB100CC4B65D@kent.ac.uk> We're advertising a lectureship at Kent, with a special twist: the candidate will spend their first three years "developing as an independent researcher of internationally excellent quality in an area relevant to the School of Computing." In other words it's a research position that morphs into a traditional lectureship, and is designed to be particularly attractive to people at the start of their careers. The functional programmers at Kent would be particularly keen to recruit someone who would work with them! Online application: http://www11.i-grasp.com/fe/tpl_kent01.asp?newms=jj&id=36123&newlang=1 Closing date: 1/4/12, interviews: 30/4/12 If you have any questions do get in touch. Regards. Simon Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt From fabrice.nourisson@REDACTED Fri Mar 2 10:27:16 2012 From: fabrice.nourisson@REDACTED (Fabrice Nourisson) Date: Fri, 02 Mar 2012 10:27:16 +0100 Subject: [erlang-questions] TDD, Erlang and Emacs In-Reply-To: References: Message-ID: <4F509274.1090403@gmail.com> Hi, I use Active-Ring (https://github.com/extremeforge/active-ring) to develop in TDD with Erlang and Emacs. In normal use my emacs's screen is split into 3 parts - one for the test module - one for the tested module - one for the continuous test results In this depot there is an emacs-plugin to run the test in emacs. This plugin bind the emacs's compile command to run tests into emacs. For more information you could read the README. Fabrice Nourisson Le 02/03/2012 00:24, Jayson Barley a ?crit : > I am wondering if anyone has any recommendations for a smoother TDD > experience with Erlang and Emacs. I am finding the process to be less > than optimal. The way I am doing it now is that I have 3 windows open. > > 1. In my test module I write my test > 2. Switch to the Erlang buffer > 3. Run my tests, see that it fails > 4. Switch to the module under test buffer, write code to get the > test to pass > 5. Switch back to the Erlang buffer, run my tests again > 6. Switch back to the test buffer and repeat. > > Ideally I would like to have 2 windows open, 1 for the module under > test and the other for the test window. From the module under test > buffer or the test module I should be able to run the tests, see that > they pass or fail and then move on.I have been considering writing an > plugin to handle this functionality for me. Before I go that route I > want to make sure I am not duplicating what someone else has already done. > > Does anyone have any recommendations for how to do this better? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Fri Mar 2 12:13:44 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Fri, 2 Mar 2012 20:13:44 +0900 Subject: [erlang-questions] Raspberry-Pi In-Reply-To: <064E04AC-7408-4934-91A6-BA0D6D495592@gmail.com> References: <064E04AC-7408-4934-91A6-BA0D6D495592@gmail.com> Message-ID: > ... a $35 Erlang machine with Ethernet ... --if based on Raspberry-Pi, would be called "Raspberry-e", wouldn't it? -michael turner On Fri, Mar 2, 2012 at 8:48 AM, G Bulmer wrote: > The Raspberry-Pi went on sale for $35. They sold the first batch of 10,000 > very, very quickly: > http://www.raspberrypi.org/ > > IMHO, a $35 Erlang machine with Ethernet, and no moving parts (boots from > SD) is very interesting. > It has some digital I/O available, and low-level interfaces like SPI and > I2C, so it should talk to accelerometers, and gyro's, making it good fun for > robots, or maybe home automation. > > Farnell quoted 23rd April for mine. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From matthias@REDACTED Fri Mar 2 12:21:37 2012 From: matthias@REDACTED (Matthias Lang) Date: Fri, 2 Mar 2012 12:21:37 +0100 Subject: [erlang-questions] has anyone cross compiled HiPE? In-Reply-To: <20120220092424.GA3161@corelatus.se> References: <20120220092424.GA3161@corelatus.se> Message-ID: <20120302112137.GA14059@corelatus.se> I have a partial answer to my own question #2 (I answered #1 earlier). Quoting myself: > Second, I need to compile my Erlang code to ARM native code. > I can't see an option to do that, i.e. if I use c(my_module, native) > on my AMD64, I get an extra chunk in the .beam file called > "HA64". I'm guessing that's AMD-64 specific code. So I need some > way to tell HiPE that I want it to make ARM native code. > > (I've looked at TFM, specifically the HiPE App and the compile > module page. Nothing there about generating code for specific > targets. Maybe there's another page I've missed.) HiPE actually has quite a bit of documentation, it's just not in the usual spot, it's edoc-style, in the source, e.g. in hipe.erl. In short, you can ask HiPE to generate ARM code even when running on an x86, like this: c(example_module, [native, {hipe, {target, arm}}]). this makes a .beam file with ARM native code in it. Great...except that the ARM VM refuses to load that code. So far, I haven't gotten around that, but it can't be that hard. Rest of post is about what I've done so far. --- In theory, there shouldn't be a problem here. Hipe is just Erlang code which makes code for some CPU. That Erlang code should generate the same thing no matter what CPU it happens to be running on. In practice, HiPE makes different things depending on what system it's running on. For starters, the generated code has a signature in it which identifies the generating system. The loading system checks that signature and refuses to load if the signatures don't match. This signature is called HIPE_SYSTEM_CRC. Right, no problem then. If I use the same hipe compiler .beams as are running on the ARM system, the HIPE_SYSTEM_CRC macro will have the right magic value in it. I can tell the code loader on my x86 system to use that code: code:del_path(hipe). code:add_path("/home/matthias/gth3/rootfs/packages/erlang/otp_src_R15B/lib/hipe/ebin"). but the resulting .beams _still_ have the wrong signature in them. It happens because the signature generation also throws in information from 'hipe_bifs', which is C code. Bummer. Maybe the signature is the only piece of information which leaks out from the C code. Quick hack: disable the signature check. And, presto, it works for simple code. For more complex code, it segfaults. There must be something more to this. To be continued in a week or two, I'm out of time for now. Matt From erlang@REDACTED Fri Mar 2 12:42:27 2012 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 2 Mar 2012 12:42:27 +0100 Subject: [erlang-questions] Frying pan bug Message-ID: -module(bug). -compile(export_all). bug() -> p(0.3, 4, 11). c(N, M) -> fac(M) div (fac(N)*fac(M-N)). fac(0) -> 1; fac(N) -> N*fac(N-1). p(P, M, N) -> 1.0 - math:pow(1-P, N-M)*c(N, M). Running bug:bug() tries to turn my machine into a frying pan Should this be renamed frying_pan.erl ? Anybody else see this? /Joe From vladdu55@REDACTED Fri Mar 2 12:47:42 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 2 Mar 2012 12:47:42 +0100 Subject: [erlang-questions] Frying pan bug In-Reply-To: References: Message-ID: Hi Joe, On Fri, Mar 2, 2012 at 12:42, Joe Armstrong wrote: > c(N, M) -> ?fac(M) div (fac(N)*fac(M-N)). > p(P, M, N) -> 1.0 - math:pow(1-P, N-M)*c(N, M). Are you sure you don't mean p(P, N, M) -> ... ? The factorial for -7 will loop forever... regards, Vlad From kenneth.lundin@REDACTED Fri Mar 2 12:53:54 2012 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Fri, 2 Mar 2012 12:53:54 +0100 Subject: [erlang-questions] Frying pan bug In-Reply-To: References: Message-ID: You don't handle negative input to fac/1 and you feed it with 4-11 -> -7 /Kenneth On Fri, Mar 2, 2012 at 12:42 PM, Joe Armstrong wrote: > -module(bug). > -compile(export_all). > > bug() -> p(0.3, 4, 11). > > c(N, M) -> fac(M) div (fac(N)*fac(M-N)). > > fac(0) -> 1; > fac(N) -> N*fac(N-1). > > p(P, M, N) -> 1.0 - math:pow(1-P, N-M)*c(N, M). > > Running bug:bug() tries to turn my machine into a frying pan > > Should this be renamed frying_pan.erl ? > > Anybody else see this? > > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Fri Mar 2 12:56:12 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 02 Mar 2012 12:56:12 +0100 Subject: [erlang-questions] Frying pan bug In-Reply-To: References: Message-ID: <4F50B55C.3050300@cs.ntua.gr> On 03/02/2012 12:42 PM, Joe Armstrong wrote: > -module(bug). > -compile(export_all). > > bug() -> p(0.3, 4, 11). > > c(N, M) -> fac(M) div (fac(N)*fac(M-N)). > > fac(0) -> 1; > fac(N) -> N*fac(N-1). > > p(P, M, N) -> 1.0 - math:pow(1-P, N-M)*c(N, M). > > Running bug:bug() tries to turn my machine into a frying pan > > Should this be renamed frying_pan.erl ? No,perhaps it should be renamed to "factorial_not_defined_properly.erl". If you write the factorial function as follows: fac(0) -> 1; fac(N) when N > 0 -> N*fac(N-1). you will be able to see what's happening. Cheers, Kostis From watson.timothy@REDACTED Fri Mar 2 12:59:27 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 2 Mar 2012 11:59:27 +0000 Subject: [erlang-questions] Type Server In-Reply-To: <26560912.2245.1330635795455.JavaMail.geo-discussion-forums@vbai14> References: <2F85B636-4713-4B77-927D-8F669BE0B0F5@gmail.com> <26560912.2245.1330635795455.JavaMail.geo-discussion-forums@vbai14> Message-ID: <70D8D146-49B7-4FED-A3C4-4F7CF233315F@gmail.com> After poking around in the dialyzer source code a little bit, I think what I'm basically after is being able to run dialyzer_contracts:check_contract/2 independently, without doing a full analysis. I don't care about processing a whole call graph, I'm really just looking to compare two function signatures to figure out if they're compatible. Possibly I'm also interesting in taking one function spec and a set of potentially matching specs and finding the best (most strict?) success typing. Is that something I can do by selectively reusing parts of dialyzer? I've poked around in the shell trying to do things like get_top_level_signatures, get_spec_info and things like that, but I can't quite figure out the right entry point to start with. I've even tried a few of the debugging functions such as dialyzer_succ_typings:doit/1 but to no avail. Any pointers would be gratefully received! Cheers, Tim On 1 Mar 2012, at 21:03, Stavros Aronis wrote: > Hi Tim! > > I am not sure what you mean by a "type server", as Dialyzer does not really have a "component" that I could imagine being separated and labeled so. Dialyzer has the ability to store the specs and exported types in the PLT and from there use them to increase the precision of the analysis it performs. > > What do you mean by "working with types"? > > Regards, > Stavros > > On Thursday, March 1, 2012 4:43:33 PM UTC+1, Tim Watson wrote: > Hi all, > Both PropEr and Dialyzer appear to have a 'type server' that provides functions for working with type specs. What are the chances of this being available as a separate unit? There is currently very little support for working with types, as the previous post about lack of support in erl_syntax pointed out. It would be nice to see type specs being better supported. > > Cheers, > > Tim > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From aronisstav@REDACTED Fri Mar 2 13:18:00 2012 From: aronisstav@REDACTED (Stavros Aronis) Date: Fri, 2 Mar 2012 13:18:00 +0100 Subject: [erlang-questions] Type Server In-Reply-To: <70D8D146-49B7-4FED-A3C4-4F7CF233315F@gmail.com> References: <2F85B636-4713-4B77-927D-8F669BE0B0F5@gmail.com> <26560912.2245.1330635795455.JavaMail.geo-discussion-forums@vbai14> <70D8D146-49B7-4FED-A3C4-4F7CF233315F@gmail.com> Message-ID: Hi Tim, It would be better in my opinion if you tried to rely only on the erl_types module. erl_types:t_from_form/1 and erl_types:t_is_subtype/2 could possibly be enough for you. Dialyzer modules export few of their functions and these usually return the data in a way that is useful to Dialyzer and difficult to see through. You can however follow the code in dialyzer_contracts module, especially dialyzer_contracts:store_tmp_contract/5 and dialyzer_contracts:process_contract_remote_types/1. dialyzer_contracts:store_tmp_contract/5 actually calls erl_types:t_from_form/1 to convert an abstract syntax spec into the representation used in erl_types and after dialyzer_contracts:process_contract_remote_types/1 has done its magic it's very easy to use erl_types:t_is_subtype/2 for the checks that you are describing! Cheers, Stavros On Fri, Mar 2, 2012 at 12:59 PM, Tim Watson wrote: > After poking around in the dialyzer source code a little bit, I think what > I'm basically after is being able to run > dialyzer_contracts:check_contract/2 independently, without doing a full > analysis. I don't care about processing a whole call graph, I'm really just > looking to compare two function signatures to figure out if they're > compatible. Possibly I'm also interesting in taking one function spec and a > set of potentially matching specs and finding the best (most strict?) > success typing. Is that something I can do by selectively reusing parts of > dialyzer? > > I've poked around in the shell trying to do things > like get_top_level_signatures, get_spec_info and things like that, but I > can't quite figure out the right entry point to start with. I've even tried > a few of the debugging functions such as dialyzer_succ_typings:doit/1 but > to no avail. > > Any pointers would be gratefully received! > > Cheers, > > Tim > > On 1 Mar 2012, at 21:03, Stavros Aronis wrote: > > Hi Tim! > > I am not sure what you mean by a "type server", as Dialyzer does not > really have a "component" that I could imagine being separated and labeled > so. Dialyzer has the ability to store the specs and exported types in the > PLT and from there use them to increase the precision of the analysis it > performs. > > What do you mean by "working with types"? > > Regards, > Stavros > > On Thursday, March 1, 2012 4:43:33 PM UTC+1, Tim Watson wrote: >> >> Hi all, >> >> Both PropEr and Dialyzer appear to have a 'type server' that provides >> functions for working with type specs. What are the chances of this being >> available as a separate unit? There is currently very little support for >> working with types, as the previous post about lack of support in >> erl_syntax pointed out. It would be nice to see type specs being better >> supported. >> >> Cheers, >> >> Tim >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> >> >> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Fri Mar 2 13:41:28 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 2 Mar 2012 12:41:28 +0000 Subject: [erlang-questions] Type Server In-Reply-To: References: <2F85B636-4713-4B77-927D-8F669BE0B0F5@gmail.com> <26560912.2245.1330635795455.JavaMail.geo-discussion-forums@vbai14> <70D8D146-49B7-4FED-A3C4-4F7CF233315F@gmail.com> Message-ID: <5D48F21E-BA24-49D4-A990-8FFB637AEA3E@gmail.com> Thank you Stavros, that sounds perfect. I'll go poke around in dialyzer_contracts and use it as a guide to consuming erl_types. Cheers, Tim On 2 Mar 2012, at 12:18, Stavros Aronis wrote: > Hi Tim, > > It would be better in my opinion if you tried to rely only on the erl_types module. erl_types:t_from_form/1 and erl_types:t_is_subtype/2 could possibly be enough for you. > > Dialyzer modules export few of their functions and these usually return the data in a way that is useful to Dialyzer and difficult to see through. You can however follow the code in dialyzer_contracts module, especially dialyzer_contracts:store_tmp_contract/5 and dialyzer_contracts:process_contract_remote_types/1. dialyzer_contracts:store_tmp_contract/5 actually calls erl_types:t_from_form/1 to convert an abstract syntax spec into the representation used in erl_types and after dialyzer_contracts:process_contract_remote_types/1 has done its magic it's very easy to use erl_types:t_is_subtype/2 for the checks that you are describing! > > Cheers, > Stavros > > > On Fri, Mar 2, 2012 at 12:59 PM, Tim Watson wrote: > After poking around in the dialyzer source code a little bit, I think what I'm basically after is being able to run dialyzer_contracts:check_contract/2 independently, without doing a full analysis. I don't care about processing a whole call graph, I'm really just looking to compare two function signatures to figure out if they're compatible. Possibly I'm also interesting in taking one function spec and a set of potentially matching specs and finding the best (most strict?) success typing. Is that something I can do by selectively reusing parts of dialyzer? > > I've poked around in the shell trying to do things like get_top_level_signatures, get_spec_info and things like that, but I can't quite figure out the right entry point to start with. I've even tried a few of the debugging functions such as dialyzer_succ_typings:doit/1 but to no avail. > > Any pointers would be gratefully received! > > Cheers, > > Tim > > On 1 Mar 2012, at 21:03, Stavros Aronis wrote: > >> Hi Tim! >> >> I am not sure what you mean by a "type server", as Dialyzer does not really have a "component" that I could imagine being separated and labeled so. Dialyzer has the ability to store the specs and exported types in the PLT and from there use them to increase the precision of the analysis it performs. >> >> What do you mean by "working with types"? >> >> Regards, >> Stavros >> >> On Thursday, March 1, 2012 4:43:33 PM UTC+1, Tim Watson wrote: >> Hi all, >> Both PropEr and Dialyzer appear to have a 'type server' that provides functions for working with type specs. What are the chances of this being available as a separate unit? There is currently very little support for working with types, as the previous post about lack of support in erl_syntax pointed out. It would be nice to see type specs being better supported. >> >> Cheers, >> >> Tim >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From emeka_1978@REDACTED Fri Mar 2 15:28:03 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Fri, 2 Mar 2012 06:28:03 -0800 (PST) Subject: [erlang-questions] Erlang and SOCKS 5 Message-ID: <750266d2-652b-442f-86c9-67f38feae2bd@s7g2000yqm.googlegroups.com> Sorry if you receive this mail a second time, i could not see the first one after i sent it. Like i said in my first mail, i am a little bit stuck at the moment. I have been writing a couple of apps in erlang in my company. Today i was told that all my tcp connections have to go through a proxy server. I was not able to find anything in the erlang documentation related to setting the proxy server host/port on the socket options. Does erlang support such thing? From erlang@REDACTED Fri Mar 2 15:31:45 2012 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 2 Mar 2012 15:31:45 +0100 Subject: [erlang-questions] Frying pan bug In-Reply-To: <4F50B55C.3050300@cs.ntua.gr> References: <4F50B55C.3050300@cs.ntua.gr> Message-ID: Silly me - thanks anyway /Joe On Fri, Mar 2, 2012 at 12:56 PM, Kostis Sagonas wrote: > On 03/02/2012 12:42 PM, Joe Armstrong wrote: >> >> -module(bug). >> -compile(export_all). >> >> bug() -> ?p(0.3, 4, 11). >> >> c(N, M) -> ? fac(M) div (fac(N)*fac(M-N)). >> >> fac(0) -> ?1; >> fac(N) -> ?N*fac(N-1). >> >> p(P, M, N) -> ?1.0 - math:pow(1-P, N-M)*c(N, M). >> >> Running bug:bug() tries to turn my machine into a frying pan >> >> Should this be renamed frying_pan.erl ? > > > No,perhaps it should be renamed to "factorial_not_defined_properly.erl". > > If you write the factorial function as follows: > > ?fac(0) -> 1; > ?fac(N) when N > 0 -> N*fac(N-1). > > you will be able to see what's happening. > > Cheers, > Kostis > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From tony@REDACTED Fri Mar 2 15:35:25 2012 From: tony@REDACTED (Tony Rogvall) Date: Fri, 2 Mar 2012 15:35:25 +0100 Subject: [erlang-questions] Erlang and SOCKS 5 In-Reply-To: <750266d2-652b-442f-86c9-67f38feae2bd@s7g2000yqm.googlegroups.com> References: <750266d2-652b-442f-86c9-67f38feae2bd@s7g2000yqm.googlegroups.com> Message-ID: <8CD95C3F-63A4-4431-ADEE-30D12D631C40@rogvall.se> They are old and gone.... http://www.trapexit.org/forum/viewtopic.php?t=3957&sid=1d0af3385415e70084885001a9e7c9fd :-) /Tony On 2 mar 2012, at 15:28, eigenfunction wrote: > Sorry if you receive this mail a second time, i could not see the > first one after i sent it. > Like i said in my first mail, i am a little bit stuck at the moment. I > have been writing a couple of apps in erlang in my company. Today i > was told that all my tcp connections have to go through a proxy > server. I was not able to find anything in the erlang documentation > related to setting the proxy server host/port on the socket options. > Does erlang support such thing? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From emeka_1978@REDACTED Fri Mar 2 15:41:56 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Fri, 2 Mar 2012 06:41:56 -0800 (PST) Subject: [erlang-questions] Erlang and SOCKS 5 In-Reply-To: <8CD95C3F-63A4-4431-ADEE-30D12D631C40@rogvall.se> References: <750266d2-652b-442f-86c9-67f38feae2bd@s7g2000yqm.googlegroups.com> <8CD95C3F-63A4-4431-ADEE-30D12D631C40@rogvall.se> Message-ID: <3d5f9a43-07e2-42ab-a624-01fe5832d0dc@p12g2000yqe.googlegroups.com> Thank you for the quick answer. Anyone has any idea or workaround about sending tcp packets through a proxy with authentication in erlang? It will be bad for me if i had to rewrite things in java only because of that. On 2 Mrz., 15:35, Tony Rogvall wrote: > They are old and gone.... > > http://www.trapexit.org/forum/viewtopic.php?t=3957&sid=1d0af3385415e7... > > :-) > > /Tony > > On 2 mar 2012, at 15:28, eigenfunction wrote: > > > Sorry if you receive this mail a second time, i could not see the > > first one after i sent it. > > Like i said in my first mail, i am a little bit stuck at the moment. I > > have been writing a couple of apps in erlang in my company. Today i > > was told that all my tcp connections have to go through a proxy > > server. I was not able to find anything in the erlang documentation > > related to setting the proxy server host/port on the socket options. > > Does erlang support such thing? > > _______________________________________________ > > erlang-questions mailing list > > erlang-questi...@REDACTED > >http://erlang.org/mailman/listinfo/erlang-questions > > "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" > > _______________________________________________ > erlang-questions mailing list > erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions From dave@REDACTED Fri Mar 2 15:42:34 2012 From: dave@REDACTED (David Goehrig) Date: Fri, 2 Mar 2012 09:42:34 -0500 Subject: [erlang-questions] Frying pan bug In-Reply-To: References: Message-ID: I'm curious, I know that people are "right" when they say fix validate the inputs to factorial, but there are equations far less well known with similar degenerate cases where writing guards are inexpressible due to there being an infinite number of degenerate points. Not all curves are continuous even in the real world. Would it be possible for Erlang's runtime to set a threshold, say looking at the thermal sensor on the board and error out if that "guard" is reached. You could do the same for disk I/O, network, or any other saturation condition. This has been recently on my mind as we just did a poorman's thermal chamber test, and I've been writing a lot of Erlang that monitors the behavior of other apps under environmental stress, but have yet to solve the "how does the monitor that monitors itself know when to throw up the white flag and surrender?" -=-=- dave@REDACTED -=-=- On Mar 2, 2012, at 6:42 AM, Joe Armstrong wrote: > -module(bug). > -compile(export_all). > > bug() -> p(0.3, 4, 11). > > c(N, M) -> fac(M) div (fac(N)*fac(M-N)). > > fac(0) -> 1; > fac(N) -> N*fac(N-1). > > p(P, M, N) -> 1.0 - math:pow(1-P, N-M)*c(N, M). > > Running bug:bug() tries to turn my machine into a frying pan > > Should this be renamed frying_pan.erl ? > > Anybody else see this? > > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From tony@REDACTED Fri Mar 2 15:43:38 2012 From: tony@REDACTED (Tony Rogvall) Date: Fri, 2 Mar 2012 15:43:38 +0100 Subject: [erlang-questions] Erlang and SOCKS 5 In-Reply-To: <3d5f9a43-07e2-42ab-a624-01fe5832d0dc@p12g2000yqe.googlegroups.com> References: <750266d2-652b-442f-86c9-67f38feae2bd@s7g2000yqm.googlegroups.com> <8CD95C3F-63A4-4431-ADEE-30D12D631C40@rogvall.se> <3d5f9a43-07e2-42ab-a624-01fe5832d0dc@p12g2000yqe.googlegroups.com> Message-ID: Maybe you can pick up the socks modules from the old release and port them ? /Tony On 2 mar 2012, at 15:41, eigenfunction wrote: > Thank you for the quick answer. Anyone has any idea or workaround > about sending tcp packets through a proxy with authentication in > erlang? > It will be bad for me if i had to rewrite things in java only because > of that. > > On 2 Mrz., 15:35, Tony Rogvall wrote: >> They are old and gone.... >> >> http://www.trapexit.org/forum/viewtopic.php?t=3957&sid=1d0af3385415e7... >> >> :-) >> >> /Tony >> >> On 2 mar 2012, at 15:28, eigenfunction wrote: >> >>> Sorry if you receive this mail a second time, i could not see the >>> first one after i sent it. >>> Like i said in my first mail, i am a little bit stuck at the moment. I >>> have been writing a couple of apps in erlang in my company. Today i >>> was told that all my tcp connections have to go through a proxy >>> server. I was not able to find anything in the erlang documentation >>> related to setting the proxy server host/port on the socket options. >>> Does erlang support such thing? >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questi...@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From hvjunk@REDACTED Fri Mar 2 16:10:47 2012 From: hvjunk@REDACTED (Hendrik Visage) Date: Fri, 2 Mar 2012 17:10:47 +0200 Subject: [erlang-questions] Hot erlang Re: Frying pan bug Message-ID: On Fri, Mar 2, 2012 at 4:42 PM, David Goehrig wrote: > Would it be possible for Erlang's runtime to set a threshold, say looking at the thermal sensor on the board and error out if that "guard" is reached. You could do the same for disk I/O, network, or any other saturation condition. I would rather prefer the OS to send a SIGimInHeat signal to the Erlang VM :) On the serious side: I see that as a function of the OS and hardware to throttle the CPU and processes whenever overheating situations occur. erlang VMs should rather track the usage times of it's processes based on simple threshold values, but isn't that getting to the point where we have the VM baby sitting bad coders?? From emeka_1978@REDACTED Fri Mar 2 16:51:29 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Fri, 2 Mar 2012 07:51:29 -0800 (PST) Subject: [erlang-questions] Erlang and SOCKS 5 In-Reply-To: References: <750266d2-652b-442f-86c9-67f38feae2bd@s7g2000yqm.googlegroups.com> <8CD95C3F-63A4-4431-ADEE-30D12D631C40@rogvall.se> <3d5f9a43-07e2-42ab-a624-01fe5832d0dc@p12g2000yqe.googlegroups.com> Message-ID: <4e0afe28-bd6d-4d53-8bba-24e78d93a15e@c21g2000yqi.googlegroups.com> If only i could find those releases? Since i am on a tight deadline, i may rewrite part of my apps in scala. But anyway, does anybody know how i can get my hand on release 10b? On 2 Mrz., 15:43, Tony Rogvall wrote: > Maybe you can pick up the socks modules from the old release > and port them ? > > /Tony > > On 2 mar 2012, at 15:41, eigenfunction wrote: > > > > > > > > > > > Thank you for the quick answer. Anyone has any idea or workaround > > about sending tcp packets through a proxy with authentication in > > erlang? > > It will be bad for me if i had to rewrite things in java only because > > of that. > > > On 2 Mrz., 15:35, Tony Rogvall wrote: > >> They are old and gone.... > > >>http://www.trapexit.org/forum/viewtopic.php?t=3957&sid=1d0af3385415e7... > > >> :-) > > >> /Tony > > >> On 2 mar 2012, at 15:28, eigenfunction wrote: > > >>> Sorry if you receive this mail a second time, i could not see the > >>> first one after i sent it. > >>> Like i said in my first mail, i am a little bit stuck at the moment. I > >>> have been writing a couple of apps in erlang in my company. Today i > >>> was told that all my tcp connections have to go through a proxy > >>> server. I was not able to find anything in the erlang documentation > >>> related to setting the proxy server host/port on the socket options. > >>> Does erlang support such thing? > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questi...@REDACTED > >>>http://erlang.org/mailman/listinfo/erlang-questions > > >> "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" > > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questi...@REDACTED > >http://erlang.org/mailman/listinfo/erlang-questions > > "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" > > _______________________________________________ > erlang-questions mailing list > erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions From sg2342@REDACTED Fri Mar 2 17:05:10 2012 From: sg2342@REDACTED (Stefan Grundmann) Date: Fri, 2 Mar 2012 16:05:10 +0000 Subject: [erlang-questions] Erlang and SOCKS 5 In-Reply-To: <4e0afe28-bd6d-4d53-8bba-24e78d93a15e@c21g2000yqi.googlegroups.com> References: <750266d2-652b-442f-86c9-67f38feae2bd@s7g2000yqm.googlegroups.com> <8CD95C3F-63A4-4431-ADEE-30D12D631C40@rogvall.se> <3d5f9a43-07e2-42ab-a624-01fe5832d0dc@p12g2000yqe.googlegroups.com> <4e0afe28-bd6d-4d53-8bba-24e78d93a15e@c21g2000yqi.googlegroups.com> Message-ID: <20120302160510.63d8efe3@googlemail.com> On Fri, 2 Mar 2012 07:51:29 -0800 (PST) eigenfunction wrote: > If only i could find those releases? Since i am on a tight deadline, i > may rewrite part of my apps in scala. > But anyway, does anybody know how i can get my hand on release 10b? > http://www.erlang.org/download/otp_src_R10B-1a.tar.gz has socks5 code. From tony@REDACTED Fri Mar 2 17:19:39 2012 From: tony@REDACTED (Tony Rogvall) Date: Fri, 2 Mar 2012 17:19:39 +0100 Subject: [erlang-questions] Erlang and SOCKS 5 In-Reply-To: <4e0afe28-bd6d-4d53-8bba-24e78d93a15e@c21g2000yqi.googlegroups.com> References: <750266d2-652b-442f-86c9-67f38feae2bd@s7g2000yqm.googlegroups.com> <8CD95C3F-63A4-4431-ADEE-30D12D631C40@rogvall.se> <3d5f9a43-07e2-42ab-a624-01fe5832d0dc@p12g2000yqe.googlegroups.com> <4e0afe28-bd6d-4d53-8bba-24e78d93a15e@c21g2000yqi.googlegroups.com> Message-ID: <32350A8F-87FF-4540-916D-55C400B6DB4B@rogvall.se> On 2 mar 2012, at 16:51, eigenfunction wrote: > If only i could find those releases? Since i am on a tight deadline, i > may rewrite part of my apps in scala. Wow, that IS a tight deadline! or a very limited code base. ;) > But anyway, does anybody know how i can get my hand on release 10b? > http://www.erlang.org/download/ /Tony > On 2 Mrz., 15:43, Tony Rogvall wrote: >> Maybe you can pick up the socks modules from the old release >> and port them ? >> >> /Tony >> >> On 2 mar 2012, at 15:41, eigenfunction wrote: >> >> >> >> >> >> >> >> >> >>> Thank you for the quick answer. Anyone has any idea or workaround >>> about sending tcp packets through a proxy with authentication in >>> erlang? >>> It will be bad for me if i had to rewrite things in java only because >>> of that. >> >>> On 2 Mrz., 15:35, Tony Rogvall wrote: >>>> They are old and gone.... >> >>>> http://www.trapexit.org/forum/viewtopic.php?t=3957&sid=1d0af3385415e7... >> >>>> :-) >> >>>> /Tony >> >>>> On 2 mar 2012, at 15:28, eigenfunction wrote: >> >>>>> Sorry if you receive this mail a second time, i could not see the >>>>> first one after i sent it. >>>>> Like i said in my first mail, i am a little bit stuck at the moment. I >>>>> have been writing a couple of apps in erlang in my company. Today i >>>>> was told that all my tcp connections have to go through a proxy >>>>> server. I was not able to find anything in the erlang documentation >>>>> related to setting the proxy server host/port on the socket options. >>>>> Does erlang support such thing? >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questi...@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >> >>>> "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" >> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questi...@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From emeka_1978@REDACTED Fri Mar 2 17:24:43 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Fri, 2 Mar 2012 08:24:43 -0800 (PST) Subject: [erlang-questions] Erlang and SOCKS 5 In-Reply-To: <32350A8F-87FF-4540-916D-55C400B6DB4B@rogvall.se> References: <750266d2-652b-442f-86c9-67f38feae2bd@s7g2000yqm.googlegroups.com> <8CD95C3F-63A4-4431-ADEE-30D12D631C40@rogvall.se> <3d5f9a43-07e2-42ab-a624-01fe5832d0dc@p12g2000yqe.googlegroups.com> <4e0afe28-bd6d-4d53-8bba-24e78d93a15e@c21g2000yqi.googlegroups.com> <32350A8F-87FF-4540-916D-55C400B6DB4B@rogvall.se> Message-ID: Thanks. It's funny how i could not google my way to old releases. How could i have known that it is download instead of download.html? ;-) On 2 Mrz., 17:19, Tony Rogvall wrote: > On 2 mar 2012, at 16:51, eigenfunction wrote: > > > If only i could find those releases? Since i am on a tight deadline, i > > may rewrite part of my apps in scala. > > Wow, that IS a tight deadline! or a very limited code base. ;) > > > But anyway, does anybody know how i can get my hand on release 10b? > > http://www.erlang.org/download/ > > /Tony > > > > > > > > > > > On 2 Mrz., 15:43, Tony Rogvall wrote: > >> Maybe you can pick up the socks modules from the old release > >> and port them ? > > >> /Tony > > >> On 2 mar 2012, at 15:41, eigenfunction wrote: > > >>> Thank you for the quick answer. Anyone has any idea or workaround > >>> about sending tcp packets through a proxy with authentication in > >>> erlang? > >>> It will be bad for me if i had to rewrite things in java only because > >>> of that. > > >>> On 2 Mrz., 15:35, Tony Rogvall wrote: > >>>> They are old and gone.... > > >>>>http://www.trapexit.org/forum/viewtopic.php?t=3957&sid=1d0af3385415e7... > > >>>> :-) > > >>>> /Tony > > >>>> On 2 mar 2012, at 15:28, eigenfunction wrote: > > >>>>> Sorry if you receive this mail a second time, i could not see the > >>>>> first one after i sent it. > >>>>> Like i said in my first mail, i am a little bit stuck at the moment. I > >>>>> have been writing a couple of apps in erlang in my company. Today i > >>>>> was told that all my tcp connections have to go through a proxy > >>>>> server. I was not able to find anything in the erlang documentation > >>>>> related to setting the proxy server host/port on the socket options. > >>>>> Does erlang support such thing? > >>>>> _______________________________________________ > >>>>> erlang-questions mailing list > >>>>> erlang-questi...@REDACTED > >>>>>http://erlang.org/mailman/listinfo/erlang-questions > > >>>> "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" > > >>>> _______________________________________________ > >>>> erlang-questions mailing list > >>>> erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questi...@REDACTED > >>>http://erlang.org/mailman/listinfo/erlang-questions > > >> "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" > > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questi...@REDACTED > >http://erlang.org/mailman/listinfo/erlang-questions > > "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" > > _______________________________________________ > erlang-questions mailing list > erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions From rtrlists@REDACTED Fri Mar 2 17:59:36 2012 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 2 Mar 2012 16:59:36 +0000 Subject: [erlang-questions] Hot erlang Re: Frying pan bug In-Reply-To: References: Message-ID: The os_mon modules would appear to be the natural starting point. http://www.erlang.org/doc/apps/os_mon/index.html On Fri, Mar 2, 2012 at 3:10 PM, Hendrik Visage wrote: > On Fri, Mar 2, 2012 at 4:42 PM, David Goehrig wrote: > > > Would it be possible for Erlang's runtime to set a threshold, say > looking at the thermal sensor on the board and error out if that "guard" is > reached. You could do the same for disk I/O, network, or any other > saturation condition. > > I would rather prefer the OS to send a SIGimInHeat signal to the Erlang VM > :) > > On the serious side: I see that as a function of the OS and hardware > to throttle the CPU and processes whenever overheating situations > occur. erlang VMs should rather track the usage times of it's > processes based on simple threshold values, but isn't that getting to > the point where we have the VM baby sitting bad coders?? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Fri Mar 2 18:39:23 2012 From: comptekki@REDACTED (Wes James) Date: Fri, 2 Mar 2012 10:39:23 -0700 Subject: [erlang-questions] Raspberry-Pi In-Reply-To: References: <064E04AC-7408-4934-91A6-BA0D6D495592@gmail.com> Message-ID: On Fri, Mar 2, 2012 at 4:13 AM, Michael Turner wrote: >> ... a $35 Erlang machine with Ethernet ... > > --if based on Raspberry-Pi, would be called "Raspberry-e", wouldn't it? How about Raspberry-PiE > > -michael turner From sverker@REDACTED Fri Mar 2 20:09:12 2012 From: sverker@REDACTED (Sverker Eriksson) Date: Fri, 2 Mar 2012 20:09:12 +0100 Subject: [erlang-questions] has anyone cross compiled HiPE? In-Reply-To: <20120302112137.GA14059@corelatus.se> References: <20120220092424.GA3161@corelatus.se> <20120302112137.GA14059@corelatus.se> Message-ID: <4F511AD8.4020801@erix.ericsson.se> In R15 there is a build flag to hipe called XCOMP. It is a _very_ limited feature for cross compiling. All that it does is to make sure that all values in hipe_literals.hrl are generated as hard coded and totally independent of the emulator the compiler is running on. I have only used this to "cross compile" between different types of emulator running on the same architecture. cd lib/hipe make XCOMP=yes FLAVOR=smp TYPE=debug This will build a hipe compiler to produce code that can be run a debug compiled emulator without having to run hipe on that same (slow) debug emulator. /Sverker, Erlang/OTP From dmkolesnikov@REDACTED Fri Mar 2 20:35:46 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Fri, 2 Mar 2012 21:35:46 +0200 Subject: [erlang-questions] Erlang and SOCKS 5 In-Reply-To: References: <750266d2-652b-442f-86c9-67f38feae2bd@s7g2000yqm.googlegroups.com> <8CD95C3F-63A4-4431-ADEE-30D12D631C40@rogvall.se> <3d5f9a43-07e2-42ab-a624-01fe5832d0dc@p12g2000yqe.googlegroups.com> <4e0afe28-bd6d-4d53-8bba-24e78d93a15e@c21g2000yqi.googlegroups.com> <32350A8F-87FF-4540-916D-55C400B6DB4B@rogvall.se> Message-ID: <-6187279760371761397@unknownmsgid> Hi, You can use cntlm http://cntlm.sourceforge.net/ As a local proxy against socks Best Regards, Dmitry >-|-|-*> On 2.3.2012, at 18.24, eigenfunction wrote: > Thanks. > It's funny how i could not google my way to old releases. > How could i have known that it is download instead of download.html? > ;-) > > On 2 Mrz., 17:19, Tony Rogvall wrote: >> On 2 mar 2012, at 16:51, eigenfunction wrote: >> >>> If only i could find those releases? Since i am on a tight deadline, i >>> may rewrite part of my apps in scala. >> >> Wow, that IS a tight deadline! or a very limited code base. ;) >> >>> But anyway, does anybody know how i can get my hand on release 10b? >> >> http://www.erlang.org/download/ >> >> /Tony >> >> >> >> >> >> >> >> >> >>> On 2 Mrz., 15:43, Tony Rogvall wrote: >>>> Maybe you can pick up the socks modules from the old release >>>> and port them ? >> >>>> /Tony >> >>>> On 2 mar 2012, at 15:41, eigenfunction wrote: >> >>>>> Thank you for the quick answer. Anyone has any idea or workaround >>>>> about sending tcp packets through a proxy with authentication in >>>>> erlang? >>>>> It will be bad for me if i had to rewrite things in java only because >>>>> of that. >> >>>>> On 2 Mrz., 15:35, Tony Rogvall wrote: >>>>>> They are old and gone.... >> >>>>>> http://www.trapexit.org/forum/viewtopic.php?t=3957&sid=1d0af3385415e7... >> >>>>>> :-) >> >>>>>> /Tony >> >>>>>> On 2 mar 2012, at 15:28, eigenfunction wrote: >> >>>>>>> Sorry if you receive this mail a second time, i could not see the >>>>>>> first one after i sent it. >>>>>>> Like i said in my first mail, i am a little bit stuck at the moment. I >>>>>>> have been writing a couple of apps in erlang in my company. Today i >>>>>>> was told that all my tcp connections have to go through a proxy >>>>>>> server. I was not able to find anything in the erlang documentation >>>>>>> related to setting the proxy server host/port on the socket options. >>>>>>> Does erlang support such thing? >>>>>>> _______________________________________________ >>>>>>> erlang-questions mailing list >>>>>>> erlang-questi...@REDACTED >>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >> >>>>>> "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" >> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questi...@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >> >>>> "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" >> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questi...@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jayson.barley@REDACTED Fri Mar 2 20:53:57 2012 From: jayson.barley@REDACTED (Jayson Barley) Date: Fri, 2 Mar 2012 11:53:57 -0800 Subject: [erlang-questions] TDD, Erlang and Emacs In-Reply-To: References: Message-ID: I found the answer I was looking for quite by accident. The Erlang guys, major kudos to you guys for this, ship out an Emacs erlang-eunit mode. Here are the default bindings: There are some other nice functions available that aren't bound, but for the majority of the cases we are going to run into, these will do it. I can't believe I haven't seen this before. Again, I want to thank you guys for including something this awesome and to Klas Johansson for putting in the work to get it done. On Thu, Mar 1, 2012 at 3:24 PM, Jayson Barley wrote: > I am wondering if anyone has any recommendations for a smoother TDD > experience with Erlang and Emacs. I am finding the process to be less than > optimal. The way I am doing it now is that I have 3 windows open. > > 1. In my test module I write my test > 2. Switch to the Erlang buffer > 3. Run my tests, see that it fails > 4. Switch to the module under test buffer, write code to get the test > to pass > 5. Switch back to the Erlang buffer, run my tests again > 6. Switch back to the test buffer and repeat. > > Ideally I would like to have 2 windows open, 1 for the module under test > and the other for the test window. From the module under test buffer or the > test module I should be able to run the tests, see that they pass or fail > and then move on.I have been considering writing an plugin to handle this > functionality for me. Before I go that route I want to make sure I am not > duplicating what someone else has already done. > > Does anyone have any recommendations for how to do this better? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sat Mar 3 01:32:36 2012 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 02 Mar 2012 16:32:36 -0800 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: <20120302064801.GA3620@k2r.org> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <20120302064801.GA3620@k2r.org> Message-ID: <4F5166A4.1050902@gmail.com> On 03/01/2012 10:48 PM, Kenji Rikitake wrote: > FYI: > A crude implementation in pure Erlang is available at > https://github.com/jj1bdx/sfmt-erlang/blob/master/src/random_wh06.erl > For the 2006 Wichmann-Hill RNG which Richard mentioned below. > > ++> Richard O'Keefe [2012-03-02 15:59:58 +1300]: > [snip] > >> Wichmann & Hill have since published a four-generator algorithm, see >> Wichmann and Hill have a new generator with a longer period. >> [2] B. A. Wichmann, I. D. Hill, >> Generating good pseudo-random numbers, >> Computational Statistics & Data Analysis, 51, 1614-1622 (2006). > BTW > >> In fact generating full-precision random doubles is something RNG writers often neglect. > Indeed true. Providing enough bits for the mantissa digits is required. > >> As for generating random integers in the range [0,N) where N is a bignum, I would love >> to be pointed to a paper on the subject. > Me too - so far all I see is for fixed-length integers. > > Kenji Rikitake It seems like we could use the Wichmann and Hill algorithm, in a way that utilizes big-integers. I noticed that the current usage of floating point should limit the uniform distribution that would otherwise be available, so the big-integers implementation should be superior. There just would be a small change required to the code, summarized below: % original usage of floating point %R = (C1 * 0.0000000004656613022697297188506231646486) + % (C2 * 0.0000000004656613100759859932486569933169) + % (C3 * 0.0000000004656613360968421314794009471615) + % (C4 * 0.0000000004656614011489951998100056779817), % transformation into big-integers without common factors %R = ((C1 * 4656613022697297188506231646486) + % (C2 * 4656613100759859932486569933169) + % (C3 * 4656613360968421314794009471615) + % (C4 * 4656614011489951998100056779817)) / 1.0e40, % usage as big-integers I = ((C1 * 100974236070208004586355635975308882395008510460084959275529243190275451744887731102642177895) + (C2 * 100974234377495411077165331191698869941147322702626405281612091220218253091371910599190635130) + (C3 * 100974228735120099379864315246347981874512822184904954644794292108235670371042659321702493478) + (C4 * 100974214629181820136611775382944281323036229785128918455846504909124232305107255133165006410)) rem 470197942641441751328779943957359348820645151704843242145900685403753713168019478536992537786552036455690641240694763626970, The numbers do get a bit big and ugly, but it is just transforming R into a fraction, which represents a single number within the large distribution provided by the algorithm (i.e., a single I within [0..470197942641441751328779943957359348820645151704843242145900685403753713168019478536992537786552036455690641240694763626970)). This is basically the same as the (R - trunc(R)), just that it avoids the division. So, if I is uniformly distributed with this large value of N, it should also be uniformly distributed within a subset of the range. So, all we should need to do is take the remainder based on the N we supply to the module, so: uniform(N) when is_integer(N), N >= 1, N < 470197942641441751328779943957359348820645151704843242145900685403753713168019478536992537786552036455690641240694763626970 -> (uniform() rem N) + 1. If you use this implementation, then it would provide 408 bits of randomness, uniformly distributed. I know you mentioned the period of this algorithm as "~ 2^120 (~ 10^36)" in a previous post, so it doesn't compare to a Mersenne Twister solution. However, this provides a decent way of avoiding a port_driver (crypto) or NIF call, which would have poorer performance (like crypto:rand_uniform/2), and it provides normal fault-tolerance (since it is all Erlang code). It would be nice if small N, i.e., less than 1000000 could used erlang:now/0 instead, since it is quicker than even random:uniform/1, in my testing, and it appears to be more uniform. However, I assume it would become less uniform with usage on timers, so that is probably a bad case to add to this type of a random module. I added the modified module to my speed test, just to look at its performance here: https://github.com/okeuday/erlbench/blob/master/src/random_wh06.erl The speed test gives: TEST pseudo_randomness N == 10000 (10 runs) crypto:rand_uniform/ get: 68999.4 ?s ( 26.0) erlang:now/0 get: 2658.6 ?s ( 1.0) random:uniform/1 get: 7011.5 ?s ( 2.6) random:uniform_wh06/ get: 21636.6 ?s ( 8.1) That is with the random N == 10, performed 10000 times, during 10 runs that are averaged with Erlang R14B04 (non-HiPE). Tell me if you think my modifications to random_wh06 are erroneous in some way. Thanks, Michael From mjtruog@REDACTED Sat Mar 3 03:53:17 2012 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 02 Mar 2012 18:53:17 -0800 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: <4F5166A4.1050902@gmail.com> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <20120302064801.GA3620@k2r.org> <4F5166A4.1050902@gmail.com> Message-ID: <4F51879D.9040806@gmail.com> On 03/02/2012 04:32 PM, Michael Truog wrote: > On 03/01/2012 10:48 PM, Kenji Rikitake wrote: >> FYI: >> A crude implementation in pure Erlang is available at >> https://github.com/jj1bdx/sfmt-erlang/blob/master/src/random_wh06.erl >> For the 2006 Wichmann-Hill RNG which Richard mentioned below. >> >> ++> Richard O'Keefe [2012-03-02 15:59:58 +1300]: >> [snip] >> >>> Wichmann & Hill have since published a four-generator algorithm, see >>> Wichmann and Hill have a new generator with a longer period. >>> [2] B. A. Wichmann, I. D. Hill, >>> Generating good pseudo-random numbers, >>> Computational Statistics & Data Analysis, 51, 1614-1622 (2006). >> BTW >> >>> In fact generating full-precision random doubles is something RNG writers often neglect. >> Indeed true. Providing enough bits for the mantissa digits is required. >> >>> As for generating random integers in the range [0,N) where N is a bignum, I would love >>> to be pointed to a paper on the subject. >> Me too - so far all I see is for fixed-length integers. >> >> Kenji Rikitake > It seems like we could use the Wichmann and Hill algorithm, in a way that utilizes big-integers. I noticed that the current usage of floating point should limit the uniform distribution that would otherwise be available, so the big-integers implementation should be superior. There just would be a small change required to the code, summarized below: > > % original usage of floating point > %R = (C1 * 0.0000000004656613022697297188506231646486) + > % (C2 * 0.0000000004656613100759859932486569933169) + > % (C3 * 0.0000000004656613360968421314794009471615) + > % (C4 * 0.0000000004656614011489951998100056779817), > > % transformation into big-integers without common factors > %R = ((C1 * 4656613022697297188506231646486) + > % (C2 * 4656613100759859932486569933169) + > % (C3 * 4656613360968421314794009471615) + > % (C4 * 4656614011489951998100056779817)) / 1.0e40, > > % usage as big-integers > I = ((C1 * 100974236070208004586355635975308882395008510460084959275529243190275451744887731102642177895) + > (C2 * 100974234377495411077165331191698869941147322702626405281612091220218253091371910599190635130) + > (C3 * 100974228735120099379864315246347981874512822184904954644794292108235670371042659321702493478) + > (C4 * 100974214629181820136611775382944281323036229785128918455846504909124232305107255133165006410)) rem 470197942641441751328779943957359348820645151704843242145900685403753713168019478536992537786552036455690641240694763626970, > > The numbers do get a bit big and ugly, but it is just transforming R into a fraction, which represents a single number within the large distribution provided by the algorithm (i.e., a single I within [0..470197942641441751328779943957359348820645151704843242145900685403753713168019478536992537786552036455690641240694763626970)). This is basically the same as the (R - trunc(R)), just that it avoids the division. So, if I is uniformly distributed with this large value of N, it should also be uniformly distributed within a subset of the range. So, all we should need to do is take the remainder based on the N we supply to the module, so: > > uniform(N) when is_integer(N), N >= 1, N < 470197942641441751328779943957359348820645151704843242145900685403753713168019478536992537786552036455690641240694763626970 -> > (uniform() rem N) + 1. > > If you use this implementation, then it would provide 408 bits of randomness, uniformly distributed. I know you mentioned the period of this algorithm as "~ 2^120 (~ 10^36)" in a previous post, so it doesn't compare to a Mersenne Twister solution. However, this provides a decent way of avoiding a port_driver (crypto) or NIF call, which would have poorer performance (like crypto:rand_uniform/2), and it provides normal fault-tolerance (since it is all Erlang code). > > It would be nice if small N, i.e., less than 1000000 could used erlang:now/0 instead, since it is quicker than even random:uniform/1, in my testing, and it appears to be more uniform. However, I assume it would become less uniform with usage on timers, so that is probably a bad case to add to this type of a random module. > > I added the modified module to my speed test, just to look at its performance here: > https://github.com/okeuday/erlbench/blob/master/src/random_wh06.erl > > The speed test gives: > TEST pseudo_randomness > N == 10000 (10 runs) > crypto:rand_uniform/ get: 68999.4 ?s ( 26.0) > erlang:now/0 get: 2658.6 ?s ( 1.0) > random:uniform/1 get: 7011.5 ?s ( 2.6) > random:uniform_wh06/ get: 21636.6 ?s ( 8.1) > > That is with the random N == 10, performed 10000 times, during 10 runs that are averaged with Erlang R14B04 (non-HiPE). I am sorry, I confused that algorithm with the one within the random module. The random module algorithm required more work to use as a fraction (it was late on a Friday, is my only excuse), however, this one is simpler to treat as a fraction. The algorithm changes as shown below: % original usage of floating point %R = (C1 * 0.0000000004656613022697297188506231646486) + % (C2 * 0.0000000004656613100759859932486569933169) + % (C3 * 0.0000000004656613360968421314794009471615) + % (C4 * 0.0000000004656614011489951998100056779817), % transformation into big-integers without common factors %R = ((C1 * 4656613022697297188506231646486) + % (C2 * 4656613100759859932486569933169) + % (C3 * 4656613360968421314794009471615) + % (C4 * 4656614011489951998100056779817)) / 1.0e40, % usage as big-integers I = ((C1 * 4656613022697297188506231646486) + (C2 * 4656613100759859932486569933169) + (C3 * 4656613360968421314794009471615) + (C4 * 4656614011489951998100056779817)) rem 10000000000000000000000000000000000000000, This should make more sense. However, this means that this algorithm actually provides a maximum of 133 random bits: 1> (math:log(10000000000000000000000000000000000000000) / math:log(2)) + 1. 133.8771237954945 The same explanation/idea is at work though, so that isn't wrong. It could be applied to the random module in the same way to avoid using floating point. The speed test is pretty much the same, since only the constants changed: TEST pseudo_randomness N == 10000 (10 runs) crypto:rand_uniform/ get: 67394.1 ?s ( 25.5) erlang:now/0 get: 2642.8 ?s ( 1.0) random:uniform/1 get: 7006.6 ?s ( 2.7) random:uniform_wh06/ get: 17741.2 ?s ( 6.7) However, please tell me if you think I have a mistake in there still. Updated the link: https://github.com/okeuday/erlbench/blob/master/src/random_wh06.erl Thanks, Michael From ok@REDACTED Sat Mar 3 04:49:18 2012 From: ok@REDACTED (Richard O'Keefe) Date: Sat, 3 Mar 2012 16:49:18 +1300 Subject: [erlang-questions] Frying pan bug In-Reply-To: References: Message-ID: On 3/03/2012, at 12:42 AM, Joe Armstrong wrote: > -module(bug). > -compile(export_all). > > bug() -> p(0.3, 4, 11). > > c(N, M) -> fac(M) div (fac(N)*fac(M-N)). > > fac(0) -> 1; > fac(N) -> N*fac(N-1). > > p(P, M, N) -> 1.0 - math:pow(1-P, N-M)*c(N, M). this calls fac(-7), which calls fac(-8), which calls fac(-9), ... Write fac(N) when N > 1 -> N * fac(N-1); fac(N) when N >= 0 -> 1. From ok@REDACTED Sat Mar 3 05:07:33 2012 From: ok@REDACTED (Richard O'Keefe) Date: Sat, 3 Mar 2012 17:07:33 +1300 Subject: [erlang-questions] Frying pan bug In-Reply-To: References: Message-ID: <2BE68D54-2705-4E02-BE08-B7F86259D6A8@cs.otago.ac.nz> On 3/03/2012, at 3:42 AM, David Goehrig wrote: > I'm curious, I know that people are "right" when they say fix validate the inputs to factorial, but there are equations far less well known with similar degenerate cases where writing guards are inexpressible due to there being an infinite number of degenerate points. Not all curves are continuous even in the real world. In fact the negative integers are the only points on the real line where factorial is undefined. There are, however, an infinite number of them, and that doesn't mean that guards are inexpressible. > Would it be possible for Erlang's runtime to set a threshold, say looking at the thermal sensor on the board and error out if that "guard" is reached. There is no guarantee - that the 'board' *has* a thermal sensor (if there is a thermal sensor on the Sun-Blade 100 I still use, that can be read by user-level code, I would be interested to hear about it) - that the thermal sensor can be read by user code - that an infinite loop will cause the temperature to rise appreciably - that some component of the system will not detect a rise and respond by reducing clock speed so the rise stops - that a temperature rise is caused by *this* process rather than some other process - that the readings are reliable. I just tried it on a Mac. "CPU A Temperature Diode" was initially at 32C. Running fac(-1), it rose over a couple of minutes to 38C, and then fell back to 34C, where it remained. Bearing in mind that the "Northbridge Chip" was at 59C and remained there, that doesn't sound like the kind of huge rise that you would expect to trigger alarms. Come to think of it, a 2C rise doesn't sound much like a frying pan either. Considering how thin the Mac is, and where it is placed, I'd be surprised if I couldn't get a similar effect by turning on the office heater. I mention the possible unreliability of the readings because the same software can monitor things like power consumption, and those results are completely crazy. (Wrong signs, for example...) > You could do the same for disk I/O, network, or any other saturation condition. I have programs that saturate I/O without anything being wrong. > This has been recently on my mind as we just did a poorman's thermal chamber test, and I've been writing a lot of Erlang that monitors the behavior of other apps under environmental stress, but have yet to solve the "how does the monitor that monitors itself know when to throw up the white flag and surrender?" I've been reading about self-stablising systems lately, and the rock bottom answer seems to be "watchdog timers". A watchdog timer would certainly have caught this. From ok@REDACTED Sat Mar 3 06:02:04 2012 From: ok@REDACTED (Richard O'Keefe) Date: Sat, 3 Mar 2012 18:02:04 +1300 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: <4F5166A4.1050902@gmail.com> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <20120302064801.GA3620@k2r.org> <4F5166A4.1050902@gmail.com> Message-ID: <209543D1-42F8-44CB-B1EB-6DA3F71F9209@cs.otago.ac.nz> On 3/03/2012, at 1:32 PM, Michael Truog wrote: > If you use this implementation, then it would provide 408 bits of randomness, uniformly distributed. I don't see how it can. Just because all the numbers are in a given range does not mean that all the numbers in that range are possible. The Wichmann-Hill generator has a state with 4 31-bit numbers, meaning that you can't possibly get more than 124 bits out of it. If you really get more, you have a different algorithm. Using 64-bit integer arithmetic, the algorithm looks like a = (a * 11600LL) % 2147483579; b = (b * 47003LL) % 2147483543; c = (c * 23000LL) % 2147483423; d = (d * 33000LL) % 2147483123; w = a/2147483579.0 + b/2147483543.0 + c/2147483423.0 + d/2147483123.0; if (w >= 2.0) w -= 2.0; if (w >= 1.0) w -= 1.0; return w; From mjtruog@REDACTED Sat Mar 3 06:48:34 2012 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 02 Mar 2012 21:48:34 -0800 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: <209543D1-42F8-44CB-B1EB-6DA3F71F9209@cs.otago.ac.nz> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <20120302064801.GA3620@k2r.org> <4F5166A4.1050902@gmail.com> <209543D1-42F8-44CB-B1EB-6DA3F71F9209@cs.otago.ac.nz> Message-ID: <4F51B0B2.4090606@gmail.com> On 03/02/2012 09:02 PM, Richard O'Keefe wrote: > On 3/03/2012, at 1:32 PM, Michael Truog wrote: >> If you use this implementation, then it would provide 408 bits of randomness, uniformly distributed. > I don't see how it can. Just because all the numbers are in a given range does not mean that > all the numbers in that range are possible. The Wichmann-Hill generator has a state with > 4 31-bit numbers, meaning that you can't possibly get more than 124 bits out of it. If you > really get more, you have a different algorithm. > > Using 64-bit integer arithmetic, the algorithm looks like > > a = (a * 11600LL) % 2147483579; > b = (b * 47003LL) % 2147483543; > c = (c * 23000LL) % 2147483423; > d = (d * 33000LL) % 2147483123; > w = a/2147483579.0 + b/2147483543.0 > + c/2147483423.0 + d/2147483123.0; > if (w >= 2.0) w -= 2.0; > if (w >= 1.0) w -= 1.0; > return w; I agree. I think the random_wh06 module had precision problems with the floating point values it was using for R. If I modify the module to fit the 64bit implementation, it does match the 124 bit maximum with the range [0...21267638781707063560975648195455661513) for the uniform distribution (coming from "w" in the 64bit version, without division, or "I" in the random_wh06 module), which is then input for the modulus N to have the algorithm use only a subset of the total range for a uniform distribution. The Erlang code equivalent is: B1 = (11600 * A1) rem 2147483579, B2 = (47003 * A2) rem 2147483543, B3 = (23000 * A3) rem 2147483423, B4 = (33000 * A4) rem 2147483123, put(random_wh06_seed, {B1, B2, B3, B4}), I = ((B1 * 9903516371291919229607132747) + (B2 * 9903516537312557910938853791) + (B3 * 9903517090714727049595319831) + (B4 * 9903518474220420479167438931)) rem 21267638781707063560975648195455661513, I updated the implementation with this, to correct the problem: https://github.com/okeuday/erlbench/blob/master/src/random_wh06.erl Thank you for finding the 64bit C implementation. Please tell me if you think my approach with the recent changes is erroneous. Thanks, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sat Mar 3 15:37:23 2012 From: g@REDACTED (Garrett Smith) Date: Sat, 3 Mar 2012 08:37:23 -0600 Subject: [erlang-questions] Frying pan bug In-Reply-To: <2BE68D54-2705-4E02-BE08-B7F86259D6A8@cs.otago.ac.nz> References: <2BE68D54-2705-4E02-BE08-B7F86259D6A8@cs.otago.ac.nz> Message-ID: On Fri, Mar 2, 2012 at 10:07 PM, Richard O'Keefe wrote: > > I've been reading about self-stablising systems lately, and the rock bottom answer seems > to be "watchdog timers". ?A watchdog timer would certainly have caught this. Richard, are there any publications you would recommend along this line? Garrett From dave@REDACTED Sat Mar 3 16:54:50 2012 From: dave@REDACTED (David Goehrig) Date: Sat, 3 Mar 2012 10:54:50 -0500 Subject: [erlang-questions] Frying pan bug In-Reply-To: <2BE68D54-2705-4E02-BE08-B7F86259D6A8@cs.otago.ac.nz> References: <2BE68D54-2705-4E02-BE08-B7F86259D6A8@cs.otago.ac.nz> Message-ID: On Fri, Mar 2, 2012 at 11:07 PM, Richard O'Keefe wrote: > > In fact the negative integers are the only points on the real line where > factorial is > undefined. There are, however, an infinite number of them, and that > doesn't mean that > guards are inexpressible. > I was thinking more along the lines of topologies that look something like: http://www.wolframalpha.com/input/?i=1+%2F%28+x+mod+y%29 which can become very expensive to sample with a Monte Carlo method or more complex systems like: http://www.wolframalpha.com/input/?i=Duffing+Differential+Equation which in most forms can only be approximated through sampling. There's a lot of interesting surfaces which while attempting to sample to gain resolution end up degenerating into the run-away case, but run away can be either positive or negative, a guard may not help, especially when the topology is driven by a heuristic. (i.e. I don't know the equation a head of time, and I can't generate guards on the fly). > I just tried it on a Mac. "CPU A Temperature Diode" was initially at 32C. > Running fac(-1), it rose over a couple of minutes to 38C, and then fell > back to > 34C, where it remained. > I have a Mac with a busted thermal sensor and it will continue to heat up until the whole thing locks. Your test actually proved you have a working thermal sensor and that the OS was safely limiting how quickly you were wasting CPU and power. The devices we're building do have termal sensors so I can guarantee that they exist, I can also guarantee that it is available in user mode (I control the kernel as well), but what interests me is Erlang has its own scheduler and threading model on top of the OS's, which means relying on Linux to "tune" performance this way is like taking a big stick and beating all the Erlang processes for one bad apple. All of our server back off CPU frequency, and with designs like bigLITTLE from ARM becoming more common place. What do you do when the outside air temperature is 50C, and if your CPU steps below a certain power threshold it can no longer keep up with throughput to ensure real time response. > I have programs that saturate I/O without anything being wrong. I have programs that if they saturate I/O it will cause a failure as it will no longer be able to guarantee real time response. Having tooling in Erlang so that one can identify a run-away I/O process would be generally useful. The wrongness of a program is a matter of how it fits it's purpose. You might be able to tolerate I/O saturation. In my case, our customers get very angry (and litigious). > I've been reading about self-stablising systems lately, and the rock > bottom answer seems > to be "watchdog timers". A watchdog timer would certainly have caught > this. > Watchdog timers help, but you typically need more instrumentation to make reasonable decisions. In one system, I have a watchdog checking each long running process every second, and a secondary process measuring the average message volume processed by each entity. We also have each process announce memory pressure, GC state, I/O totals, and CPU load on a per OS process level, which are used to determine how to dynamically route data through the system. Without sufficient data, a watchdog will not make the right decision. Basically, what I'd be interesting in seeing is power, heat, time, cycles, and I/O traffic accounting on a per-Erlang process level, so that the supervisors could do a better job at managing a system under stress. -- -=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From fhunleth@REDACTED Sat Mar 3 22:32:02 2012 From: fhunleth@REDACTED (Frank Hunleth) Date: Sat, 3 Mar 2012 16:32:02 -0500 Subject: [erlang-questions] Erlang development workflow Message-ID: I was wondering if anyone could share ideas or experiences on their Erlang workflow especially for when the development machine is not the best test machine. My situation is that I am running Erlang on an embedded ARM that has Ethernet and decent DRAM and local storage. I can do some development on my PC, but there's special hardware attached to the ARM that my code needs to control sometimes. It would really be nice if I could have an efficient workflow that lets me develop on the PC and run on the ARM with the minimum number of manual steps between editing and testing. I'm coming to Erlang from an embedded C/C++ background. At the moment, my process is not much better than what I'm used to. It seems like there are many potential options, and I'm not sure what the best route is. Thanks, Frank From watson.timothy@REDACTED Sun Mar 4 01:19:25 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 4 Mar 2012 00:19:25 +0000 Subject: [erlang-questions] Erlang development workflow In-Reply-To: References: Message-ID: <05865BE6-FFC3-4BF9-A6FE-BFAF7F64A40F@gmail.com> I suspect you have a few choices about how you might do this. In terms of getting the latest 'code' onto the ARM, you can either 1. synchronise the file systems 2. synchronise the loaded beam code on two emulators How do you get your code onto the ARM's local storage currently? There is a 'reloader' module floating around the internet (especially on github) which takes any recent file system changes and does a code purge and reload (hence the name) on updated modules. If you find a way to synchronise local file system changes so they're reflected on the ARM, the reloader module should do the rest. The way it works IIRC is that it tracks either the beam files (in ebin) or the source files (in src) - I can't remember which - and when something changes, it reloads. You could also use common test to run your test suites on the ARM node using ct_master, although I have no idea how the file system requirements will work in this scenario. Another option that might suit you would be to keep a test 'server' running continuously on the ARM and control this from your PC using the ethernet connection. There are several mechanisms that could work well here, the simplest of which is to ship the code updates to the ARM over the ethernet connection (e.g., pass the updated beam code to code:load_binary using an rpc:call). In any of these cases, once you've set up the synchronisation and test mechanism(s), you could easily make sure that any changes trigger either a smoke test or a full test run. I suspect many others on this list will have more ideas too, as there are quite of lot of ways to skin this particular type of cat using Erlang. Cheers, Tim On 3 Mar 2012, at 21:32, Frank Hunleth wrote: > I was wondering if anyone could share ideas or experiences on their > Erlang workflow especially for when the development machine is not the > best test machine. My situation is that I am running Erlang on an > embedded ARM that has Ethernet and decent DRAM and local storage. I > can do some development on my PC, but there's special hardware > attached to the ARM that my code needs to control sometimes. It would > really be nice if I could have an efficient workflow that lets me > develop on the PC and run on the ARM with the minimum number of manual > steps between editing and testing. I'm coming to Erlang from an > embedded C/C++ background. At the moment, my process is not much > better than what I'm used to. It seems like there are many potential > options, and I'm not sure what the best route is. > > Thanks, > Frank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From als@REDACTED Sun Mar 4 02:37:03 2012 From: als@REDACTED (Anthony Shipman) Date: Sun, 4 Mar 2012 12:37:03 +1100 Subject: [erlang-questions] gen_sctp throughput mystery Message-ID: <201203041237.03563.als@iinet.net.au> I've been experimenting with the SCTP implementation in R15B. There seems to be a throughput problem. I run a client and server connecting via localhost with these options -define(MAX_STREAMS, 1). Opts = [ binary, {active, once}, {sctp_nodelay, false}, {sctp_initmsg, #sctp_initmsg{ num_ostreams = ?MAX_STREAMS, max_instreams = ?MAX_STREAMS, max_attempts = 5, max_init_timeo = 1000 }} ], The client fires 100 of 9KB packets at 10ms intervals. The server sees this at the beginning before the socket buffer starts overflowing: 4-Mar-2012 12:28:18.288: received data {99,{1330,824498,288005}} transit 0.211ms speed 836.3866077068857KB/s 4-Mar-2012 12:28:18.487: received data {98,{1330,824498,299006}} transit 188.851ms speed 46.42332987712945KB/s 4-Mar-2012 12:28:18.687: received data {97,{1330,824498,310005}} transit 377.591ms speed 46.400552721301295KB/s 4-Mar-2012 12:28:18.887: received data {96,{1330,824498,321005}} transit 566.59ms speed 46.34023170115851KB/s 4-Mar-2012 12:28:19.087: received data {95,{1330,824498,332005}} transit 755.584ms speed 46.34139024170725KB/s 4-Mar-2012 12:28:19.287: received data {94,{1330,824498,343006}} transit 944.562ms speed 46.34486621095215KB/s 4-Mar-2012 12:28:19.487: received data {93,{1330,824498,354006}} transit 1133.563ms speed 46.33976830115849KB/s 4-Mar-2012 12:28:19.687: received data {92,{1330,824498,365005}} transit 1322.613ms speed 46.32864948087718KB/s 4-Mar-2012 12:28:19.887: received data {91,{1330,824498,376006}} transit 1511.665ms speed 46.32772315336436KB/s The now() value shows the packets being written to the socket at 11ms intervals on the client but they are received at strict 200ms intervals on the server. The server is just doing a simple receive loop. Is there something in SCTP that limits the rate on an SCTP stream? -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From als@REDACTED Sun Mar 4 02:43:49 2012 From: als@REDACTED (Anthony Shipman) Date: Sun, 4 Mar 2012 12:43:49 +1100 Subject: [erlang-questions] gen_sctp throughput mystery In-Reply-To: <201203041237.03563.als@iinet.net.au> References: <201203041237.03563.als@iinet.net.au> Message-ID: <201203041243.49277.als@iinet.net.au> On Sun, 4 Mar 2012 12:37:03 pm Anthony Shipman wrote: > I've been experimenting with the SCTP implementation in R15B. > There seems to be a throughput problem. I run a client and server > connecting via localhost with these options > > -define(MAX_STREAMS, 1). > > Opts = [ > binary, > {active, once}, > {sctp_nodelay, false}, > {sctp_initmsg, #sctp_initmsg{ > num_ostreams = ?MAX_STREAMS, > max_instreams = ?MAX_STREAMS, > max_attempts = 5, > max_init_timeo = 1000 > }} > ], > > > The now() value shows the packets being written to the socket at 11ms > intervals on the client but they are received at strict 200ms intervals on > the server. The server is just doing a simple receive loop. > > Is there something in SCTP that limits the rate on an SCTP stream? > I forgot to mention that the behaviour is the same for sctp_nodelay=true. -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From als@REDACTED Sun Mar 4 03:19:00 2012 From: als@REDACTED (Anthony Shipman) Date: Sun, 4 Mar 2012 13:19:00 +1100 Subject: [erlang-questions] gen_sctp throughput mystery In-Reply-To: <201203041243.49277.als@iinet.net.au> References: <201203041237.03563.als@iinet.net.au> <201203041243.49277.als@iinet.net.au> Message-ID: <201203041319.00947.als@iinet.net.au> On Sun, 4 Mar 2012 12:43:49 pm Anthony Shipman wrote: > > The now() value shows the packets being written to the socket at 11ms > > intervals on the client but they are received at strict 200ms intervals > > on the server. The server is just doing a simple receive loop. > > > > Is there something in SCTP that limits the rate on an SCTP stream? > > I forgot to mention that the behaviour is the same for sctp_nodelay=true. > Replying to my own question. It's the net.sctp.sack_timeout = 200 kernel parameter. But I don't understand why it should limit the server to 5 packets per second over a localhost link. -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From jwatte@REDACTED Sun Mar 4 08:33:36 2012 From: jwatte@REDACTED (Jon Watte) Date: Sat, 3 Mar 2012 23:33:36 -0800 Subject: [erlang-questions] Erlang development workflow In-Reply-To: References: Message-ID: Does the target machine support ssh/rsync? If so, you could have your make file just deploy to the machine after a successful compile. We actually rely 100% on automated unit and functional tests. We mercilessly mock any external dependencies (such as production databases) in development. Our deployment pushes to production machines that are similar-ish to the development machines, but a cluster of many more nodes. The code actually hits end users (a hundred thousand at a time!) automatically at some point after I check in and tests pass. Sincerely, jw Sincerely, Jon Watte -- "I pledge allegiance to the flag of the United States of America, and to the republic for which it stands, one nation indivisible, with liberty and justice for all." ~ Adopted by U.S. Congress, June 22, 1942 On Sat, Mar 3, 2012 at 1:32 PM, Frank Hunleth wrote: > I was wondering if anyone could share ideas or experiences on their > Erlang workflow especially for when the development machine is not the > best test machine. My situation is that I am running Erlang on an > embedded ARM that has Ethernet and decent DRAM and local storage. I > can do some development on my PC, but there's special hardware > attached to the ARM that my code needs to control sometimes. It would > really be nice if I could have an efficient workflow that lets me > develop on the PC and run on the ARM with the minimum number of manual > steps between editing and testing. I'm coming to Erlang from an > embedded C/C++ background. At the moment, my process is not much > better than what I'm used to. It seems like there are many potential > options, and I'm not sure what the best route is. > > Thanks, > Frank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Sun Mar 4 11:07:38 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Sun, 4 Mar 2012 11:07:38 +0100 Subject: [erlang-questions] TDD, Erlang and Emacs In-Reply-To: <669ACA5C-B6D2-485C-8BCF-99F8E47CB575@klarna.com> References: <669ACA5C-B6D2-485C-8BCF-99F8E47CB575@klarna.com> Message-ID: On Fri, Mar 2, 2012 at 08:34, Uwe Dauernheim wrote: > Sounds great! Where can I find that Emacs update? Is it in the OTP github pu branch? > /Uwe Hi Uwe I haven't considered adding this features into erlang-mode itself. At the moment this rebar-aware compilation mechanism can be found in Emacs Prelude package [1], which I'm using in my day-to-day work and I'm it's contributor. The problem with moving it into erlang-mode is it depends on Projectile library [2], which helps defining project's root directory and provides caching of project's information and few other neat features. Basically only two function would have to be copied into erlang-mode to make it work, but I feel like it would be a duplication of efforts. In my opinion erlang-mode should be packaged as ELPA-defined [3] package (I have submitted a patch for it [4]), so it can use other packages as dependencies. Starting with version 24.1 Emacs contains support for package management (with dependencies support). In my opinion this is a proper way to evolve Erlang mode. OTP team, what is your vision how Erlang mode should be evolving? Best regards, Gleb Peregud 1: https://github.com/bbatsov/emacs-prelude 2: https://github.com/bbatsov/projectile 3: http://www.emacswiki.org/emacs/ELPA 4: http://erlang.org/pipermail/erlang-patches/2012-February/002664.html From g@REDACTED Sun Mar 4 18:35:37 2012 From: g@REDACTED (Garrett Smith) Date: Sun, 4 Mar 2012 11:35:37 -0600 Subject: [erlang-questions] Erlang development workflow In-Reply-To: References: Message-ID: Hi Frank, On Sat, Mar 3, 2012 at 3:32 PM, Frank Hunleth wrote: > I was wondering if anyone could share ideas or experiences on their > Erlang workflow especially for when the development machine is not the > best test machine. My situation is that I am running Erlang on an > embedded ARM that has Ethernet and decent DRAM and local storage. I > can do some development on my PC, but there's special hardware > attached to the ARM that my code needs to control sometimes. It would > really be nice if I could have an efficient workflow that lets me > develop on the PC and run on the ARM with the minimum number of manual > steps between editing and testing. I'm coming to Erlang from an > embedded C/C++ background. At the moment, my process is not much > better than what I'm used to. It seems like there are many potential > options, and I'm not sure what the best route is. What are you used to in C/C++ land? Apart from automating the code deployment to the target, I'm not sure there's a lot more you can do. Erlang gives you great remote debugging and code reloading, but that doesn't seem to be your problem. If you wanted to keep the target Erlang VM running throughout your development, you could run Mochiweb's "reloader" module to watch for local changes to beam files and forcefully reload them. You'd still need to push the compiled files over -- but that could be a part of your make file (e.g. extend the compile target to compile and ftp/rsync/etc to your target device). Garrett From max.lapshin@REDACTED Sun Mar 4 18:55:31 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 4 Mar 2012 20:55:31 +0300 Subject: [erlang-questions] Erlang development workflow In-Reply-To: References: Message-ID: On Sun, Mar 4, 2012 at 1:32 AM, Frank Hunleth wrote: > I was wondering if anyone could share ideas or experiences on their > Erlang workflow especially for when the development machine is not the > best test machine. My situation is that I am running Erlang on an > embedded ARM that has Ethernet and decent DRAM and local storage. I would just mount remote filesystem via sshfs + fuse and open ssh console on that machine. From fhunleth@REDACTED Sun Mar 4 20:45:36 2012 From: fhunleth@REDACTED (Frank Hunleth) Date: Sun, 4 Mar 2012 14:45:36 -0500 Subject: [erlang-questions] Erlang development workflow In-Reply-To: <05865BE6-FFC3-4BF9-A6FE-BFAF7F64A40F@gmail.com> References: <05865BE6-FFC3-4BF9-A6FE-BFAF7F64A40F@gmail.com> Message-ID: On Sat, Mar 3, 2012 at 7:19 PM, Tim Watson wrote: > I suspect you have a few choices about how you might do this. In terms of getting the latest 'code' onto the ARM, you can either > > 1. synchronise the file systems > 2. synchronise the loaded beam code on two emulators > > How do you get your code onto the ARM's local storage currently? There is a 'reloader' module floating around the internet (especially on github) which takes any recent file system changes and does a code purge and reload (hence the name) on updated modules. If you find a way to synchronise local file system changes so they're reflected on the ARM, the reloader module should do the rest. The way it works IIRC is that it tracks either the beam files (in ebin) or the source files (in src) - I can't remember which - and when something changes, it reloads. Thanks for the pointer to the 'reloader' module. I'll take a look at it. In answer to the question about how I get code onto ARM's local storage now, I currently use scp to copy the beam files over and then I switch over to the ARM console window and restart the VM. That's for most development. When I feel good about a change, I run a build script that bundles the Linux kernel, drivers, erlang vm, utilities etc. into a big binary software image. Those would be the bits that would be programmed to Flash during production. I then verify that the software image still runs the same. Building the whole software image takes on the order of 10-20 minutes for a full build and maybe a minute for a small update. Writing the image to Flash takes about a minute or two. I.e. long enough to get out of the flow. > You could also use common test to run your test suites on the ARM node using ct_master, although I have no idea how the file system requirements will work in this scenario. > > Another option that might suit you would be to keep a test 'server' running continuously on the ARM and control this from your PC using the ethernet connection. There are several mechanisms that could work well here, the simplest of which is to ship the code updates to the ARM over the ethernet connection (e.g., pass the updated beam code to code:load_binary using an rpc:call). Lot's of options. Thank you very much. I'll take a look. Frank From fhunleth@REDACTED Sun Mar 4 21:13:41 2012 From: fhunleth@REDACTED (Frank Hunleth) Date: Sun, 4 Mar 2012 15:13:41 -0500 Subject: [erlang-questions] Erlang development workflow In-Reply-To: References: Message-ID: On Sun, Mar 4, 2012 at 2:33 AM, Jon Watte wrote: > Does the target machine support ssh/rsync? If so, you could have your make > file just deploy to the machine after a successful compile. I can get rsync onto the target. FWIW, I could probably get most programs onto this board for development so long as they are command line and don't need too much CPU or memory. For production builds, I may have to take them off if they take up too much space, or slow things down, or present a security or other risk. Obviously, it's not good to have a huge difference between development and production builds. Rsync doesn't seem like a big deal to add, though. When I was asking the question, I was thinking that there might be a "magic Erlang button" that I press when I want to try out new code and the button will compile and update state on the ARM instantaneously. It's sounding like maybe a few updates to the Makefile to sync filesystems and then the reloader module might get there. > > We actually rely 100% on automated unit and functional tests. We mercilessly > mock any external dependencies (such as production databases) in > development. Our deployment pushes to production machines that are > similar-ish to the development machines, but a cluster of many more nodes. > The code actually hits end users (a hundred thousand at a time!) > automatically at some point after I check in and tests pass. Holy cow. Having that kind of test coverage sounds awesome. Thanks, Frank From fhunleth@REDACTED Sun Mar 4 21:26:32 2012 From: fhunleth@REDACTED (Frank Hunleth) Date: Sun, 4 Mar 2012 15:26:32 -0500 Subject: [erlang-questions] Erlang development workflow In-Reply-To: References: Message-ID: Hi Garrett, On Sun, Mar 4, 2012 at 12:35 PM, Garrett Smith wrote: > What are you used to in C/C++ land? Depending on the project, I do one or more of the following: 1. Use NFS to mount a directory from my PC where I have the executables or root filesystem 2. Build the program that I'm working on and copy the new binary over using whatever interface is most convenient 3. Build the complete software image and burn the whole Flash > Apart from automating the code deployment to the target, I'm not sure > there's a lot more you can do. Erlang gives you great remote debugging > and code reloading, but that doesn't seem to be your problem. > > If you wanted to keep the target Erlang VM running throughout your > development, you could run Mochiweb's "reloader" module to watch for > local changes to beam files and forcefully reload them. You'd still > need to push the compiled files over -- but that could be a part of > your make file (e.g. extend the compile target to compile and > ftp/rsync/etc to your target device). Thanks. I think that this is what I'm looking for. From als@REDACTED Mon Mar 5 00:11:32 2012 From: als@REDACTED (Anthony Shipman) Date: Mon, 5 Mar 2012 10:11:32 +1100 Subject: [erlang-questions] gen_sctp doesn't stream Message-ID: <201203051011.32675.als@iinet.net.au> gen_tcp does let me stream data. The driver will put data into a queue and block my process when the queue reaches a high water mark. Then it will allow the process to continue when the low water mark is reached. I can push data into a socket continuously as fast as it will go. The gen_sctp driver is built on the UDP driver. There is no queue. If I send too many packets I just get an eagain error code. It seems that the best I can do is to keep polling until I can send the next message. Am I right? -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From ok@REDACTED Mon Mar 5 04:46:32 2012 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 5 Mar 2012 16:46:32 +1300 Subject: [erlang-questions] Frying pan bug In-Reply-To: References: <2BE68D54-2705-4E02-BE08-B7F86259D6A8@cs.otago.ac.nz> Message-ID: On 4/03/2012, at 4:54 AM, David Goehrig wrote: > > Basically, what I'd be interesting in seeing is power, heat, time, cycles, and I/O traffic accounting on a per-Erlang process level, so that the supervisors could do a better job at managing a system under stress. Problem 1: the machine I'm typing on has two cores, but only one temperature diode. If Erlang process X is running on Core 0, and Erlang process Y is running on Core 1, and process X is doing something to make the chip overheat the temperature diode cannot tell me which core is producing the heat, so it cannot tell me which Erlang process is doing it. Problem 2: something I have not been able to find an answer to yet (and I've asked a couple of people I was sure would know) is how *fast* are the temperature diodes? Or more accurately, given a certain power change by a core and the thermal characteristics of the chip as a whole, and given that the sensors only report to the nearest 1 degree C, how long does it take before a power change causes a change in the reading from the temperature diode? If that change is not less than half the time slicing interval used by the scheduler, it's not clear to me that you can discriminate two processes running on the same core. I can imagine an averaging method to sort of deal with problem 2, but it doesn't survive problem 1. Another alternative would be to monitor the power used by each core, if that is possible, but the readings I see are frankly insane. (As are the f_bsize results that I get back from statvfs() on a Mac; can the OS *really* be recommending 32 MiB as the block size for read() on a particular partition? But that's another story.) From eppa.ramu@REDACTED Mon Mar 5 05:57:55 2012 From: eppa.ramu@REDACTED (raamu eppa) Date: Mon, 5 Mar 2012 10:27:55 +0530 Subject: [erlang-questions] Target System. Message-ID: Hi, Iam starting my erlang program as, mongodb:init([]). sendmail:send_message("your option"). How to start above as a OTP generic server for creating target system. Thanks, Ramu -------------- next part -------------- An HTML attachment was scrubbed... URL: From eppa.ramu@REDACTED Mon Mar 5 06:19:56 2012 From: eppa.ramu@REDACTED (raamu eppa) Date: Mon, 5 Mar 2012 10:49:56 +0530 Subject: [erlang-questions] target system Message-ID: Hi, In my erlang files there is no option for start OTP gen_server.But i want to create my project as a target system. Is there any option for creating target system without using OTP gen_server. Iam starting my project as, mongodb:init([]). sendmail:send_message("your option"). Thanks, Ramu. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gustav.simonsson@REDACTED Mon Mar 5 10:26:30 2012 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Mon, 05 Mar 2012 09:26:30 -0000 (GMT) Subject: [erlang-questions] Frying pan bug In-Reply-To: Message-ID: Afaik the temperature sensors of Intel CPU cores are designed to provide thermal throttling and thermal shutdown - not to report accurate temperatures. Even if the sensors are fast enough to react to temperature differences between scheduler time slices you wouldn't see that since they can typically only report changes within 1 degree (and the error margin can be as high as 5-10 degrees depending on the CPU model). Also in modern CPUs you typically see a few degrees difference between cores, part due to the error margin of the sensors base calibration and in part because the cooling paste and heatsink mounting is never perfectly symmetrical over the chip surface. // Gustav Simonsson Sent from my PC ----- Original Message ----- > From: "Richard O'Keefe" > To: "David Goehrig" > Cc: "Erlang" > Sent: Monday, 5 March, 2012 4:46:32 AM > Subject: Re: [erlang-questions] Frying pan bug > > > On 4/03/2012, at 4:54 AM, David Goehrig wrote: > > > > Basically, what I'd be interesting in seeing is power, heat, time, > > cycles, and I/O traffic accounting on a per-Erlang process level, > > so that the supervisors could do a better job at managing a system > > under stress. > > Problem 1: the machine I'm typing on has two cores, but only one > temperature diode. > If Erlang process X is running on Core 0, and > Erlang process Y is running on Core 1, and > process X is doing something to make the chip overheat > the temperature diode cannot tell me which core is producing the > heat, so it cannot tell me which Erlang process is doing it. > > Problem 2: something I have not been able to find an answer to > yet (and I've asked a couple of people I was sure would know) > is how *fast* are the temperature diodes? Or more accurately, > given a certain power change by a core and the thermal > characteristics of the chip as a whole, and given that the > sensors only report to the nearest 1 degree C, how long does > it take before a power change causes a change in the reading > from the temperature diode? If that change is not less than > half the time slicing interval used by the scheduler, it's not > clear to me that you can discriminate two processes running on > the same core. > > I can imagine an averaging method to sort of deal with problem 2, > but it doesn't survive problem 1. > > Another alternative would be to monitor the power used by each core, > if that is possible, but the readings I see are frankly insane. > (As are the f_bsize results that I get back from statvfs() on a Mac; > can the OS *really* be recommending 32 MiB as the block size for > read() on a particular partition? But that's another story.) > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From watson.timothy@REDACTED Mon Mar 5 10:32:21 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 5 Mar 2012 09:32:21 +0000 Subject: [erlang-questions] target system In-Reply-To: References: Message-ID: <4A42EE9F-6DB1-487B-BA4B-3B8A35876D62@gmail.com> Hi Ramu, Frankly this doesn't make a lot of sense to me, but from I gather you're trying to create a release (aka target system) and want to know how to start your 'project' up without using a gen_server. OTP is predicated on the concept of an 'Application' - see http://erlang.org/doc/design_principles/applications.html for details - and without this, you will be hard pressed to build a release. Your application does not have to be built using gen_server, but there ought to be a top level supervisor or at a minimum, a top level process that is to be run. Aside: think about this like you would a Java or .NET (or python/ruby/perl/etc if you) program. You will start the program from some main class (or script) and this must block the main thread in order to prevent the program from terminating - so you will spawn some background thread(s) to do the actual work - and in many cases if you're running inside of some kind of container (like a web application server e.g., tomcat/weblogic, IIS, tornado, etc) or framework (like WCF/fuse/camel/mule/twisted/event-machine/etc) so you just implement the 'callback' part of the contract and the framework/container does the rest. It is necessary to have your code running in a process, and whilst you can forgo using gen_servers and supervisors if you really wish to, you will still need to spawn a process in which your application shall run. There is such a thing as a library application, which has no start module. You can achieve this by removing the 'mod' element from your application resource file, like so: %% foobar.app {application, foobar, [ {description, ""}, {vsn, "0.0.1"}, {applications, [kernel, stdlib]} ]}. In this case however, because your library (i.e., an 'inactive' application) has no startup function - because there is no application callback module - there is no way to initialise the start state. This option is really intended for library code that has no state, hence the name. I would strongly suggest reading the documentation, especially the system and design principles, before trying to build a target system in a non-standard way. I would also highly recommend reading http://learnyousomeerlang.com/, especially the section http://learnyousomeerlang.com/building-applications-with-otp, before proceeding. Cheers Tim On 5 Mar 2012, at 05:19, raamu eppa wrote: > Hi, > > In my erlang files there is no option for start OTP gen_server.But i want to create my project as a target system. > Is there any option for creating target system without using OTP gen_server. > Iam starting my project as, > mongodb:init([]). > sendmail:send_message("your option"). > > Thanks, > Ramu. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From aleksandr.vin@REDACTED Mon Mar 5 14:49:48 2012 From: aleksandr.vin@REDACTED (Aleksandr Vinokurov) Date: Mon, 5 Mar 2012 17:49:48 +0400 Subject: [erlang-questions] Strings usage caveats Message-ID: Hello all, I study Erlang strings usage in production. In doc/efficiency_guide/myths.html there is a paragraph that say "Actually, string handling could be slow if done improperly. In Erlang, you'll have to think a little more about how the strings are used and choose an appropriate representation and use the?re module instead of the obsolete?regexp?module if you are going to use regular expressions." I have a very poor experience in programming in Erlang/OTP so that sentence was rather abstract for me. I suppose that the root of the problems with strings is in variables immutability and thus a copying of the whole source string in case of its modification. But it seems to me that it's not that all. Can you please supply me with the sources to read or examples and hints about strings performance in Erlang. -- ????????? ????????? +7 (921) 982-21-43 @aleksandrvin From mfidelman@REDACTED Mon Mar 5 16:36:35 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Mon, 05 Mar 2012 10:36:35 -0500 Subject: [erlang-questions] models for replicated/distributed processors Message-ID: <4F54DD83.5030305@meetinghouse.net> Hi Folks, I'm trying to think through various approaches to fault-tolerance for an actor-based system architecture - generally around the notion of replicated copies of actors operating on different nodes. Two questions to the assembled wisdom: 1. Has anybody done any work with replicated, synchronized processes spread across multiple erlang nodes? If so, can you share anything about architectural concepts? (Pointers to papers or slide decks would be much appreciated). 2. A more specific question: I notice that spawn-link has a from that allows creating a linked process on a different node - spawn_link(Node, Module, Function, Args) -> pid() - which seems like a good start on building supervision trees across nodes. But... there doesn't seem to be an equivalent form of spawn_monitor - seems like spawn_opt/5 can be used instead, but sort of curious about the omission. Anybody know the story? Thanks, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From loris.fichera@REDACTED Mon Mar 5 17:11:57 2012 From: loris.fichera@REDACTED (Loris Fichera) Date: Mon, 05 Mar 2012 17:11:57 +0100 Subject: [erlang-questions] mochi/erl_img Message-ID: <4F54E5CD.8040900@gmail.com> Hello list, I was looking for a library able to read (and, possibly, author) images in various formats and I decided to give erl_img [1] a try. Apparently, it perfectly suits my needs. Anyway, I'm not able to read jpeg images, that is: 1> erl_img:load ("/home/loris/shuttle_iss.jpg"). {ok,{erl_image,image_jpeg,undefined, "/home/loris/shuttle_iss.jpg",undefined,undefined,undefined, undefined,[],undefined,6048,4032,8,3,1,[],upper_left, undefined,[]}} As you can see, the last field of the #erl_image record (which, according to [2] should be the 'pixmap') is set to be the empty list. Having a look at the code, it seems the 'image_jpeg:read' function, which should be the responsible for reading the content of the image, does nothing more than return the empty #erl_image [3]. Am I missing something really basic? Thanks in advance. Loris. [1] https://github.com/mochi/erl_img [2] https://github.com/mochi/erl_img/blob/master/include/erl_img.hrl [3] https://github.com/mochi/erl_img/blob/master/src/image_jpeg.erl#L55 From mattevans123@REDACTED Mon Mar 5 18:01:39 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Mon, 5 Mar 2012 12:01:39 -0500 Subject: [erlang-questions] Strings usage caveats In-Reply-To: References: Message-ID: In a nut-shell a string in Erlang is represented as a list of (ASCII) characters. So "hello" becomes: [104,101,108,108,111]. This has many advantages in terms of been able to process strings. But there are problems: 1) It can take up lots of memory. A list is 1 word + 1 word for each element + size of element. So "hello" would (on a 64 bit machine) be 53 bytes. 2) Many of the modules are implemented in Erlang (rather than in a BIF). Doing extensive string manipulation this way *could* be slow (when compared to C or other languages. Fortunately if you need performance you can represent strings as binaries (I personally think that we should be thinking strings as binaries all the time now). So the string "hello" would become <<"hello">> as a binary. The memory efficiency is much better than with lists (for anything over a few 10's of bytes it's pretty much "native" size - there is a small overhead IRC). Better still you can use the very fast binary module to do much of the processing. That with binary comprehensions and binary pattern matching allows you to buil powerful applications. Personally I've refactored much of my "string handling" code to use binaries now. What would be nice is for the "re" and "string" modules to allow binaries and lists as input. Matt > Date: Mon, 5 Mar 2012 17:49:48 +0400 > From: aleksandr.vin@REDACTED > To: erlang-questions@REDACTED > Subject: [erlang-questions] Strings usage caveats > > Hello all, > > I study Erlang strings usage in production. In > doc/efficiency_guide/myths.html there is a paragraph that say > "Actually, string handling could be slow if done improperly. In > Erlang, you'll have to think a little more about how the strings are > used and choose an appropriate representation and use the re module > instead of the obsolete regexp module if you are going to use regular > expressions." > > I have a very poor experience in programming in Erlang/OTP so that > sentence was rather abstract for me. I suppose that the root of the > problems with strings is in variables immutability and thus a copying > of the whole source string in case of its modification. But it seems > to me that it's not that all. > > Can you please supply me with the sources to read or examples and > hints about strings performance in Erlang. > > -- > ????????? ????????? > +7 (921) 982-21-43 > @aleksandrvin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From wrr@REDACTED Mon Mar 5 18:09:44 2012 From: wrr@REDACTED (Jan Wrobel) Date: Mon, 5 Mar 2012 18:09:44 +0100 Subject: [erlang-questions] models for replicated/distributed processors In-Reply-To: <4F54DD83.5030305@meetinghouse.net> References: <4F54DD83.5030305@meetinghouse.net> Message-ID: On Mon, Mar 5, 2012 at 4:36 PM, Miles Fidelman wrote: > Hi Folks, > > I'm trying to think through various approaches to fault-tolerance for an > actor-based system architecture - generally around the notion of replicated > copies of actors operating on different nodes. > > Two questions to the assembled wisdom: > > 1. Has anybody done any work with replicated, synchronized processes spread > across multiple erlang nodes? ?If so, can you share anything about > architectural concepts? ?(Pointers to papers or slide decks would be much > appreciated). Paxos algorithm allows to keep state of distributed processes consistent. An accessible paper about it is: http://research.microsoft.com/en-us/um/people/lamport/pubs/paxos-simple.pdf It seems there were several attempts at implementing generic Paxos library in Erlang (https://www.google.com/search?q=paxos+erlang), I don't know how matured these implementations are, maybe they can serve you as a starting point. Cheers, Jan From emmiller@REDACTED Mon Mar 5 18:20:27 2012 From: emmiller@REDACTED (Evan Miller) Date: Mon, 5 Mar 2012 11:20:27 -0600 Subject: [erlang-questions] mochi/erl_img In-Reply-To: <4F54E5CD.8040900@gmail.com> References: <4F54E5CD.8040900@gmail.com> Message-ID: Hi Loris, Tony Rogvall implemented JPEG read support for erl_img a while back. I've incorporated it into my branch of erl_img which you can find here: https://github.com/evanmiller/erl_img That branch also includes some basic image manipulation functions (crop and scale) as well as PNG write support. Hope you find it helpful. Evan On Mon, Mar 5, 2012 at 10:11 AM, Loris Fichera wrote: > Hello list, > > I was looking for a library able to read (and, possibly, author) images > in various formats and I decided to give erl_img [1] a try. > Apparently, it perfectly suits my needs. Anyway, I'm not able to read > jpeg images, that is: > > 1> erl_img:load ("/home/loris/shuttle_iss.jpg"). > {ok,{erl_image,image_jpeg,undefined, > "/home/loris/shuttle_iss.jpg",undefined,undefined,undefined, > undefined,[],undefined,6048,4032,8,3,1,[],upper_left, undefined,[]}} > > As you can see, the last field of the #erl_image record (which, > according to [2] should be the 'pixmap') is set to be the empty list. > > Having a look at the code, it seems the 'image_jpeg:read' function, > which should be the responsible for reading the content of the image, > does nothing more than return the empty #erl_image [3]. > > Am I missing something really basic? > Thanks in advance. > > ?Loris. > > [1] https://github.com/mochi/erl_img > [2] https://github.com/mochi/erl_img/blob/master/include/erl_img.hrl > [3] https://github.com/mochi/erl_img/blob/master/src/image_jpeg.erl#L55 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mattevans123@REDACTED Mon Mar 5 18:30:16 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Mon, 5 Mar 2012 12:30:16 -0500 Subject: [erlang-questions] Strings usage caveats In-Reply-To: References: , Message-ID: Oh, and one more nice thing with binaries is that when they are greater than 64 bytes (IRC) they are reference counted and can be shared if passed between processes. Matt From: mattevans123@REDACTED To: aleksandr.vin@REDACTED; erlang-questions@REDACTED Date: Mon, 5 Mar 2012 12:01:39 -0500 Subject: Re: [erlang-questions] Strings usage caveats In a nut-shell a string in Erlang is represented as a list of (ASCII) characters. So "hello" becomes: [104,101,108,108,111]. This has many advantages in terms of been able to process strings. But there are problems: 1) It can take up lots of memory. A list is 1 word + 1 word for each element + size of element. So "hello" would (on a 64 bit machine) be 53 bytes. 2) Many of the modules are implemented in Erlang (rather than in a BIF). Doing extensive string manipulation this way *could* be slow (when compared to C or other languages. Fortunately if you need performance you can represent strings as binaries (I personally think that we should be thinking strings as binaries all the time now). So the string "hello" would become <<"hello">> as a binary. The memory efficiency is much better than with lists (for anything over a few 10's of bytes it's pretty much "native" size - there is a small overhead IRC). Better still you can use the very fast binary module to do much of the processing. That with binary comprehensions and binary pattern matching allows you to buil powerful applications. Personally I've refactored much of my "string handling" code to use binaries now. What would be nice is for the "re" and "string" modules to allow binaries and lists as input. Matt > Date: Mon, 5 Mar 2012 17:49:48 +0400 > From: aleksandr.vin@REDACTED > To: erlang-questions@REDACTED > Subject: [erlang-questions] Strings usage caveats > > Hello all, > > I study Erlang strings usage in production. In > doc/efficiency_guide/myths.html there is a paragraph that say > "Actually, string handling could be slow if done improperly. In > Erlang, you'll have to think a little more about how the strings are > used and choose an appropriate representation and use the re module > instead of the obsolete regexp module if you are going to use regular > expressions." > > I have a very poor experience in programming in Erlang/OTP so that > sentence was rather abstract for me. I suppose that the root of the > problems with strings is in variables immutability and thus a copying > of the whole source string in case of its modification. But it seems > to me that it's not that all. > > Can you please supply me with the sources to read or examples and > hints about strings performance in Erlang. > > -- > ????????? ????????? > +7 (921) 982-21-43 > @aleksandrvin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Mon Mar 5 18:44:03 2012 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 5 Mar 2012 17:44:03 +0000 Subject: [erlang-questions] Strings usage caveats In-Reply-To: References: Message-ID: With Unicode, a string becomes a list of code points! ASCII is a thing of the past. See http://www.erlang.org/doc/man/unicode.html But always remember, strings in general are not as easy as you think. Robby 2012/3/5 Matthew Evans > In a nut-shell a string in Erlang is represented as a list of (ASCII) > characters. So "hello" becomes: [104,101,108,108,111]. > > This has many advantages in terms of been able to process strings. But > there are problems: > > 1) It can take up lots of memory. A list is 1 word + 1 word for each > element + size of element. So "hello" would (on a 64 bit machine) be 53 > bytes. > > 2) Many of the modules are implemented in Erlang (rather than in a BIF). > Doing extensive string manipulation this way *could* be slow (when compared > to C or other languages. > > Fortunately if you need performance you can represent strings as binaries > (I personally think that we should be thinking strings as binaries all the > time now). > > So the string "hello" would become <<"hello">> as a binary. The memory > efficiency is much better than with lists (for anything over a few 10's of > bytes it's pretty much "native" size - there is a small overhead IRC). > Better still you can use the very fast binary module to do much of the > processing. That with binary comprehensions and binary pattern matching > allows you to buil powerful applications. > > Personally I've refactored much of my "string handling" code to use > binaries now. What would be nice is for the "re" and "string" modules to > allow binaries and lists as input. > > Matt > > > Date: Mon, 5 Mar 2012 17:49:48 +0400 > > From: aleksandr.vin@REDACTED > > To: erlang-questions@REDACTED > > Subject: [erlang-questions] Strings usage caveats > > > > > Hello all, > > > > I study Erlang strings usage in production. In > > doc/efficiency_guide/myths.html there is a paragraph that say > > "Actually, string handling could be slow if done improperly. In > > Erlang, you'll have to think a little more about how the strings are > > used and choose an appropriate representation and use the re module > > instead of the obsolete regexp module if you are going to use regular > > expressions." > > > > I have a very poor experience in programming in Erlang/OTP so that > > sentence was rather abstract for me. I suppose that the root of the > > problems with strings is in variables immutability and thus a copying > > of the whole source string in case of its modification. But it seems > > to me that it's not that all. > > > > Can you please supply me with the sources to read or examples and > > hints about strings performance in Erlang. > > > > -- > > ????????? ????????? > > +7 (921) 982-21-43 > > @aleksandrvin > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hukl@REDACTED Mon Mar 5 18:55:27 2012 From: hukl@REDACTED (John-Paul Bader) Date: Mon, 05 Mar 2012 18:55:27 +0100 Subject: [erlang-questions] Erlang Community Conference Message-ID: <4F54FE0F.6090700@berlin.ccc.de> Hey, I want to announce that a bunch of people here in Berlin/Germany are about to organize a community driven erlang conference. We know that there are quite a lot of erlang related conferences by erlang solutions already and we really like them, however we believe it would be great to have an independant, community driven event as well. So we don't want to compete with the existing conferences but add another nice one to the mix. We are still at a very early stage as we are just gathering together all the details and people so please consider any information on this eventual consistent. We don't even have a name yet. Current project title is "Erlounge Conference". Yet we would really like to get some early feedback on this from the community so we don't run into the wrong direction right from the start. This is the brief description of what we want to do: What is it? =========== The erlounge conference is a small single track conference about Erlang and other programming languages with similar concepts like Scala, Clojure and Haskell. It is a non-profit conference which is organized by community effort. The entrance fee will be as low as we can make it so that everybody has a chance attending the conference. Think of a range between 40-80 Euro. The talks will be about real projects and real people, about hacking and inspiraton for what can be done with those languages. It will have talks for beginners as well as experienced developers and in addition there will be lighting talks for showing other cool stuff. Where is it? ============ The conference will take place in Berlin / Germany But we have to find a proper venue yet which is suitable for 200-400 people. When is it? =========== We will try to make it happen in fall 2012. Could be one or two days - preferrably on a weekend. What is it about? ================= erlang, other functional languages and functional programming in general, building systems that never fail, writing backends for the next big web app, advantages of erlang over other languages like ruby, python, javascript Any feedback is appreciated! We will have a dedicated interface for tracking who would be interested to come later on. Kind regards, John From loris.fichera@REDACTED Mon Mar 5 19:06:22 2012 From: loris.fichera@REDACTED (Loris Fichera) Date: Mon, 05 Mar 2012 19:06:22 +0100 Subject: [erlang-questions] mochi/erl_img In-Reply-To: References: <4F54E5CD.8040900@gmail.com> Message-ID: <4F55009E.8020003@gmail.com> Hello Evan, On 03/05/2012 06:20 PM, Evan Miller wrote: > Hi Loris, > > Tony Rogvall implemented JPEG read support for erl_img a while back. > I've incorporated it into my branch of erl_img which you can find > here: > > https://github.com/evanmiller/erl_img > > That branch also includes some basic image manipulation functions > (crop and scale) as well as PNG write support. Hope you find it > helpful. Thanks for sharing! I cloned your branch and ran the tests but, unfortunately, I got errors: http://pastebin.com/93nQxbkM Loris. From loris.fichera@REDACTED Mon Mar 5 19:22:02 2012 From: loris.fichera@REDACTED (Loris Fichera) Date: Mon, 05 Mar 2012 19:22:02 +0100 Subject: [erlang-questions] mochi/erl_img In-Reply-To: <4F55009E.8020003@gmail.com> References: <4F54E5CD.8040900@gmail.com> <4F55009E.8020003@gmail.com> Message-ID: <4F55044A.2050202@gmail.com> On 03/05/2012 07:06 PM, Loris Fichera wrote: > Hello Evan, > > On 03/05/2012 06:20 PM, Evan Miller wrote: >> Hi Loris, >> >> Tony Rogvall implemented JPEG read support for erl_img a while back. >> I've incorporated it into my branch of erl_img which you can find >> here: >> >> https://github.com/evanmiller/erl_img >> >> That branch also includes some basic image manipulation functions >> (crop and scale) as well as PNG write support. Hope you find it >> helpful. > > Thanks for sharing! > I cloned your branch and ran the tests but, unfortunately, I got errors: > > http://pastebin.com/93nQxbkM I forgot to say that I'm running Erlang R15B on top of Debian Wheezy - 64 bit. Loris. From sunwood360@REDACTED Mon Mar 5 21:09:46 2012 From: sunwood360@REDACTED (envelopes envelopes) Date: Mon, 5 Mar 2012 12:09:46 -0800 Subject: [erlang-questions] Firewall friendly erlang node Message-ID: Hi: One challenge we faced is the communication between erlang nodes. Is it possible to plugin other underline transport , such as SPDY, websocks, etc? So the system will be more firewall friendly. Please redirect me if the question has been discussed. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Mon Mar 5 21:39:04 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Mon, 05 Mar 2012 15:39:04 -0500 Subject: [erlang-questions] models for replicated/distributed processors In-Reply-To: References: <4F54DD83.5030305@meetinghouse.net> Message-ID: <4F552468.1000504@meetinghouse.net> Jan Wrobel wrote: > On Mon, Mar 5, 2012 at 4:36 PM, Miles Fidelman > wrote: >> Hi Folks, >> >> I'm trying to think through various approaches to fault-tolerance for an >> actor-based system architecture - generally around the notion of replicated >> copies of actors operating on different nodes. >> >> Two questions to the assembled wisdom: >> >> 1. Has anybody done any work with replicated, synchronized processes spread >> across multiple erlang nodes? If so, can you share anything about >> architectural concepts? (Pointers to papers or slide decks would be much >> appreciated). > Paxos algorithm allows to keep state of distributed processes > consistent. An accessible paper about it is: > http://research.microsoft.com/en-us/um/people/lamport/pubs/paxos-simple.pdf > It seems there were several attempts at implementing generic Paxos > library in Erlang (https://www.google.com/search?q=paxos+erlang), I > don't know how matured these implementations are, maybe they can > serve you as a starting point. > Thanks! -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From lukas@REDACTED Mon Mar 5 21:51:41 2012 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 5 Mar 2012 21:51:41 +0100 Subject: [erlang-questions] Firewall friendly erlang node In-Reply-To: References: Message-ID: Hi, Distributed Erlang is not really designed to be used in environment where there are firewalls in between the nodes. You can do what you ask in the distribution protocol[1] and possibly setup epmd[2] to detect nodes thorugh firewalls but most people (that I know of) give up at this point and switch to an application specific carrier like http and use that for inter-node communication. Lukas [1] see http://www.erlang.org/doc/apps/erts/alt_dist.html for details. [2] http://www.erlang.org/doc/man/epmd.html On Mon, Mar 5, 2012 at 9:09 PM, envelopes envelopes wrote: > Hi: > > ??? One challenge we faced is the communication between erlang nodes. Is it > possible to plugin other underline transport , such as SPDY, websocks, etc? > So the system will be more firewall friendly. > > Please redirect me if the question has been discussed. > > Thanks. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From nx@REDACTED Mon Mar 5 21:51:23 2012 From: nx@REDACTED (nx) Date: Mon, 5 Mar 2012 15:51:23 -0500 Subject: [erlang-questions] mochi/erl_img In-Reply-To: References: <4F54E5CD.8040900@gmail.com> Message-ID: <20120305155123.22bbfdcf.nx@nu-ex.com> You may also want to check out erl_gm, which is an Erlang wrapper for the GraphicsMagick command line application: https://github.com/nuex/erl_gm It supports getting all image characteristics defined in GraphicsMagick "format" option and currently has a few image manipulation options like crop, resize, magnify, blur and more. On Mon, 5 Mar 2012 11:20:27 -0600 Evan Miller wrote: > Hi Loris, > > Tony Rogvall implemented JPEG read support for erl_img a while back. > I've incorporated it into my branch of erl_img which you can find > here: > > https://github.com/evanmiller/erl_img > > That branch also includes some basic image manipulation functions > (crop and scale) as well as PNG write support. Hope you find it > helpful. > > Evan > > On Mon, Mar 5, 2012 at 10:11 AM, Loris Fichera wrote: > > Hello list, > > > > I was looking for a library able to read (and, possibly, author) images > > in various formats and I decided to give erl_img [1] a try. > > Apparently, it perfectly suits my needs. Anyway, I'm not able to read > > jpeg images, that is: > > > > 1> erl_img:load ("/home/loris/shuttle_iss.jpg"). > > {ok,{erl_image,image_jpeg,undefined, > > "/home/loris/shuttle_iss.jpg",undefined,undefined,undefined, > > undefined,[],undefined,6048,4032,8,3,1,[],upper_left, undefined,[]}} > > > > As you can see, the last field of the #erl_image record (which, > > according to [2] should be the 'pixmap') is set to be the empty list. > > > > Having a look at the code, it seems the 'image_jpeg:read' function, > > which should be the responsible for reading the content of the image, > > does nothing more than return the empty #erl_image [3]. > > > > Am I missing something really basic? > > Thanks in advance. > > > > ?Loris. > > > > [1] https://github.com/mochi/erl_img > > [2] https://github.com/mochi/erl_img/blob/master/include/erl_img.hrl > > [3] https://github.com/mochi/erl_img/blob/master/src/image_jpeg.erl#L55 > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- nx From ok@REDACTED Mon Mar 5 22:11:44 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 6 Mar 2012 10:11:44 +1300 Subject: [erlang-questions] Strings usage caveats In-Reply-To: References: Message-ID: <3B523449-7023-4FDB-92D0-9CA8946A5FCF@cs.otago.ac.nz> On 6/03/2012, at 2:49 AM, Aleksandr Vinokurov wrote: > I have a very poor experience in programming in Erlang/OTP so that > sentence was rather abstract for me. I suppose that the root of the > problems with strings is in variables immutability and thus a copying > of the whole source string in case of its modification. But it seems > to me that it's not that all. No, do not suppose that. After all, strings are immutable in Java as well, and nobody talks about _them_ being inefficient. (Except me, but that's another story, and it's not related to mutability.) In a Unicode world, there are very very very few cases where it makes sense to change part of a string other than by taking a substring concatenating with other strings applying a regular expression rewrite (like the 's/old/new/' command in Vi(m)). There are three representations of strings commonly used in Erlang, and they have different tradeoffs. (1) Linked lists of character codes. "abc" is the same thing as [97,98,99]. These are simple to process from left to right. Dropping the first D characters takes O(D) time (Java: O(1)). Taking the first T characters takes O(T) time (Java: O(1)). Concatenating length M with length N takes O(M) time (Java: O(M+N)). The problem is SPACE. Each character requires one machine word for the character code and another to point to the rest of the list. So a string of N characters requires 8N bytes in a 32-bit environment, or 16N bytes in a 64-bit environment. C using UTF-8 would probably take about 1.1N bytes for the kind of stuff I see these days; Java would take 2N. Of course, to handle full Unicode without the kind of contortions Java requires, you'd need 4N bytes anyway, so 8N doesn't look as bad as it did when everyone either rejoiced in ASCII (or EBCDIC) or cursed its foreign limitations. (2) Historically, people used "iolists" as strings. It has now been clarified that "iolists" are sequences of BYTES, not sequences of CHARACTERS. iolist --> [] | [byte | iolist] | [iolist | iolist]. Nothing stops us using essentially the same data structure for characters, we just have to realise that what we get is not an iolist any more and built-ins that expect an iolist won't be happy with it. That need not be a problem. fclist --> [] | [codepoint | fclist] | [fclist | fclist] These are slightly trickier to process from left to right, but only *slightly*. Selecting substrings is not particularly pleasant, and I cannot make it happen without turning over some garbage. Where these things shine is concatenation, which is O(1). The way to use them is to build up a complex string by concatenating all the pieces, then flattening it at the end. This is, if you will, the Erlang analogue of Java's StringBuilder. (Analogue, NOT homologue!) Dropping the first D characters and then taking the next T costs (3) Binaries. <<"abc">> is the same as <<97,98,99>>. A binary is a packed immutable array of bytes. (Well, it can be thought of as a bit string these days, but "bytes" are all we need for strings.) Binaries are very close to Java strings, except for usually taking less space. They can be sliced in constant time. They don't support fast concatenation, but then you can build up a list of binaries and concatenate them at the end. In all three cases, the data structure itself does not keep track of what the encoding is. (And of course, neither do Java or C#.) > > Can you please supply me with the sources to read or examples and > hints about strings performance in Erlang. It is all completely obvious once you know what the data structure is. Simple one-way linked lists of character codes, or (references to slices of) packed arrays of bytes. From watson.timothy@REDACTED Mon Mar 5 23:54:35 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 5 Mar 2012 22:54:35 +0000 Subject: [erlang-questions] support for .ez archives Message-ID: Hello list, I have a few questions about support for .ez archives, and one or two suggestions as well. - is a timeline for fully supporting them? - does the -include_lib(...) directive now work (in R15) when the headers are in an archive? - does code:lib_dir/2 work properly with archives? As a suggestion: I've noticed that the way most people build an escript binary is to glue everything together into a zip file with the appropriate header content and I wonder whether having 'executable (.ez) archives' would be a better approach? Perhaps this works today and I haven't figured it out? What are peoples thoughts about this? Cheers, Tim From prouleau001@REDACTED Tue Mar 6 01:04:12 2012 From: prouleau001@REDACTED (Pierre Rouleau) Date: Mon, 5 Mar 2012 19:04:12 -0500 Subject: [erlang-questions] Erlang on ARM In-Reply-To: References: <72786C06-997C-4FFD-800E-5BD2B3E92220@cs.otago.ac.nz> <5BC624B5-3013-4C46-94C2-8983AA883BD4@cs.otago.ac.nz> Message-ID: Richard, You might also be interested in other ARM-based boards. A friend of mine referred me to the Friendly Mini2440 has an ethernet port along with several I/O (including USB). They have an SDK and several packaging options. See http://www.friendlyarm.net/products/micro2440 . I have not tried this board yet. Just mentioning it after seeing your post. /Pierre -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Tue Mar 6 03:21:42 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 6 Mar 2012 03:21:42 +0100 Subject: [erlang-questions] More "eper" help or tutos Message-ID: <1C73706A-9C51-4B30-A3F5-1850F70FF257@gmail.com> Hi Mats, Hi guys, Is there any elaborated tuto about eper: http://code.google.com/p/eper/ The only one I found was a small but useful wiki page about redbug: http://code.google.com/p/eper/wiki/redbug I'm interested to learn how to use gperf and sherk Help or pointers appreciated! Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From mapandfold@REDACTED Tue Mar 6 04:22:35 2012 From: mapandfold@REDACTED (Justus) Date: Tue, 6 Mar 2012 11:22:35 +0800 Subject: [erlang-questions] memory leak in zlib:unzip Message-ID: Hi all, I think there are memory leaks in zlib:unzip, when it gets a wrong stream. Test code is listed below. check_erl() -> Bin = erlang:list_to_binary([X rem 256 || X <- lists:seq(0, 10000)]), check0(Bin, 0). check0(Bin, Off) when Off > byte_size(Bin) -> 0; check0(Bin, Off) -> <<_X:Off/binary, SB/binary>> = Bin, (catch zlib:unzip(SB)), check0(Bin, Off + 1). Have a check on erlang:memory() before and after check_erl() is called. -- Best Regards, Justus From jeffm@REDACTED Tue Mar 6 06:43:50 2012 From: jeffm@REDACTED (jm) Date: Tue, 06 Mar 2012 16:43:50 +1100 Subject: [erlang-questions] SNMP get,set,walk, etc Message-ID: <4F55A416.9070202@ghostgun.com> Are there any decent guides anyone can point me to that deal with using Erlang to retrieve SNMP measurements from devices such as network routers and switches? The stuff I've been able to dig up in the past has dealt mostly with how to add a SNMP agent to your own applications. Jeff. From mapandfold@REDACTED Tue Mar 6 06:58:23 2012 From: mapandfold@REDACTED (Justus) Date: Tue, 6 Mar 2012 13:58:23 +0800 Subject: [erlang-questions] memory leak in zlib:unzip In-Reply-To: References: Message-ID: Hi all, I found that when exceptions occur in zlib:unzip, the zlib_drv port will be left open. So the system will run out of ports quickly. So, when we call zlib:unzip, we must insure that the stream can be unzipped successfully. But how could we know that without unzip it? Maybe we need a try_unzip. But if we have try_unzip, do we need zlib:unzip anymore? Quite a paradox here. try_unzip(Data) -> Z = zlib:open(), zlib:inflateInit(Z, -?MAX_WBITS), R = case (catch zlib:inflate(Z, Data)) of {'EXIT', _Reason} -> error; Bs -> {ok, iolist_to_binary(Bs)} end, (catch zlib:inflateEnd(Z)), zlib:close(Z), R. On Tue, Mar 6, 2012 at 11:22 AM, Justus wrote: > Hi all, > > I think there are memory leaks in zlib:unzip, when it gets a wrong > stream. Test code is listed below. > > check_erl() -> > ? ?Bin = erlang:list_to_binary([X rem 256 || X <- lists:seq(0, 10000)]), > ? ?check0(Bin, 0). > > check0(Bin, Off) when Off > byte_size(Bin) -> 0; > check0(Bin, Off) -> > ? ?<<_X:Off/binary, SB/binary>> = Bin, > ? ?(catch zlib:unzip(SB)), > ? ?check0(Bin, Off + 1). > > Have a check on erlang:memory() before and after check_erl() is called. > > -- > Best Regards, > Justus -- Best Regards, Justus From sunwood360@REDACTED Tue Mar 6 07:07:15 2012 From: sunwood360@REDACTED (envelopes envelopes) Date: Mon, 5 Mar 2012 22:07:15 -0800 Subject: [erlang-questions] dumb question from rookie Message-ID: I have a sample program generated from rebar 1. mysample_app.erl -module(mysample_app). -behaviour(application). %% Application callbacks -export([start/2, stop/1, main/1]). %% =================================================================== %% Application callbacks %% =================================================================== start(_StartType, _StartArgs) -> mysample_sup:start_link(). stop(_State) -> ok. main([A]) -> I = list_to_integer(atom_to_list(A)), F = fac(I), io:format("factorial ~w = ~w~n" ,[I, F]), init:stop(). fac(0) -> 1; fac(N) -> N*fac(N-1). 2. mysample_sup.erl -module(mysample_sup). -behaviour(supervisor). %% API -export([start_link/0]). %% Supervisor callbacks -export([init/1]). %% Helper macro for declaring children of supervisor -define(CHILD(I, Type), {I, {I, start_link, []}, permanent, 5000, Type, [I]}). %% =================================================================== %% API functions %% =================================================================== start_link() -> supervisor:start_link({local, ?MODULE}, ?MODULE, []). %% =================================================================== %% Supervisor callbacks %% =================================================================== init([]) -> {ok, { {one_for_one, 5, 10}, []} }. 3) the problem. erl -pa ebin -s mysample_app main 23 ===> worked fine erl -pa ebin -s mysample_app start ===> crashed on boot. {"init terminating in do_boot",{undef,[{mysample_app,start,[],[]},{init,start_it ,1,[{file,"init.erl"},{line,1042}]},{init,start_em,1,[{file,"init.erl"},{line,10 22}]}]}} Crash dump was written to: erl_crash.dump init terminating in do_boot () what is the cause ? my erlang is the latest R15. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ryanhuffman@REDACTED Tue Mar 6 07:21:47 2012 From: ryanhuffman@REDACTED (Ryan Huffman) Date: Mon, 5 Mar 2012 22:21:47 -0800 Subject: [erlang-questions] dumb question from rookie In-Reply-To: References: Message-ID: <451378EF51F74C12B22B409C5884057D@gmail.com> It looks like you are trying to call your state function with 0 arguments, but the start function has an arity of 2 (it expects 2 arguments). You can add 2 dummy arguments or use `-s application start mysample` instead. On Monday, March 5, 2012 at 10:07 PM, envelopes envelopes wrote: > > I have a sample program generated from rebar > > 1. mysample_app.erl > > -module(mysample_app). > > -behaviour(application). > > %% Application callbacks > -export([start/2, stop/1, main/1]). > > %% =================================================================== > %% Application callbacks > %% =================================================================== > > start(_StartType, _StartArgs) -> > mysample_sup:start_link(). > > stop(_State) -> > ok. > > > main([A]) -> > I = list_to_integer(atom_to_list(A)), > F = fac(I), > io:format("factorial ~w = ~w~n" ,[I, F]), > init:stop(). > fac(0) -> 1; > fac(N) -> N*fac(N-1). > > > 2. mysample_sup.erl > > > -module(mysample_sup). > > -behaviour(supervisor). > > %% API > -export([start_link/0]). > > %% Supervisor callbacks > -export([init/1]). > > %% Helper macro for declaring children of supervisor > -define(CHILD(I, Type), {I, {I, start_link, []}, permanent, 5000, Type, [I]}). > > %% =================================================================== > %% API functions > %% =================================================================== > > start_link() -> > supervisor:start_link({local, ?MODULE}, ?MODULE, []). > > %% =================================================================== > %% Supervisor callbacks > %% =================================================================== > > init([]) -> > {ok, { {one_for_one, 5, 10}, []} }. > > > 3) the problem. > erl -pa ebin -s mysample_app main 23 ===> worked fine > erl -pa ebin -s mysample_app start ===> crashed on boot. > > {"init terminating in do_boot",{undef,[{mysample_app,start,[],[]},{init,start_it > ,1,[{file,"init.erl"},{line,1042}]},{init,start_em,1,[{file,"init.erl"},{line,10 > 22}]}]}} > > Crash dump was written to: erl_crash.dump > init terminating in do_boot () > > what is the cause ? my erlang is the latest R15. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sunwood360@REDACTED Tue Mar 6 07:34:31 2012 From: sunwood360@REDACTED (envelopes envelopes) Date: Mon, 5 Mar 2012 22:34:31 -0800 Subject: [erlang-questions] dumb question from rookie In-Reply-To: <451378EF51F74C12B22B409C5884057D@gmail.com> References: <451378EF51F74C12B22B409C5884057D@gmail.com> Message-ID: weird. Why "erl -pa ebin -s mysample_app start x y" doesn't work, but "erl -pa ebin -s application start mysample" works, is it a bug in erlang? On Mon, Mar 5, 2012 at 10:21 PM, Ryan Huffman wrote: > It looks like you are trying to call your state function with 0 arguments, > but the start function has an arity of 2 (it expects 2 arguments). You can > add 2 dummy arguments or use `-s application start mysample` instead. > > On Monday, March 5, 2012 at 10:07 PM, envelopes envelopes wrote: > > > I have a sample program generated from rebar > > 1. mysample_app.erl > > -module(mysample_app). > > -behaviour(application). > > %% Application callbacks > -export([start/2, stop/1, main/1]). > > %% =================================================================== > %% Application callbacks > %% =================================================================== > > start(_StartType, _StartArgs) -> > mysample_sup:start_link(). > > stop(_State) -> > ok. > > > main([A]) -> > I = list_to_integer(atom_to_list(A)), > F = fac(I), > io:format("factorial ~w = ~w~n" ,[I, F]), > init:stop(). > fac(0) -> 1; > fac(N) -> N*fac(N-1). > > > 2. mysample_sup.erl > > > -module(mysample_sup). > > -behaviour(supervisor). > > %% API > -export([start_link/0]). > > %% Supervisor callbacks > -export([init/1]). > > %% Helper macro for declaring children of supervisor > -define(CHILD(I, Type), {I, {I, start_link, []}, permanent, 5000, Type, > [I]}). > > %% =================================================================== > %% API functions > %% =================================================================== > > start_link() -> > supervisor:start_link({local, ?MODULE}, ?MODULE, []). > > %% =================================================================== > %% Supervisor callbacks > %% =================================================================== > > init([]) -> > {ok, { {one_for_one, 5, 10}, []} }. > > > 3) the problem. > erl -pa ebin -s mysample_app main 23 ===> worked fine > erl -pa ebin -s mysample_app start ===> crashed on boot. > > {"init terminating in > do_boot",{undef,[{mysample_app,start,[],[]},{init,start_it > > ,1,[{file,"init.erl"},{line,1042}]},{init,start_em,1,[{file,"init.erl"},{line,10 > 22}]}]}} > > Crash dump was written to: erl_crash.dump > init terminating in do_boot () > > what is the cause ? my erlang is the latest R15. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ates@REDACTED Tue Mar 6 07:34:47 2012 From: ates@REDACTED (Artem Teslenko) Date: Tue, 6 Mar 2012 08:34:47 +0200 Subject: [erlang-questions] SNMP get,set,walk, etc In-Reply-To: <4F55A416.9070202@ghostgun.com> References: <4F55A416.9070202@ghostgun.com> Message-ID: <20120306063446.GA29324@mail.ipv6.dp.ua> Hi, Look at the repo below, it's a simple implementation of 'snmp walk': git://github.com/ates/snmpcl.git On Tue, 06 Mar 2012, jm wrote: > Are there any decent guides anyone can point me to that deal with using > Erlang to retrieve SNMP measurements from devices such as network > routers and switches? The stuff I've been able to dig up in the past > has dealt mostly with how to add a SNMP agent to your own applications. > > Jeff. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From eppa.ramu@REDACTED Tue Mar 6 08:29:26 2012 From: eppa.ramu@REDACTED (raamu eppa) Date: Tue, 6 Mar 2012 12:59:26 +0530 Subject: [erlang-questions] Crash Report Message-ID: Hi, when am running this command, ./bin/erl -boot releases/1.0/start Iam getting this error,how to solve this error. =CRASH REPORT==== 6-Mar-2012::12:50:20 === crasher: initial call: application_master:init/4 pid: <0.40.0> registered_name: [] exception exit: {bad_return, {{schedule,start,[normal,[]]}, {'EXIT', {undef, [{schedule,start,[normal,[]]}, {application_master,start_it_old,4}]}}}} in function application_master:init/4 ancestors: [<0.39.0>] messages: [{'EXIT',<0.41.0>,normal}] links: [<0.39.0>,<0.6.0>] dictionary: [] trap_exit: true status: running heap_size: 377 stack_size: 24 reductions: 108 neighbours: =INFO REPORT==== 6-Mar-2012::12:50:20 === application: emailmessage exited: {bad_return, {{schedule,start,[normal,[]]}, {'EXIT', {undef, [{schedule,start,[normal,[]]}, {application_master,start_it_old,4}]}}}} type: permanent Eshell V5.8.4 (abort with ^G) 1> {"Kernel pid terminated",application_controller,"{application_start_failure,emailmessage,{bad_return,{{schedule,start,[normal,[]]},{'EXIT',{undef,[{schedule,start,[normal,[]]},{application_master,start_it_old,4}]}}}}}"} Crash dump was written to: erl_crash.dump Kernel pid terminated (application_controller) ({application_start_failure,emailmessage,{bad_return,{{schedule,start,[normal,[]]},{'EXIT',{undef,[{schedule,start,[normal,[]]},{ap Thanks, Ramu. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sunwood360@REDACTED Tue Mar 6 08:32:25 2012 From: sunwood360@REDACTED (envelopes envelopes) Date: Mon, 5 Mar 2012 23:32:25 -0800 Subject: [erlang-questions] dumb question from rookie In-Reply-To: References: <451378EF51F74C12B22B409C5884057D@gmail.com> Message-ID: Another interesting thing is : though erl -pa ebin -s application start mysample works, mysample_app is not listed in loaded modules from shell command m(). ??? On Mon, Mar 5, 2012 at 10:34 PM, envelopes envelopes wrote: > weird. > > Why "erl -pa ebin -s mysample_app start x y" doesn't work, but "erl -pa > ebin -s application start mysample" works, > > is it a bug in erlang? > > > On Mon, Mar 5, 2012 at 10:21 PM, Ryan Huffman wrote: > >> It looks like you are trying to call your state function with 0 >> arguments, but the start function has an arity of 2 (it expects 2 >> arguments). You can add 2 dummy arguments or use `-s application start >> mysample` instead. >> >> On Monday, March 5, 2012 at 10:07 PM, envelopes envelopes wrote: >> >> >> I have a sample program generated from rebar >> >> 1. mysample_app.erl >> >> -module(mysample_app). >> >> -behaviour(application). >> >> %% Application callbacks >> -export([start/2, stop/1, main/1]). >> >> %% =================================================================== >> %% Application callbacks >> %% =================================================================== >> >> start(_StartType, _StartArgs) -> >> mysample_sup:start_link(). >> >> stop(_State) -> >> ok. >> >> >> main([A]) -> >> I = list_to_integer(atom_to_list(A)), >> F = fac(I), >> io:format("factorial ~w = ~w~n" ,[I, F]), >> init:stop(). >> fac(0) -> 1; >> fac(N) -> N*fac(N-1). >> >> >> 2. mysample_sup.erl >> >> >> -module(mysample_sup). >> >> -behaviour(supervisor). >> >> %% API >> -export([start_link/0]). >> >> %% Supervisor callbacks >> -export([init/1]). >> >> %% Helper macro for declaring children of supervisor >> -define(CHILD(I, Type), {I, {I, start_link, []}, permanent, 5000, Type, >> [I]}). >> >> %% =================================================================== >> %% API functions >> %% =================================================================== >> >> start_link() -> >> supervisor:start_link({local, ?MODULE}, ?MODULE, []). >> >> %% =================================================================== >> %% Supervisor callbacks >> %% =================================================================== >> >> init([]) -> >> {ok, { {one_for_one, 5, 10}, []} }. >> >> >> 3) the problem. >> erl -pa ebin -s mysample_app main 23 ===> worked fine >> erl -pa ebin -s mysample_app start ===> crashed on boot. >> >> {"init terminating in >> do_boot",{undef,[{mysample_app,start,[],[]},{init,start_it >> >> ,1,[{file,"init.erl"},{line,1042}]},{init,start_em,1,[{file,"init.erl"},{line,10 >> 22}]}]}} >> >> Crash dump was written to: erl_crash.dump >> init terminating in do_boot () >> >> what is the cause ? my erlang is the latest R15. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From loris.fichera@REDACTED Tue Mar 6 09:23:32 2012 From: loris.fichera@REDACTED (Loris Fichera) Date: Tue, 06 Mar 2012 09:23:32 +0100 Subject: [erlang-questions] mochi/erl_img In-Reply-To: <20120305155123.22bbfdcf.nx@nu-ex.com> References: <4F54E5CD.8040900@gmail.com> <20120305155123.22bbfdcf.nx@nu-ex.com> Message-ID: <4F55C984.5080701@gmail.com> On 03/05/2012 09:51 PM, nx wrote: > You may also want to check out erl_gm, which is an Erlang wrapper for the GraphicsMagick command line application: > > https://github.com/nuex/erl_gm > > It supports getting all image characteristics defined in GraphicsMagick "format" option and currently has a few image manipulation options like crop, resize, magnify, blur and more. Thanks a lot. Actually, I need to access the pixels values, as far as I know GraphicsMagick hasn't got a facility to do so via the command line. Loris. From watson.timothy@REDACTED Tue Mar 6 11:11:14 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 6 Mar 2012 10:11:14 +0000 Subject: [erlang-questions] dumb question from rookie In-Reply-To: References: <451378EF51F74C12B22B409C5884057D@gmail.com> Message-ID: <57E2C777-88A1-4226-9667-45766D2B06F0@gmail.com> Hi there, I'm not sure about this second issue (of the _app module not being loaded in the shell) but I can explain the former. When you run the 'erl' program with -s you are telling the built in 'init' module to run the following module, function with optional arguments. When you pass '-s mysample_app start' you are telling the init module to call that function, but 'init' is a special module, part of the runtime system. Your module isn't special, so the code server hasn't loaded it yet and 'init' can't find it so it is 'undefined' hence the error message. Now when you call '-s application start mysample' you're telling 'init' to run application:start(mysample) where application is a module in the 'kernel' application (which is loaded by the runtime system before everything else) and 'mysample' is an atom and the name of the application you're asking to start. The application:start/1 function uses the code server to find the application resource file (mysample.app) and load the required modules and then calls 'mysample_app:start(Type, Args)' for you, which sets up the top level supervisor (process) and so on and so forth. Hope that makes a bit better sense of it. Cheers, Tim On 6 Mar 2012, at 07:32, envelopes envelopes wrote: > Another interesting thing is : though erl -pa ebin -s application start mysample works, mysample_app is not listed in loaded modules from shell command m(). > > ??? > > > > On Mon, Mar 5, 2012 at 10:34 PM, envelopes envelopes wrote: > weird. > > Why "erl -pa ebin -s mysample_app start x y" doesn't work, but "erl -pa ebin -s application start mysample" works, > > is it a bug in erlang? > > > On Mon, Mar 5, 2012 at 10:21 PM, Ryan Huffman wrote: > It looks like you are trying to call your state function with 0 arguments, but the start function has an arity of 2 (it expects 2 arguments). You can add 2 dummy arguments or use `-s application start mysample` instead. > On Monday, March 5, 2012 at 10:07 PM, envelopes envelopes wrote: > >> >> I have a sample program generated from rebar >> >> 1. mysample_app.erl >> >> -module(mysample_app). >> >> -behaviour(application). >> >> %% Application callbacks >> -export([start/2, stop/1, main/1]). >> >> %% =================================================================== >> %% Application callbacks >> %% =================================================================== >> >> start(_StartType, _StartArgs) -> >> mysample_sup:start_link(). >> >> stop(_State) -> >> ok. >> >> >> main([A]) -> >> I = list_to_integer(atom_to_list(A)), >> F = fac(I), >> io:format("factorial ~w = ~w~n" ,[I, F]), >> init:stop(). >> fac(0) -> 1; >> fac(N) -> N*fac(N-1). >> >> >> 2. mysample_sup.erl >> >> >> -module(mysample_sup). >> >> -behaviour(supervisor). >> >> %% API >> -export([start_link/0]). >> >> %% Supervisor callbacks >> -export([init/1]). >> >> %% Helper macro for declaring children of supervisor >> -define(CHILD(I, Type), {I, {I, start_link, []}, permanent, 5000, Type, [I]}). >> >> %% =================================================================== >> %% API functions >> %% =================================================================== >> >> start_link() -> >> supervisor:start_link({local, ?MODULE}, ?MODULE, []). >> >> %% =================================================================== >> %% Supervisor callbacks >> %% =================================================================== >> >> init([]) -> >> {ok, { {one_for_one, 5, 10}, []} }. >> >> >> 3) the problem. >> erl -pa ebin -s mysample_app main 23 ===> worked fine >> erl -pa ebin -s mysample_app start ===> crashed on boot. >> >> {"init terminating in do_boot",{undef,[{mysample_app,start,[],[]},{init,start_it >> ,1,[{file,"init.erl"},{line,1042}]},{init,start_em,1,[{file,"init.erl"},{line,10 >> 22}]}]}} >> >> Crash dump was written to: erl_crash.dump >> init terminating in do_boot () >> >> what is the cause ? my erlang is the latest R15. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Tue Mar 6 11:46:14 2012 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 6 Mar 2012 10:46:14 +0000 Subject: [erlang-questions] Crash Report In-Reply-To: References: Message-ID: It looks like the call to schedule:start(normal, []) fails because no function schedule:start/2 is defined. On Tue, Mar 6, 2012 at 7:29 AM, raamu eppa wrote: > Hi, > when am running this command, > ./bin/erl -boot releases/1.0/start > Iam getting this error,how to solve this error. > > =CRASH REPORT==== 6-Mar-2012::12:50:20 === > crasher: > initial call: application_master:init/4 > pid: <0.40.0> > registered_name: [] > exception exit: {bad_return, > {{schedule,start,[normal,[]]}, > {'EXIT', > {undef, > [{schedule,start,[normal,[]]}, > {application_master,start_it_old,4}]}}}} > in function application_master:init/4 > ancestors: [<0.39.0>] > messages: [{'EXIT',<0.41.0>,normal}] > links: [<0.39.0>,<0.6.0>] > dictionary: [] > trap_exit: true > status: running > heap_size: 377 > stack_size: 24 > reductions: 108 > neighbours: > > =INFO REPORT==== 6-Mar-2012::12:50:20 === > application: emailmessage > exited: {bad_return, > {{schedule,start,[normal,[]]}, > {'EXIT', > {undef, > [{schedule,start,[normal,[]]}, > {application_master,start_it_old,4}]}}}} > type: permanent > Eshell V5.8.4 (abort with ^G) > 1> {"Kernel pid > terminated",application_controller,"{application_start_failure,emailmessage,{bad_return,{{schedule,start,[normal,[]]},{'EXIT',{undef,[{schedule,start,[normal,[]]},{application_master,start_it_old,4}]}}}}}"} > > Crash dump was written to: erl_crash.dump > Kernel pid terminated (application_controller) > ({application_start_failure,emailmessage,{bad_return,{{schedule,start,[normal,[]]},{'EXIT',{undef,[{schedule,start,[normal,[]]},{ap > > Thanks, > Ramu. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Tue Mar 6 11:46:34 2012 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 6 Mar 2012 11:46:34 +0100 Subject: [erlang-questions] gen_sctp throughput mystery In-Reply-To: <201203041319.00947.als@iinet.net.au> References: <201203041237.03563.als@iinet.net.au> <201203041243.49277.als@iinet.net.au> <201203041319.00947.als@iinet.net.au> Message-ID: <20120306104634.GA13185@erix.ericsson.se> On Sun, Mar 04, 2012 at 01:19:00PM +1100, Anthony Shipman wrote: > On Sun, 4 Mar 2012 12:43:49 pm Anthony Shipman wrote: > > > The now() value shows the packets being written to the socket at 11ms > > > intervals on the client but they are received at strict 200ms intervals > > > on the server. The server is just doing a simple receive loop. > > > > > > Is there something in SCTP that limits the rate on an SCTP stream? > > > > I forgot to mention that the behaviour is the same for sctp_nodelay=true. > > > > Replying to my own question. It's the net.sctp.sack_timeout = 200 kernel > parameter. But I don't understand why it should limit the server to 5 packets > per second over a localhost link. Great finding if true... Can you elaborate? What happens when you tweak this parameter? Note that 1 / 200ms is exactly 5 per second. I found this link: http://sandarenu.blogspot.com/2009/11/reducing-time-between-sctp-message.html And a quote from it it says: That 200ms is a configuration based on the SCTP RFC. According to that an acknowledgment(SACK) should be generated for at least every second packet received, and SHOULD be generated within 200 ms of the arrival of any unacknowledged DATA chunk. So, might it be so that if you send just one data chunk there will be a 200 ms delay before an ack? I have focused on the SCTP socket API, not dug into the RFC:s... > > -- > Anthony Shipman Mamas don't let your babies > als@REDACTED grow up to be outsourced. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Tue Mar 6 11:57:20 2012 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 6 Mar 2012 11:57:20 +0100 Subject: [erlang-questions] gen_sctp doesn't stream In-Reply-To: <201203051011.32675.als@iinet.net.au> References: <201203051011.32675.als@iinet.net.au> Message-ID: <20120306105720.GB13185@erix.ericsson.se> On Mon, Mar 05, 2012 at 10:11:32AM +1100, Anthony Shipman wrote: > gen_tcp does let me stream data. The driver will put data into a queue and > block my process when the queue reaches a high water mark. Then it will allow > the process to continue when the low water mark is reached. I can push data > into a socket continuously as fast as it will go. > > The gen_sctp driver is built on the UDP driver. There is no queue. If I send > too many packets I just get an eagain error code. It seems that the best I can > do is to keep polling until I can send the next message. Am I right? Yes, unfortunately. So far that has not been an issue, but since SCTP in this respect behaves more like TCP it would be desirable to have the same behaviour. I will put it on our ToDo list... > > -- > Anthony Shipman Mamas don't let your babies > als@REDACTED grow up to be outsourced. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From eppa.ramu@REDACTED Tue Mar 6 12:14:46 2012 From: eppa.ramu@REDACTED (raamu eppa) Date: Tue, 6 Mar 2012 16:44:46 +0530 Subject: [erlang-questions] Crash Report Message-ID: Hi, But iam starting as ,/usr/local/otp_src_R14B03/target/bin/erl, It was working fine. Thanks, Ramu. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hm@REDACTED Tue Mar 6 12:21:21 2012 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Tue, 6 Mar 2012 12:21:21 +0100 Subject: [erlang-questions] support for .ez archives In-Reply-To: References: Message-ID: On Mon, Mar 5, 2012 at 23:54, Tim Watson wrote: > Hello list, > > I have a few questions about support for .ez archives, and one or two > suggestions as well. > > - is a timeline for fully supporting them? > - does the -include_lib(...) directive now work (in R15) when the headers > are in an archive? > Bj?rn, is there a timeline? > - does code:lib_dir/2 work properly with archives? > Yes. > > As a suggestion: I've noticed that the way most people build an escript > binary is to glue everything together into a zip file with the appropriate > header content and I wonder whether having 'executable (.ez) archives' > would be a better approach? Perhaps this works today and I haven't figured > it out? What are peoples thoughts about this? > > The current trick with the shebang header works on many platforms. It would however be nice with executable archives (without any shebang), but these solutions tend to be more platform specific (non-portable). /H?kan -------------- next part -------------- An HTML attachment was scrubbed... URL: From baliulia@REDACTED Tue Mar 6 12:31:04 2012 From: baliulia@REDACTED (=?UTF-8?B?SWduYXMgVnnFoW5pYXVza2Fz?=) Date: Tue, 06 Mar 2012 11:31:04 +0000 Subject: [erlang-questions] EUnit not failing test cases which timeout Message-ID: <4F55F578.6000806@gmail.com> Hi, if I have a test case which is timing out for some reason, EUnit will not consider that as a failure, i.e. a module like this: -module(foobar). -include_lib("eunit/include/eunit.hrl"). timing_out_test() -> timer:sleep(10001). will produce this: ======================== EUnit ======================== foobar: timing_out_test (module 'foobar')...*timed out* undefined ======================================================= Failed: 0. Skipped: 0. Passed: 0. One or more tests were cancelled. ERROR: One or more eunit tests failed. As you can see in the last 3 lines, there is a bunch of information which contradicts itself. Terrible! On the other hand, eunit_surefire will generate a report which claims the test was skipped: Why is timing out being considered a skipped test case rather than a failed one? Time-out's happen quite a lot in Erlang, but due to this, one might not experience the failure. I believe this is wrong and tests which time-out should be marked as failures. It would be good to have any feedback on this. Thanks, Ignas P.S. Maybe I should post this to erlang-bugs instead? From kenneth.lundin@REDACTED Tue Mar 6 13:02:52 2012 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 6 Mar 2012 13:02:52 +0100 Subject: [erlang-questions] support for .ez archives In-Reply-To: References: Message-ID: 2012/3/6 H?kan Mattsson > On Mon, Mar 5, 2012 at 23:54, Tim Watson wrote: > >> Hello list, >> >> I have a few questions about support for .ez archives, and one or two >> suggestions as well. >> >> - is a timeline for fully supporting them? >> - does the -include_lib(...) directive now work (in R15) when the headers >> are in an archive? >> > > Bj?rn, is there a timeline? > There is no timeline and it is even questionable if it should ever work. What is the argument for having the include files inside the archive? Include files are for off-line use during the build process. The ebin directory is for runtime use and the contents could be inside an archive. The privdir is also for runtime use but must be possible to open from the OS and is therefore not suitable to have inside the archive. /Kenneth Erlang/OTP, Ericsson > > > >> - does code:lib_dir/2 work properly with archives? >> > > Yes. > > >> >> As a suggestion: I've noticed that the way most people build an escript >> binary is to glue everything together into a zip file with the appropriate >> header content and I wonder whether having 'executable (.ez) archives' >> would be a better approach? Perhaps this works today and I haven't figured >> it out? What are peoples thoughts about this? >> >> > The current trick with the shebang header works on many platforms. > It would however be nice with executable archives (without any > shebang), but these solutions tend to be more platform specific > (non-portable). > > /H?kan > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dvdougdv@REDACTED Tue Mar 6 13:20:23 2012 From: dvdougdv@REDACTED (douglas) Date: Tue, 06 Mar 2012 07:20:23 -0500 Subject: [erlang-questions] erlang types Message-ID: <4F560107.2080403@gmail.com> I have been reading here and there about erlang types. My thoughts are that anything dynamic is a moving target. But even moving targets occupy space in time, a camera witnessing a race horse's moment time/space occupation while at the kentucky derby. This obviously is not the same space occupation as the next moment/s. So, it seems to me that it's all about the "Witness's" "Bookkeeping" of "Moments" in this case the camera. The Bookkeeping meta-data, includes a fixed position of pixels for only that moment of time occupation which clearly will not be the same for the next moment.How, would you call a particular moment?Would you want too? So it seems that what is legal and illegal is important; a list can't end in a term etc if you want the list to end in a term then a new class would be needed say "Perfidy;"Container of "Perfidies". which would have to define the exceptional use cases. So that additional "bookkeeping/diary" Snapshot of time would be presented to the compiler with no archive available or wanted, it is a fleeting moment. After all it's Dynamic! From ulf@REDACTED Tue Mar 6 13:20:43 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 6 Mar 2012 13:20:43 +0100 Subject: [erlang-questions] support for .ez archives In-Reply-To: References: Message-ID: On 6 Mar 2012, at 13:02, Kenneth Lundin wrote: > There is no timeline and it is even questionable if it should ever work. What is the argument > for having the include files inside the archive? I ran into this problem, and made an "OTP patch" for it: http://article.gmane.org/gmane.comp.lang.erlang.general/52060 (Don't think I ever submitted it as a patch, since H?kan and I disagreed on some of the implementation details). I wrote: "The idea (see http://github.com/esl/run_eqc) was that it should be possible to deliver an application as a single archive, supporting command-line operations, interactive code loading, and compilation including hrl files, parse_transforms etc., without having to unpack the archive." Philip Stanhope wrote: "Very cool. I can see how this will make deployments very easy." http://article.gmane.org/gmane.comp.lang.erlang.general/52061 Dizzy writes: "+1 on this feature -- it opens some very interesting doors." -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre@REDACTED Tue Mar 6 13:40:01 2012 From: andre@REDACTED (=?ISO-8859-1?Q?Andr=E9_Graf?=) Date: Tue, 6 Mar 2012 13:40:01 +0100 Subject: [erlang-questions] EUnit not failing test cases which timeout In-Reply-To: <4F55F578.6000806@gmail.com> References: <4F55F578.6000806@gmail.com> Message-ID: Hi Ignas You can use the timeout control in eunit and specify how long your testcases may take to complete e.g. -module(foobar). -include_lib("eunit/include/eunit.hrl"). timing_out_test_() -> { setup, fun() ->ok end, [ {timeout, 1, fun() -> timer:sleep(10000) end} ] }. In the example above your testcase may not take longer than one second, since it obviously does it will fail. I don't know another way than using a test suite tough, maybe there are other hints on that. Cheers, Andre 2012/3/6 Ignas Vy?niauskas : > Hi, > > if I have a test case which is timing out for some reason, EUnit will > not consider that as a failure, i.e. a module like this: > > -module(foobar). > -include_lib("eunit/include/eunit.hrl"). > > timing_out_test() -> > ? ?timer:sleep(10001). > > will produce this: > > ======================== EUnit ======================== > foobar: timing_out_test (module 'foobar')...*timed out* > undefined > ======================================================= > ?Failed: 0. ?Skipped: 0. ?Passed: 0. > One or more tests were cancelled. > ERROR: One or more eunit tests failed. > > As you can see in the last 3 lines, there is a bunch of information > which contradicts itself. Terrible! > > On the other hand, eunit_surefire will generate a report which claims > the test was skipped: > > name="module 'foobar'"> > ? description="module 'foobar'"> > ? ? > ? ? > ? ? > ? > > > Why is timing out being considered a skipped test case rather than a > failed one? Time-out's happen quite a lot in Erlang, but due to this, > one might not experience the failure. I believe this is wrong and tests > which time-out should be marked as failures. > > It would be good to have any feedback on this. > > Thanks, > Ignas > > P.S. Maybe I should post this to erlang-bugs instead? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From baliulia@REDACTED Tue Mar 6 14:03:58 2012 From: baliulia@REDACTED (=?UTF-8?B?SWduYXMgVnnFoW5pYXVza2Fz?=) Date: Tue, 06 Mar 2012 13:03:58 +0000 Subject: [erlang-questions] EUnit not failing test cases which timeout In-Reply-To: References: <4F55F578.6000806@gmail.com> Message-ID: <4F560B3E.4020808@gmail.com> Hi Andre, On 2012.03.06 12:40, Andr? Graf wrote: > You can use the timeout control in eunit and specify how long your > testcases may take to complete e.g. thanks for your answer, but I think you misunderstood my question a bit, perhaps my code example is a bit confusing. The timer:sleep(10001) is only a proof of concept, but I don't actually intend the test case to run for a particular time and I don't know beforehand that it will take X seconds to complete. I want it to fail nevertheless if it takes too long. I think this is simply wrong behaviour in EUnit's code, I shouldn't be forced to explicitly add a {timeout, _, _} fixture to each test case to avoid this problem. Ignas From dawid.figiel@REDACTED Tue Mar 6 14:25:04 2012 From: dawid.figiel@REDACTED (Dawid Figiel) Date: Tue, 6 Mar 2012 14:25:04 +0100 Subject: [erlang-questions] Raspberry-Pi In-Reply-To: References: <064E04AC-7408-4934-91A6-BA0D6D495592@gmail.com> Message-ID: Something to compare with : MMnet1002 - 200MHz, 64MB RAM, 1GB Flash http://www.shop.propox.com/index.php?d=produkt&id=2119 This one is more expensive 86 EUR... but maybe it is better or maybe not. Can you find the specs for Raspberry-Pi ? //Dawid Figiel On 3/2/12, Wes James wrote: > On Fri, Mar 2, 2012 at 4:13 AM, Michael Turner > wrote: >>> ... a $35 Erlang machine with Ethernet ... >> >> --if based on Raspberry-Pi, would be called "Raspberry-e", wouldn't it? > > How about Raspberry-PiE > >> >> -michael turner > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From eppa.ramu@REDACTED Tue Mar 6 14:54:26 2012 From: eppa.ramu@REDACTED (raamu eppa) Date: Tue, 6 Mar 2012 19:24:26 +0530 Subject: [erlang-questions] Crash Report Message-ID: when running boot command am getting error. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth.lundin@REDACTED Tue Mar 6 15:03:28 2012 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 6 Mar 2012 15:03:28 +0100 Subject: [erlang-questions] support for .ez archives In-Reply-To: References: Message-ID: What I meant was that I am not convinced that it is necessary and very useful to have the include files inside an archive (while using them). But it might be possible to convince me :) /Kenneth On Tue, Mar 6, 2012 at 1:20 PM, Ulf Wiger wrote: > > On 6 Mar 2012, at 13:02, Kenneth Lundin wrote: > > There is no timeline and it is even questionable if it should ever work. > What is the argument > for having the include files inside the archive? > > > I ran into this problem, and made an "OTP patch" for it: > > http://article.gmane.org/gmane.comp.lang.erlang.general/52060 > > (Don't think I ever submitted it as a patch, since H?kan and I disagreed > on some of the implementation details). > > I wrote: "The idea (see http://github.com/esl/run_eqc) was that it should > be possible to deliver an application as a single archive, supporting > command-line operations, interactive code loading, and compilation > including hrl files, parse_transforms etc., without having to unpack the > archive." > > Philip Stanhope wrote: "Very cool. I can see how this will make > deployments very easy." > > http://article.gmane.org/gmane.comp.lang.erlang.general/52061 > > Dizzy writes: "+1 on this feature -- it opens some very interesting doors." > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Tue Mar 6 15:17:36 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 6 Mar 2012 15:17:36 +0100 Subject: [erlang-questions] support for .ez archives In-Reply-To: References: Message-ID: <2127335B-24C9-4618-BDBB-9413646ED794@feuerlabs.com> On 6 Mar 2012, at 15:03, Kenneth Lundin wrote: > What I meant was that I am not convinced that it is necessary and very useful to have the include files > inside an archive (while using them). > > But it might be possible to convince me :) > In the case of run_eqc, the idea was to be able to have either PropEr or EQC Lite available as zipped archives. This worked beautifully, at least in terms of executing them from within the zip file. However, in order to compile the test code that is supposed to be executed by PropEr/EQC, you need the include file, and this became the _only_ reason why the archive had to be unpacked. And given that you _had_ to extract the .hrl, there wasn't much point in not also extracting the beam files. But this made the whole idea of run_eqc less attractive, so I put the project on ice. In this setting, there is really not much difference between run-time and compile-time. Running EQC without being able to first compile with the help of eqc.hrl is fairly useless. BR, Ulf W -------------- next part -------------- An HTML attachment was scrubbed... URL: From anders.nygren@REDACTED Tue Mar 6 15:42:28 2012 From: anders.nygren@REDACTED (Anders Nygren) Date: Tue, 6 Mar 2012 08:42:28 -0600 Subject: [erlang-questions] support for .ez archives In-Reply-To: References: Message-ID: On Tue, Mar 6, 2012 at 8:03 AM, Kenneth Lundin wrote: > What I meant was that I am not convinced that it is necessary and very > useful to have the include files > inside an archive (while using them). > > But it might be possible to convince me :) Yaws and gettext requires some .hrl files for compiling dynamic pages. I think there may be other cases where code is compiled during runtime where it is useful to have access to the .hrl files. /Anders > > /Kenneth > > > On Tue, Mar 6, 2012 at 1:20 PM, Ulf Wiger wrote: >> >> >> On 6 Mar 2012, at 13:02, Kenneth Lundin wrote: >> >> There is no timeline and it is even questionable if it should ever work. >> What is the argument >> for having the include files inside the archive? >> >> >> I ran into this problem, and made an "OTP patch" for it: >> >> http://article.gmane.org/gmane.comp.lang.erlang.general/52060 >> >> (Don't think I ever submitted it as a patch, since H?kan and I disagreed >> on some of the implementation details). >> >> I wrote: "The idea (see?http://github.com/esl/run_eqc) was that it should >> be possible to deliver an?application as a single archive, supporting >> command-line operations,?interactive code loading, and compilation including >> hrl files,?parse_transforms etc., without having to unpack the archive." >> >> Philip Stanhope wrote: "Very cool. I can see how this will make >> deployments very easy." >> >> http://article.gmane.org/gmane.comp.lang.erlang.general/52061 >> >> Dizzy writes: "+1 on this feature -- it opens some very interesting >> doors." >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From gbulmer@REDACTED Tue Mar 6 16:24:47 2012 From: gbulmer@REDACTED (G Bulmer) Date: Tue, 6 Mar 2012 15:24:47 +0000 Subject: [erlang-questions] Raspberry-Pi In-Reply-To: References: <064E04AC-7408-4934-91A6-BA0D6D495592@gmail.com> Message-ID: Raspberry-pi specs are at: http://elinux.org/Rpi_Hardware Essentially - 700MHz ARM11 - 256MBytes RAM - 10/100 Ethernet - SD memory card cost about 27 EUR (under 1/3rd the price of that MMnet1002) I think Raspberry-Pi (R-Pi) is relevant to Erlang because: 1. It appears that there is a *lot* of educational interest in using R-Pi to teach programming, so having Erlang available makes a lot of sense (to me) 2. it might be a cheap, portable server for development 'on-the-go' for folks doing laptop development, without VM-infrastructure, 3. it might be very compact, cheap, approach to guaranteed-working development machines, for folks running Erlang training courses, (at that price, it might be included in the course fees) 4. it's so small and cheap, I could post a 'working server', or even a small cluster, to a client for testing, etc. GB-) On Tue, Mar 6, 2012 at 1:25 PM, Dawid Figiel wrote: > Something to compare with : > > MMnet1002 - 200MHz, 64MB RAM, 1GB Flash > > http://www.shop.propox.com/index.php?d=produkt&id=2119 > > This one is more expensive 86 EUR... but maybe it is better or maybe not. > > Can you find the specs for Raspberry-Pi ? > > //Dawid Figiel > > > On 3/2/12, Wes James wrote: > > On Fri, Mar 2, 2012 at 4:13 AM, Michael Turner > > wrote: > >>> ... a $35 Erlang machine with Ethernet ... > >> > >> --if based on Raspberry-Pi, would be called "Raspberry-e", wouldn't it? > > > > How about Raspberry-PiE > > > >> > >> -michael turner > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dfkettle@REDACTED Tue Mar 6 16:33:57 2012 From: dfkettle@REDACTED (David) Date: Tue, 6 Mar 2012 10:33:57 -0500 Subject: [erlang-questions] Raspberry-Pi In-Reply-To: References: <064E04AC-7408-4934-91A6-BA0D6D495592@gmail.com> Message-ID: There are two models of the RPi, Model A and Model B, with slightly different specs and pricing. See http://www.thestar.com/business/article/1138709--35-computer-with-toronto-designed-software-sells-out-worldwide-in-minutes?bn=1 and http://www.raspberrypi.org/ David. On Tue, Mar 6, 2012 at 10:24 AM, G Bulmer wrote: > Raspberry-pi specs are at: http://elinux.org/Rpi_Hardware > > Essentially > - 700MHz ARM11 > - 256MBytes RAM > - 10/100 Ethernet > - SD memory card > > cost about 27 EUR (under 1/3rd the price of that MMnet1002) > > I think Raspberry-Pi (R-Pi) is relevant to Erlang because: > 1. It appears that there is a *lot* of educational interest in using R-Pi to > teach programming, so having Erlang available makes a lot of sense (to me) > 2. it might be a cheap, portable server for development 'on-the-go' for > folks doing laptop development, without VM-infrastructure, > 3. it might be very compact, cheap, approach to guaranteed-working > development machines, for folks running Erlang training courses, (at that > price, it might be included in the course fees) > 4. it's so small and cheap, I could post a 'working server', or even a small > cluster, to a client for testing, etc. > > GB-) > > > On Tue, Mar 6, 2012 at 1:25 PM, Dawid Figiel wrote: >> >> Something to compare with : >> >> MMnet1002 - 200MHz, 64MB RAM, 1GB Flash >> >> http://www.shop.propox.com/index.php?d=produkt&id=2119 >> >> This one is more expensive 86 EUR... but maybe it is better or maybe not. >> >> Can you find the specs for Raspberry-Pi ? >> >> //Dawid Figiel >> >> >> On 3/2/12, Wes James wrote: >> > On Fri, Mar 2, 2012 at 4:13 AM, Michael Turner >> > wrote: >> >>> ... a $35 Erlang machine with Ethernet ... >> >> >> >> --if based on Raspberry-Pi, would be called "Raspberry-e", wouldn't it? >> > >> > How about Raspberry-PiE >> > >> >> >> >> -michael turner >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From fredrik.svahn@REDACTED Tue Mar 6 16:36:29 2012 From: fredrik.svahn@REDACTED (Fredrik Svahn) Date: Tue, 6 Mar 2012 16:36:29 +0100 Subject: [erlang-questions] Running distributed tests with common test In-Reply-To: References: Message-ID: Hi, I have a setup where I am using ct_master:run_test/2 to run tests on target nodes which each have a local file system. On the server I get the following html result pages: master_runs.html ct_master_run.2012-03-06_14.17.16/index.html The links on the latter page are however broken, since they refer to files on each target's local file system. In the Users Guide it is stated: "The CT Master server writes progress information to HTML log files similarly to the regular CT server. The logs contain test statistics and links to the log files written by each independent CT server" which seems to support that what I am seeing is indeed the intended behaviour. Nevertheless, since I know CT is used for a lot of embedded testing I am wondering if there is a way of having the test targets report back *all* results and logs to the master server (as opposed to writing the results locally)? Would writing a custom event handler help? BR /Fredrik PS. this is the full command I use to start the tests, just in case I have missed some simple config option: ct_master:run_test(target@REDACTED, [{suite,[array_SUITE]},{auto_compile, false}, {basic_html, true}, batch]). -------------- next part -------------- An HTML attachment was scrubbed... URL: From gbulmer@REDACTED Tue Mar 6 16:45:28 2012 From: gbulmer@REDACTED (G Bulmer) Date: Tue, 6 Mar 2012 15:45:28 +0000 Subject: [erlang-questions] Raspberry-Pi In-Reply-To: References: <064E04AC-7408-4934-91A6-BA0D6D495592@gmail.com> Message-ID: Only R-Pi model B were being sold last week, and AFAIK is model B is the only one that Farnell and RS were taking pre-orders for (I was only offered a B). A is aimed to cost $25, but has no ethernet making it less useful (to me) as an Erlang machine. GB On Tue, Mar 6, 2012 at 3:33 PM, David wrote: > There are two models of the RPi, Model A and Model B, with slightly > different specs and pricing. See > > > http://www.thestar.com/business/article/1138709--35-computer-with-toronto-designed-software-sells-out-worldwide-in-minutes?bn=1 > > and > > http://www.raspberrypi.org/ > > David. > > On Tue, Mar 6, 2012 at 10:24 AM, G Bulmer wrote: > > Raspberry-pi specs are at: http://elinux.org/Rpi_Hardware > > > > Essentially > > - 700MHz ARM11 > > - 256MBytes RAM > > - 10/100 Ethernet > > - SD memory card > > > > cost about 27 EUR (under 1/3rd the price of that MMnet1002) > > > > I think Raspberry-Pi (R-Pi) is relevant to Erlang because: > > 1. It appears that there is a *lot* of educational interest in using > R-Pi to > > teach programming, so having Erlang available makes a lot of sense (to > me) > > 2. it might be a cheap, portable server for development 'on-the-go' for > > folks doing laptop development, without VM-infrastructure, > > 3. it might be very compact, cheap, approach to guaranteed-working > > development machines, for folks running Erlang training courses, (at that > > price, it might be included in the course fees) > > 4. it's so small and cheap, I could post a 'working server', or even a > small > > cluster, to a client for testing, etc. > > > > GB-) > > > > > > On Tue, Mar 6, 2012 at 1:25 PM, Dawid Figiel > wrote: > >> > >> Something to compare with : > >> > >> MMnet1002 - 200MHz, 64MB RAM, 1GB Flash > >> > >> http://www.shop.propox.com/index.php?d=produkt&id=2119 > >> > >> This one is more expensive 86 EUR... but maybe it is better or maybe > not. > >> > >> Can you find the specs for Raspberry-Pi ? > >> > >> //Dawid Figiel > >> > >> > >> On 3/2/12, Wes James wrote: > >> > On Fri, Mar 2, 2012 at 4:13 AM, Michael Turner > >> > wrote: > >> >>> ... a $35 Erlang machine with Ethernet ... > >> >> > >> >> --if based on Raspberry-Pi, would be called "Raspberry-e", wouldn't > it? > >> > > >> > How about Raspberry-PiE > >> > > >> >> > >> >> -michael turner > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Tue Mar 6 17:29:57 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 6 Mar 2012 16:29:57 +0000 Subject: [erlang-questions] support for .ez archives In-Reply-To: References: Message-ID: On 6 Mar 2012, at 14:42, Anders Nygren wrote: > On Tue, Mar 6, 2012 at 8:03 AM, Kenneth Lundin wrote: >> What I meant was that I am not convinced that it is necessary and very >> useful to have the include files >> inside an archive (while using them). >> >> But it might be possible to convince me :) > > > Yaws and gettext requires some .hrl files for compiling dynamic pages. > I think there may be other cases where code is compiled during runtime > where it is useful to have access to the .hrl files. > > /Anders > In general, I think .ez archives are an ideal unit of packaging and distribution (as dependencies) and therefore making them fully self contained is very helpful. If you specify 'lager >= 1.0.2' as a dependency and all I have to do is find and fetch the archive (and check the MD5 and whatnot) then stick it somewhere useful on the machine, the subsequent build (handling of dependent packages), release assembly and other things just work really nicely. Having to 'manually' pull the includes out before being able to utilise the archive as a compile time dependency is just a bit annoying for tools authors that's all. From what Ulf and Anders have mentioned, it sounds like there are other use cases too. Some other thoughts: If you want to be able to bundle an executable archive, are you at the point where you are going to want to bundle multiple applications in it (like it was a release) perhaps? This is certainly how bundled escripts seem to work (in terms of having lots of modules in them) although admittedly the 'application' concept doesn't really exist within those any more. Also, there is a 'where_is_file' function in the code module, which is quite useful for things that are in 'known' places. What would be a lovely increment beyond that, would be a way to resolve a resource path within a 'known' place inside an archive (such as an application's priv dir and so on) to a binary without having to unpack the whole archive. For loading templates and config files and the like, not having to unpack is really useful and this feels like the kind of thing you'd want to have done properly in one place, rather than lots of people implementing different mechanisms to do the same thing. Sure this isn't completely essential, but it's the kind of thing that avoids 'fiddling around' with the packages post installation and makes the feature more useful and accessible to developers - much like Java's ClassLoader.getClassPathResource(String) or .NET's equivalent. I do realise that loading shared libraries from inside of archives isn't feasible though, so there are obviously some cases where you've got to extract at least part of the contents onto the file system. Cheers, Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From sunwood360@REDACTED Tue Mar 6 17:39:01 2012 From: sunwood360@REDACTED (envelopes envelopes) Date: Tue, 6 Mar 2012 08:39:01 -0800 Subject: [erlang-questions] dumb question from rookie In-Reply-To: <57E2C777-88A1-4226-9667-45766D2B06F0@gmail.com> References: <451378EF51F74C12B22B409C5884057D@gmail.com> <57E2C777-88A1-4226-9667-45766D2B06F0@gmail.com> Message-ID: On Mar 6, 2012 2:11 AM, "Tim Watson" wrote: > > Hi there, > > I'm not sure about this second issue (of the _app module not being loaded in the shell) but I can explain the former. When you run the 'erl' program with -s you are telling the built in 'init' module to run the following module, function with optional arguments. When you pass '-s mysample_app start' you are telling the init module to call that function, but 'init' is a special module, part of the runtime system. Your module isn't special, so the code server hasn't loaded it yet and 'init' can't find it so it is 'undefined' hence the error message. how come ' erl -pa ebin -s mysample_app main 23' works? If the code server has not loaded it yet. The behavior is not consistent. > > Now when you call '-s application start mysample' you're telling 'init' to run application:start(mysample) where application is a module in the 'kernel' application (which is loaded by the runtime system before everything else) and 'mysample' is an atom and the name of the application you're asking to start. The application:start/1 function uses the code server to find the application resource file (mysample.app) and load the required modules and then calls 'mysample_app:start(Type, Args)' for you, which sets up the top level supervisor (process) and so on and so forth. > > Hope that makes a bit better sense of it. > > Cheers, > > Tim > > On 6 Mar 2012, at 07:32, envelopes envelopes wrote: > >> Another interesting thing is : though erl -pa ebin -s application start mysample works, mysample_app is not listed in loaded modules from shell command m(). >> >> ??? >> >> >> >> On Mon, Mar 5, 2012 at 10:34 PM, envelopes envelopes < sunwood360@REDACTED> wrote: >>> >>> weird. >>> >>> Why "erl -pa ebin -s mysample_app start x y" doesn't work, but "erl -pa ebin -s application start mysample" works, >>> >>> is it a bug in erlang? >>> >>> >>> On Mon, Mar 5, 2012 at 10:21 PM, Ryan Huffman wrote: >>>> >>>> It looks like you are trying to call your state function with 0 arguments, but the start function has an arity of 2 (it expects 2 arguments). You can add 2 dummy arguments or use `-s application start mysample` instead. >>>> >>>> On Monday, March 5, 2012 at 10:07 PM, envelopes envelopes wrote: >>>>> >>>>> >>>>> I have a sample program generated from rebar >>>>> >>>>> 1. mysample_app.erl >>>>> >>>>> -module(mysample_app). >>>>> >>>>> -behaviour(application). >>>>> >>>>> %% Application callbacks >>>>> -export([start/2, stop/1, main/1]). >>>>> >>>>> %% =================================================================== >>>>> %% Application callbacks >>>>> %% =================================================================== >>>>> >>>>> start(_StartType, _StartArgs) -> >>>>> mysample_sup:start_link(). >>>>> >>>>> stop(_State) -> >>>>> ok. >>>>> >>>>> >>>>> main([A]) -> >>>>> I = list_to_integer(atom_to_list(A)), >>>>> F = fac(I), >>>>> io:format("factorial ~w = ~w~n" ,[I, F]), >>>>> init:stop(). >>>>> fac(0) -> 1; >>>>> fac(N) -> N*fac(N-1). >>>>> >>>>> >>>>> 2. mysample_sup.erl >>>>> >>>>> >>>>> -module(mysample_sup). >>>>> >>>>> -behaviour(supervisor). >>>>> >>>>> %% API >>>>> -export([start_link/0]). >>>>> >>>>> %% Supervisor callbacks >>>>> -export([init/1]). >>>>> >>>>> %% Helper macro for declaring children of supervisor >>>>> -define(CHILD(I, Type), {I, {I, start_link, []}, permanent, 5000, Type, [I]}). >>>>> >>>>> %% =================================================================== >>>>> %% API functions >>>>> %% =================================================================== >>>>> >>>>> start_link() -> >>>>> supervisor:start_link({local, ?MODULE}, ?MODULE, []). >>>>> >>>>> %% =================================================================== >>>>> %% Supervisor callbacks >>>>> %% =================================================================== >>>>> >>>>> init([]) -> >>>>> {ok, { {one_for_one, 5, 10}, []} }. >>>>> >>>>> >>>>> 3) the problem. >>>>> erl -pa ebin -s mysample_app main 23 ===> worked fine >>>>> erl -pa ebin -s mysample_app start ===> crashed on boot. >>>>> >>>>> {"init terminating in do_boot",{undef,[{mysample_app,start,[],[]},{init,start_it >>>>> ,1,[{file,"init.erl"},{line,1042}]},{init,start_em,1,[{file,"init.erl"},{line,10 >>>>> 22}]}]}} >>>>> >>>>> Crash dump was written to: erl_crash.dump >>>>> init terminating in do_boot () >>>>> >>>>> what is the cause ? my erlang is the latest R15. >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shahrdad1@REDACTED Tue Mar 6 20:18:26 2012 From: shahrdad1@REDACTED (Shahrdad Shadab) Date: Tue, 6 Mar 2012 14:18:26 -0500 Subject: [erlang-questions] exit (kill) vs. exit(PID,kill) Message-ID: I am amazed by difference between the effects of exit (kill) and exit (PID,kill). Assume PA and PB are two processes and PA has its trap_exit set to true. When PB evaluates exit(kill), the process terminates but PA doesn't die and actually gets message {EXIT,PB,kill}. However When PB evaluates exit(PA,kill), PA dies right a way ! Will someone please explain the intention behind this behavior? (or I am missing something! Thanks a lot -- Software Architect & Computer Scientist -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Tue Mar 6 21:30:49 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 6 Mar 2012 15:30:49 -0500 Subject: [erlang-questions] exit (kill) vs. exit(PID,kill) In-Reply-To: References: Message-ID: exit/1 and exit/2 are wildly different things. exit/1 is a local thing, much like erlang:error/1 and throw/1. They interact with the current stack, and you catch them with either 'catch' or 'try ... catch'. exit/2 on the other hand, sends 'exit signls' to other processes. They can't be caught, unless you use trap_exit. In these cases, trap_exit can make a process basically unkillable, so a special one ('kill') exists that bypasses the issue. When your process dies of reason 'kill' (either because I called exit(kill) or exit(self(), kill)), the reason is changed to 'killed' to avoid cascading failures of linked processes that trap exits, and the signal received is in fact 'killed', similar to 'exit(Pid, killed)' in some ways, although they operate differently. Hopefully this makes it a bit clearer. I've got more complete explanations at http://learnyousomeerlang.com/errors-and-processes, but it's a lengthy read for the question. On Tue, Mar 6, 2012 at 2:18 PM, Shahrdad Shadab wrote: > I am amazed by difference between the effects of exit (kill) and exit > (PID,kill). > > Assume PA and PB are two processes and PA has its trap_exit set to true. > When PB evaluates exit(kill), the process terminates but PA doesn't die > and actually gets message {EXIT,PB,kill}. > However When PB evaluates exit(PA,kill), PA dies right a way ! > > Will someone please explain the intention behind this behavior? (or I am > missing something! > > Thanks a lot > > -- > Software Architect & Computer Scientist > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Tue Mar 6 21:53:23 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 6 Mar 2012 20:53:23 +0000 Subject: [erlang-questions] dumb question from rookie In-Reply-To: References: <451378EF51F74C12B22B409C5884057D@gmail.com> <57E2C777-88A1-4226-9667-45766D2B06F0@gmail.com> Message-ID: On 6 Mar 2012, at 16:39, envelopes envelopes wrote: > On Mar 6, 2012 2:11 AM, "Tim Watson" wrote: > > > > Hi there, > > > > I'm not sure about this second issue (of the _app module not being loaded in the shell) but I can explain the former. When you run the 'erl' program with -s you are telling the built in 'init' module to run the following module, function with optional arguments. When you pass '-s mysample_app start' you are telling the init module to call that function, but 'init' is a special module, part of the runtime system. Your module isn't special, so the code server hasn't loaded it yet and 'init' can't find it so it is 'undefined' hence the error message. > > how come ' erl -pa ebin -s mysample_app main 23' works? If the code server has not loaded it yet. The behavior is not consistent. > > > My mistake I withdraw my original explanation as it was completely wrong. Ryan was correct it is due to the arity of your function. In the 'erl' documentation: "... If no arguments are provided, the function is assumed to be of arity 0. Otherwise it is assumed to be of arity 1, taking the list[Arg1,Arg2,...] as argument. " If you want to pass more than one argument on startup, you'll need to use -eval instead: 'erl pa ebin -eval 'prototype_app:start(25, 30)' My misdirection notwithstanding, the idiomatic way to start an application is to either put it in a release or to pass '-s application start ' rather than call the module's start/2 function explicitly. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Wed Mar 7 00:28:17 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 6 Mar 2012 15:28:17 -0800 (PST) Subject: [erlang-questions] support for .ez archives In-Reply-To: References: Message-ID: <13615880.4065.1331076497231.JavaMail.geo-discussion-forums@ynlt17> Some .hrl files count as interfaces - a particular case being "public" record definitions made in header files. Usually, if a .hrl contains "application private" definitions, I'm inclined to leave it in src. If it includes "public" definitions then I'd drop it in include. Probably the greatest case for having .ez files is libraries -- where the public api is the most important. So I'd support +1 the idea that .hrl files should be accessible from within a zipped ez - even if constrained (or maybe should be so?) to just an include directory... My 2c /s On Tuesday, March 6, 2012 10:29:57 AM UTC-6, Tim Watson wrote: > > In general, I think .ez archives are an ideal unit of packaging and > distribution (as dependencies) and therefore making them fully self > contained is very helpful. If you specify 'lager >= 1.0.2' as a dependency > and all I have to do is find and fetch the archive (and check the MD5 and > whatnot) then stick it somewhere useful on the machine, the subsequent > build (handling of dependent packages), release assembly and other things > just work really nicely. Having to 'manually' pull the includes out before > being able to utilise the archive as a compile time dependency is just a > bit annoying for tools authors that's all. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sunwood360@REDACTED Wed Mar 7 03:49:42 2012 From: sunwood360@REDACTED (envelopes envelopes) Date: Tue, 6 Mar 2012 18:49:42 -0800 Subject: [erlang-questions] dumb question from rookie In-Reply-To: References: <451378EF51F74C12B22B409C5884057D@gmail.com> <57E2C777-88A1-4226-9667-45766D2B06F0@gmail.com> Message-ID: cool, ==> erl -pa apps/dummy/ebin -eval 'dummy_app:start(25,30)' works. Another workaround is adding one extra start/0 function, like : start() -> dummy_sup:start_link(). then both ==> erl -pa apps/dummy/ebin -s dummy_app and ==> erl -pa apps/dummy/ebin -s dummy_app start worked. On Tue, Mar 6, 2012 at 12:53 PM, Tim Watson wrote: > On 6 Mar 2012, at 16:39, envelopes envelopes wrote: > > On Mar 6, 2012 2:11 AM, "Tim Watson" wrote: > > > > Hi there, > > > > I'm not sure about this second issue (of the _app module not being > loaded in the shell) but I can explain the former. When you run the 'erl' > program with -s you are telling the built in 'init' module to run the > following module, function with optional arguments. When you pass '-s > mysample_app start' you are telling the init module to call that function, > but 'init' is a special module, part of the runtime system. Your module > isn't special, so the code server hasn't loaded it yet and 'init' can't > find it so it is 'undefined' hence the error message. > > how come ' erl -pa ebin -s mysample_app main 23' works? If the code server > has not loaded it yet. The behavior is not consistent. > > > My mistake I withdraw my original explanation as it was completely wrong. > Ryan was correct it is due to the arity of your function. In the 'erl' > documentation: > > "... If no arguments are provided, the function is assumed to be of arity > 0. Otherwise it is assumed to be of arity 1, taking the list > [Arg1,Arg2,...] as argument. " > > If you want to pass more than one argument on startup, you'll need to use > -eval instead: 'erl pa ebin -eval 'prototype_app:start(25, 30)' > > My misdirection notwithstanding, the idiomatic way to start an application > is to either put it in a release or to pass '-s application start ' > rather than call the module's start/2 function explicitly. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Wed Mar 7 08:38:12 2012 From: per@REDACTED (Per Hedeland) Date: Wed, 7 Mar 2012 08:38:12 +0100 (CET) Subject: [erlang-questions] New ssl vs Google Chrome Message-ID: <201203070738.q277cCSN032320@pluto.hedeland.org> Hello, Is anyone else seeing (or even better, fixing:-) performance problems when Chrome is used towards a HTTPS server with "new ssl"? With the server running on a "pretty slow" PPC system, I'm seeing a simple test (connect and log in to webui) where beam + ssl_esock used a total of ~ 1 second CPU with "old ssl" (in R14B01) use upwards of 12 seconds CPU with "new" (still R14B01) - i.e. essentially unusable. Running ssldump reveals that Chrome's SSL behavior is pretty braindead (a technical term) - it opens loads of SSL connections, most of them without session reuse, and if it's unhappy with the server cert it will also close most of them without even completing the client side of the handshake, let alone doing any application data transfer. The test used a cert with a 4096 bit RSA key - dropping it to 1024 makes the problem basically go away (~ 0.4 second CPU with "new"). This would seem to confirm that the bad performance is due to the many full handshakes generated by Chrome - but it's obviously not a fix, nor does it explain the huge difference between old and new (the Chrome behavior is basically the same with both). I tried doing just the RSA signing with a 4096 bit key on the "pretty slow" system - via public_key it uses ~ 1.4 seconds CPU, "directly" with libcrypto ~ 0.5 seconds. This seems rather strange, since as far as I can see there isn't much code executed besides the libcrypto call when public_key is used. And there still remains an unexplained factor of 4 or more in lost performance. The performance seems to have improved a bit in R14B04, with 6 seconds CPU for the above test - but it's still not really usable, and I don't see anything relevant in the READMEs. I should add that the comparison above also used different versions of yaws, 1.62 vs 1.91 - but it's hard to see how this could have a significant impact on the SSL handshake. Any info appreciated... Thanks --Per Hedeland From rexxe98@REDACTED Wed Mar 7 08:43:02 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Tue, 6 Mar 2012 23:43:02 -0800 Subject: [erlang-questions] Help with Sublime Text 2 Code Completion Message-ID: All, I'm trying to build a plug-in for Sublime Text 2 to do proper Erlang code completion. My thought is to have a server or node running and then talk to that node using erl_call. I have it working with the stdlib, but I want the code completion to be dynamic such that the plug-in tells the remote node to add code paths. I have that working too, but the issue is loading the actual modules in the new code path. Since the erlang vm loads modules only upon access, when I call erlang:loaded() or code:all_loaded(), the modules added with the new code path do not show. Do I have to loop through the BEAMs in the new code path and manually load them or is there a better way to do it? I'm trying to make the completion use a fuzzy search such that something like "li" brings up all modules with "li" in it which is why I need to be able to have the entire list of modules loaded already. Thanks, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Wed Mar 7 09:22:21 2012 From: per@REDACTED (Per Hedeland) Date: Wed, 7 Mar 2012 09:22:21 +0100 (CET) Subject: [erlang-questions] New ssl vs Google Chrome In-Reply-To: <201203070738.q277cCSN032320@pluto.hedeland.org> Message-ID: <201203070822.q278MLdQ033146@pluto.hedeland.org> Sorry, forgot to mention that the DH cipher suites were not enabled for "new", since they're pretty CPU-intensive and weren't supported by "old" anyway. I.e. all tests used AES256-SHA a.k.a. TLS_RSA_WITH_AES_256_CBC_SHA. --Per Per Hedeland wrote: > >Hello, > >Is anyone else seeing (or even better, fixing:-) performance problems >when Chrome is used towards a HTTPS server with "new ssl"? With the >server running on a "pretty slow" PPC system, I'm seeing a simple test >(connect and log in to webui) where beam + ssl_esock used a total of ~ 1 >second CPU with "old ssl" (in R14B01) use upwards of 12 seconds CPU with >"new" (still R14B01) - i.e. essentially unusable. > >Running ssldump reveals that Chrome's SSL behavior is pretty braindead >(a technical term) - it opens loads of SSL connections, most of them >without session reuse, and if it's unhappy with the server cert it will >also close most of them without even completing the client side of the >handshake, let alone doing any application data transfer. > >The test used a cert with a 4096 bit RSA key - dropping it to 1024 makes >the problem basically go away (~ 0.4 second CPU with "new"). This would >seem to confirm that the bad performance is due to the many full >handshakes generated by Chrome - but it's obviously not a fix, nor does >it explain the huge difference between old and new (the Chrome behavior >is basically the same with both). > >I tried doing just the RSA signing with a 4096 bit key on the "pretty >slow" system - via public_key it uses ~ 1.4 seconds CPU, "directly" with >libcrypto ~ 0.5 seconds. This seems rather strange, since as far as I >can see there isn't much code executed besides the libcrypto call when >public_key is used. And there still remains an unexplained factor of 4 >or more in lost performance. > >The performance seems to have improved a bit in R14B04, with 6 seconds >CPU for the above test - but it's still not really usable, and I don't >see anything relevant in the READMEs. I should add that the comparison >above also used different versions of yaws, 1.62 vs 1.91 - but it's hard >to see how this could have a significant impact on the SSL handshake. > >Any info appreciated... > >Thanks > >--Per Hedeland >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions > From pekadan@REDACTED Wed Mar 7 11:55:59 2012 From: pekadan@REDACTED (Peter Andersson) Date: Wed, 7 Mar 2012 11:55:59 +0100 Subject: [erlang-questions] Running distributed tests with common test In-Reply-To: References: Message-ID: [Answered this yesterday, but appearently the mail didn't get through, so I try again...] Hi Fredrik, Unfortunately, the current ct_master implementation requires that all test host nodes have a shared file system (for the links from the master_runs page to be valid anyway). A custom event handler won't help much since it's mainly statistics and error messages that the event handler can receive and report anyway. Common Test doesn't send general io such as printouts from the SUT and the test cases, the CT framework log, and other debug info to the event manager, and I assume you want to be able to read all that info on the master node, right? What you could do is maybe copy the log files from the test host nodes to the master node during the test run, by means of a CT hook. The links in master_runs.html will then still be invalid of course, but this we could implement a fix for - like introduce a new/modified test specification term that tells ct_master which path to use for the links in the master log file. The links between files inside the log file structure on each test host are relative, so that shouldn't cause any problems. I'm sure a solution like this is not as simple as it may sound, but I don't mind playing around a bit with this myself to see what works and what doesn't and how we can improve the current ct_master implementation in this regard. In the future, we'd like to make the io handling part of Common Test open for other formats than html (by means of plugins) and that should also make it much easier to implement a distributed test mode where io can be passed from host nodes to master node. That's a big job though, and not one we've had time to start yet. Best regards, Peter Ericsson AB, Erlang/OTP 2012/3/6, Fredrik Svahn : > Hi, > > I have a setup where I am using ct_master:run_test/2 to run tests on target > nodes which each have a local file system. > > On the server I get the following html result pages: > master_runs.html > ct_master_run.2012-03-06_14.17.16/index.html > > The links on the latter page are however broken, since they refer to files > on each target's local file system. In the Users Guide it is stated: "The > CT Master server writes progress information to HTML log files similarly to > the regular CT server. The logs contain test statistics and links to the > log files written by each independent CT server" which seems to support > that what I am seeing is indeed the intended behaviour. > > Nevertheless, since I know CT is used for a lot of embedded testing I am > wondering if there is a way of having the test targets report back *all* > results and logs to the master server (as opposed to writing the results > locally)? Would writing a custom event handler help? > > BR /Fredrik > > PS. this is the full command I use to start the tests, just in case I have > missed some simple config option: > ct_master:run_test(target@REDACTED, [{suite,[array_SUITE]},{auto_compile, > false}, {basic_html, true}, batch]). > From cbenac@REDACTED Wed Mar 7 14:51:30 2012 From: cbenac@REDACTED (Clara Benac Earle) Date: Wed, 07 Mar 2012 14:51:30 +0100 Subject: [erlang-questions] Reminder: Madrid Erlounge 8th of March Message-ID: <4F5767E2.2030109@fi.upm.es> Dear all, We will meet up for the Fourth Madrid Erlounge the 8th of March. In principle the talk will be in Spanish but please let us know if anybody planning to attend prefers English and then we will switch to Spanglish :-) Beer and tapas afterwards. Everybody welcome! Clara ------------------------------------------------------------------------------------------- Hola a todos, Anunciamos la pr?xima Madrid Erlounge. *Cu?ndo:* 8 de Marzo 2012, charla a las 19:30 *D?nde:* Sala de Grados (1? planta Facultad de Inform?tica, Universidad Complutense de Madrid) (http://gpd.sip.ucm.es/fraguas/wflp06/campus_ucm.jpg) *Qui?n:* Manuel ?ngel Rubio (alias Bombadil) Manuel ?ngel trabaja en el departamento de I+D de Jet Multimedia Espa?a (http://www.jetmultimedia.es), como gerente de desarrollos de I+D ?rea de voz. Desde 2008 usan Erlang para el desarrollo de soluciones escalables y para telefon?a con ?xito, consiguiendo desarrollos como un ACD con m?s de 200 agentes (algo totalmente imposible con otros entornos) y sistemas de gesti?n de llamadas salientes de hasta 20 mil llamadas concurrentes. *Qu?:* Gesti?n de Llamadas con Erlang/OTP La charla va encauzada a mostrar un ejemplo, en propia piel, del uso de Erlang/OTP en el entorno de la prestaci?n de servicios por Internet. En este caso, de telefon?a, donde se ver? c?mo se di? soluci?n a la gesti?n de llamadas miles de llamadas concurrentes con sistemas que en principio solo soportaban decenas. ?Hasta ma?ana! Clara -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Wed Mar 7 15:04:39 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 7 Mar 2012 15:04:39 +0100 Subject: [erlang-questions] New ssl vs Google Chrome In-Reply-To: <201203070738.q277cCSN032320@pluto.hedeland.org> References: <201203070738.q277cCSN032320@pluto.hedeland.org> Message-ID: Hi! We will have to look into this more, as a start see comments below: 2012/3/7, Per Hedeland : > Hello, > > Is anyone else seeing (or even better, fixing:-) performance problems > when Chrome is used towards a HTTPS server with "new ssl"? With the > server running on a "pretty slow" PPC system, I'm seeing a simple test > (connect and log in to webui) where beam + ssl_esock used a total of ~ 1 > second CPU with "old ssl" (in R14B01) use upwards of 12 seconds CPU with > "new" (still R14B01) - i.e. essentially unusable. > > Running ssldump reveals that Chrome's SSL behavior is pretty braindead > (a technical term) - it opens loads of SSL connections, most of them > without session reuse, and if it's unhappy with the server cert it will > also close most of them without even completing the client side of the > handshake, let alone doing any application data transfer. > > The test used a cert with a 4096 bit RSA key - dropping it to 1024 makes > the problem basically go away (~ 0.4 second CPU with "new"). This would > seem to confirm that the bad performance is due to the many full > handshakes generated by Chrome - but it's obviously not a fix, nor does > it explain the huge difference between old and new (the Chrome behavior > is basically the same with both). > > I tried doing just the RSA signing with a 4096 bit key on the "pretty > slow" system - via public_key it uses ~ 1.4 seconds CPU, "directly" with > libcrypto ~ 0.5 seconds. This seems rather strange, since as far as I > can see there isn't much code executed besides the libcrypto call when > public_key is used. crypto:mpint/1 ofcourse is an overhad that could somehow be worked out of the equation. I tried it with a big key on my machine but does not appear to make such a big difference that it does for you. > And there still remains an unexplained factor of 4 > or more in lost performance. Does your server do client verification? If crypto:mpint/1 is an issue you would get a few more of those in that case. > The performance seems to have improved a bit in R14B04, with 6 seconds > CPU for the above test - but it's still not really usable, and I don't > see anything relevant in the READMEs. I should add that the comparison > above also used different versions of yaws, 1.62 vs 1.91 - but it's hard > to see how this could have a significant impact on the SSL handshake. > > Any info appreciated... > > Thanks > > --Per Hedeland > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Regards Ingela Erlang/OTP team - Ericsson AB From jack@REDACTED Wed Mar 7 16:26:05 2012 From: jack@REDACTED (Jack Moffitt) Date: Wed, 7 Mar 2012 08:26:05 -0700 Subject: [erlang-questions] support for .ez archives In-Reply-To: References: Message-ID: Whoops, accidentally sent this only to Tim. Resending to the list: > In general, I think .ez archives are an ideal unit of packaging and > distribution (as dependencies) and therefore making them fully self > contained is very helpful. If you specify 'lager >= 1.0.2' as a dependency > and all I have to do is find and fetch the archive (and check the MD5 and > whatnot) then stick it somewhere useful on the machine, the subsequent build > (handling of dependent packages), release assembly and other things just > work really nicely. Having to 'manually' pull the includes out before being > able to utilise the archive as a compile time dependency is just a bit > annoying for tools authors that's all. +1 Clojure gets something like this for free since it uses Java's jar system. Jars can contain configuration, source, images, or anything else. Not only do all the distribution and build tools make use of this, but the libraries contain things to make reading files from inside the jar really trivial. I would love to see something similar in Erlang. jack. From emeka_1978@REDACTED Wed Mar 7 17:03:01 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Wed, 7 Mar 2012 08:03:01 -0800 (PST) Subject: [erlang-questions] wxStaticText resizing Message-ID: <44b0c067-39b0-441f-9461-bffe59f7954b@o16g2000yqg.googlegroups.com> I have a static text centered in a panel. When i resize the panel, i would like the text to resize as well. This is my code: wxFrame:connect(Parent, close_window), Panel = wxPanel:new(Parent, []), %% Setup sizers TextSizer = wxBoxSizer:new(?wxVERTICAL), ClockText = wxStaticText:new(Panel,1,"00 : 00 : 00",[{style,? wxALIGN_CENTER}]), %% Add to sizers wxSizer:add(TextSizer,ClockText,[{flag,?wxEXPAND},{proportion, 1}]), wxPanel:setSizer(Panel,TextSizer), wxFrame:show(Parent) From john.hughes@REDACTED Wed Mar 7 17:35:59 2012 From: john.hughes@REDACTED (John Hughes) Date: Wed, 7 Mar 2012 17:35:59 +0100 Subject: [erlang-questions] Serving PDF using Nitrogen/Inets Message-ID: <37BFB75479424820BA16E6B9EA531748@JohnsTablet2012> I?m having trouble serving static pdf files using nitrogen (running on inets for the time being). I think they?re being served with the wrong content type: my browser treats them as text and displays the PDF source; moreover, if I try to ?save the page? then a .txt is appended to the file name (I?m running on Windows 7). This happens if I visit a URL such as http://localhost:8000/file.pdf in my browser (with nitrogen running on the same machine). Any idea what?s going on, or how I can fix it? John -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmercer@REDACTED Wed Mar 7 18:25:33 2012 From: dmercer@REDACTED (David Mercer) Date: Wed, 7 Mar 2012 11:25:33 -0600 Subject: [erlang-questions] Reading a file before it has been completely written Message-ID: <000001ccfc87$4e9d2dd0$ebd78970$@com> While this isn't an Erlang-specific question, the problem arises from my using Richard Carlsson's file_monitor (https://github.com/richcarl/eunit/blob/master/src/file_monitor.erl), which sends messages when a file or directory is changed. I have found that it is not unusual to get a message about a new file before the file has been completely written. I had thought that by doing a file:open(Filepath, [read]) and making sure I got back {ok, _} rather than {error, eacces} I could avoid those cases, but that approach has failed for me: this morning, I got back {ok, _}, but the file was not completely written yet. Another approach I tried was to attempt to obtain an exclusive lock (I think it was file:open(Filepath, [read, exclusive])), but in my testing I came across the bizarre scenario where I would copy a file into the monitored directory, the file_monitor would send the message, but the Erlang process that does the file-open didn't see it, so created the file (the documentation says it creates the file if it does not exist), and then I got a message in my window where I was copying that the file already exists, do I want to overwrite it. Another approach I tried was renaming the file to itself. All my tests indicated that that approach would work, but all my tests also indicated that just doing the file:open(Filepath, [read]) would work, too, so I chose it, as it seemed cleaner. I could revert to the rename approach, but I'm not even sure now that that will work. I imagine others among us have encountered this issue, and rather than reinvent the wheel, what is the favored approach to handling this issue? Cheers, David Mercer -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Wed Mar 7 18:39:41 2012 From: tony@REDACTED (Tony Rogvall) Date: Wed, 7 Mar 2012 18:39:41 +0100 Subject: [erlang-questions] Reading a file before it has been completely written In-Reply-To: <000001ccfc87$4e9d2dd0$ebd78970$@com> References: <000001ccfc87$4e9d2dd0$ebd78970$@com> Message-ID: <1836F72C-3CD3-4D88-BF88-F90176578271@rogvall.se> - Create and open a file with a temporary name. - Write the file content. - Close the file. - Rename the file to the name/place you want. works ? /Tony On 7 mar 2012, at 18:25, David Mercer wrote: > While this isn?t an Erlang-specific question, the problem arises from my using Richard Carlsson?s file_monitor(https://github.com/richcarl/eunit/blob/master/src/file_monitor.erl), which sends messages when a file or directory is changed. I have found that it is not unusual to get a message about a new file before the file has been completely written. > > I had thought that by doing a file:open(Filepath, [read]) and making sure I got back {ok, _} rather than{error, eacces} I could avoid those cases, but that approach has failed for me: this morning, I got back {ok, _}, but the file was not completely written yet. > > Another approach I tried was to attempt to obtain an exclusive lock (I think it was file:open(Filepath, [read, exclusive])), but in my testing I came across the bizarre scenario where I would copy a file into the monitored directory, the file_monitor would send the message, but the Erlang process that does the file-open didn?t see it, so created the file (the documentation says it creates the file if it does not exist), and then I got a message in my window where I was copying that the file already exists, do I want to overwrite it. > > Another approach I tried was renaming the file to itself. All my tests indicated that that approach would work, but all my tests also indicated that just doing the file:open(Filepath, [read]) would work, too, so I chose it, as it seemed cleaner. I could revert to the rename approach, but I?m not even sure now that that will work. > > I imagine others among us have encountered this issue, and rather than reinvent the wheel, what is the favored approach to handling this issue? > > Cheers, > > David Mercer > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Wed Mar 7 18:54:08 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 07 Mar 2012 18:54:08 +0100 Subject: [erlang-questions] Reading a file before it has been completely written In-Reply-To: <000001ccfc87$4e9d2dd0$ebd78970$@com> References: <000001ccfc87$4e9d2dd0$ebd78970$@com> Message-ID: <4F57A0C0.9030009@gmail.com> On 03/07/2012 06:25 PM, David Mercer wrote: > While this isn?t an Erlang-specific question, the problem arises from my > using Richard Carlsson?s /file_monitor/ > (https://github.com/richcarl/eunit/blob/master/src/file_monitor.erl), > which sends messages when a file or directory is changed. I have found > that it is not unusual to get a message about a new file before the file > has been completely written. > > I had thought that by doing a file:open(Filepath, [read]) and making > sure I got back {ok, _} rather than {error, eacces} I could avoid those > cases, but that approach has failed for me: this morning, I got back > {ok, _}, but the file was not completely written yet. > > Another approach I tried was to attempt to obtain an exclusive lock (I > think it was file:open(Filepath, [read, exclusive])), but in my testing > I came across the bizarre scenario where I would copy a file into the > monitored directory, the file_monitor would send the message, but the > Erlang process that does the file-open didn?t see it, so created the > file (the documentation says it creates the file if it does not exist), > and then I got a message in my window where I was copying that the file > already exists, do I want to overwrite it. > > Another approach I tried was renaming the file to itself. All my tests > indicated that that approach would work, but all my tests also indicated > that just doing the file:open(Filepath, [read]) would work, too, so I > chose it, as it seemed cleaner. I could revert to the rename approach, > but I?m not even sure now that that will work. > > I imagine others among us have encountered this issue, and rather than > reinvent the wheel, what is the favored approach to handling this issue? > > Cheers, > > David Mercer Hey, a user! I haven't had any reports about this module before (and the fact that it's still in my development branch of eunit is more of a historical accident; it's not shipped with OTP). I don't know of any real issues with it though. In this case, I think the problem is just the underlying file system semantics. I presume it's Linux, and in Unix:y file systems a file can be seen to exist and can be opened for reading as soon as it has been created. Trying to fiddle with exclusive locks is probably always going to have corner cases. The only techniques you can trust to practically always work and be portable across file systems are directory creation and file renaming. So what Tony suggested is likely to be the best solution: create the file under another name or in a separate directory, and when it's completely written, rename it. /Richard From dmercer@REDACTED Wed Mar 7 19:06:05 2012 From: dmercer@REDACTED (David Mercer) Date: Wed, 7 Mar 2012 12:06:05 -0600 Subject: [erlang-questions] Reading a file before it has been completely written In-Reply-To: <1836F72C-3CD3-4D88-BF88-F90176578271@rogvall.se> References: <000001ccfc87$4e9d2dd0$ebd78970$@com> <1836F72C-3CD3-4D88-BF88-F90176578271@rogvall.se> Message-ID: <002201ccfc8c$f7553800$e5ffa800$@com> I'm not the one writing the file. I'm the one reading it. I have no control over the writing. Thanks for the thoughts, though. DBM From: Tony Rogvall [mailto:tony@REDACTED] Sent: Wednesday, March 07, 2012 11:40 AM To: David Mercer Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Reading a file before it has been completely written - Create and open a file with a temporary name. - Write the file content. - Close the file. - Rename the file to the name/place you want. works ? /Tony On 7 mar 2012, at 18:25, David Mercer wrote: While this isn't an Erlang-specific question, the problem arises from my using Richard Carlsson's file_monitor(https://github.com/richcarl/eunit/blob/master/src/file_monitor. erl), which sends messages when a file or directory is changed. I have found that it is not unusual to get a message about a new file before the file has been completely written. I had thought that by doing a file:open(Filepath, [read]) and making sure I got back {ok, _} rather than{error, eacces} I could avoid those cases, but that approach has failed for me: this morning, I got back {ok, _}, but the file was not completely written yet. Another approach I tried was to attempt to obtain an exclusive lock (I think it was file:open(Filepath, [read, exclusive])), but in my testing I came across the bizarre scenario where I would copy a file into the monitored directory, the file_monitor would send the message, but the Erlang process that does the file-open didn't see it, so created the file (the documentation says it creates the file if it does not exist), and then I got a message in my window where I was copying that the file already exists, do I want to overwrite it. Another approach I tried was renaming the file to itself. All my tests indicated that that approach would work, but all my tests also indicated that just doing the file:open(Filepath, [read]) would work, too, so I chose it, as it seemed cleaner. I could revert to the rename approach, but I'm not even sure now that that will work. I imagine others among us have encountered this issue, and rather than reinvent the wheel, what is the favored approach to handling this issue? Cheers, David Mercer _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmercer@REDACTED Wed Mar 7 19:08:44 2012 From: dmercer@REDACTED (David Mercer) Date: Wed, 7 Mar 2012 12:08:44 -0600 Subject: [erlang-questions] Reading a file before it has been completely written In-Reply-To: <4F57A0C0.9030009@gmail.com> References: <000001ccfc87$4e9d2dd0$ebd78970$@com> <4F57A0C0.9030009@gmail.com> Message-ID: <002c01ccfc8d$56adc9c0$04095d40$@com> On Wednesday, March 07, 2012, Richard Carlsson wrote: > Hey, a user! I haven't had any reports about this module before (and > the > fact that it's still in my development branch of eunit is more of a > historical accident; it's not shipped with OTP). I don't know of any > real issues with it though. It works fine. If you know of a better one, I'd be OK switching. This was just the one that came up when I Googled. > In this case, I think the problem is just the underlying file system > semantics. I presume it's Linux, and in Unix:y file systems a file can > be seen to exist and can be opened for reading as soon as it has been > created. Trying to fiddle with exclusive locks is probably always going > to have corner cases. The only techniques you can trust to practically > always work and be portable across file systems are directory creation > and file renaming. So what Tony suggested is likely to be the best > solution: create the file under another name or in a separate > directory, > and when it's completely written, rename it. I might go back to my renaming approach, which also had no failures during testing. I just attempt to rename the file to itself. If it fails, I try again 5 seconds later. Thanks. Cheers, DBM From daniel@REDACTED Wed Mar 7 19:09:32 2012 From: daniel@REDACTED (Daniel Luna) Date: Wed, 7 Mar 2012 13:09:32 -0500 Subject: [erlang-questions] Reading a file before it has been completely written In-Reply-To: References: <000001ccfc87$4e9d2dd0$ebd78970$@com> <4F57A0C0.9030009@gmail.com> Message-ID: (Sorry, Richard for getting this multiple times, forgot to include the list) Just as an added caveat to what Richard mentions: you probably want to make sure that the temp file is created on the same file system as your test directory. ?A rename between file systems is really just a copy and delete, with the exact same problems you had previously (depending on OS I guess), while a rename within a file system is a link/unlink with the contents being untouched. /Daniel On 7 March 2012 12:54, Richard Carlsson wrote: > On 03/07/2012 06:25 PM, David Mercer wrote: >> >> While this isn?t an Erlang-specific question, the problem arises from my >> using Richard Carlsson?s /file_monitor/ >> >> (https://github.com/richcarl/eunit/blob/master/src/file_monitor.erl), >> which sends messages when a file or directory is changed. I have found >> that it is not unusual to get a message about a new file before the file >> has been completely written. >> >> I had thought that by doing a file:open(Filepath, [read]) and making >> sure I got back {ok, _} rather than {error, eacces} I could avoid those >> cases, but that approach has failed for me: this morning, I got back >> {ok, _}, but the file was not completely written yet. >> >> Another approach I tried was to attempt to obtain an exclusive lock (I >> think it was file:open(Filepath, [read, exclusive])), but in my testing >> I came across the bizarre scenario where I would copy a file into the >> monitored directory, the file_monitor would send the message, but the >> Erlang process that does the file-open didn?t see it, so created the >> file (the documentation says it creates the file if it does not exist), >> and then I got a message in my window where I was copying that the file >> already exists, do I want to overwrite it. >> >> Another approach I tried was renaming the file to itself. All my tests >> indicated that that approach would work, but all my tests also indicated >> that just doing the file:open(Filepath, [read]) would work, too, so I >> chose it, as it seemed cleaner. I could revert to the rename approach, >> but I?m not even sure now that that will work. >> >> I imagine others among us have encountered this issue, and rather than >> reinvent the wheel, what is the favored approach to handling this issue? >> >> Cheers, >> >> David Mercer > > > Hey, a user! I haven't had any reports about this module before (and the > fact that it's still in my development branch of eunit is more of a > historical accident; it's not shipped with OTP). I don't know of any real > issues with it though. > > In this case, I think the problem is just the underlying file system > semantics. I presume it's Linux, and in Unix:y file systems a file can be > seen to exist and can be opened for reading as soon as it has been created. > Trying to fiddle with exclusive locks is probably always going to have > corner cases. The only techniques you can trust to practically always work > and be portable across file systems are directory creation and file > renaming. So what Tony suggested is likely to be the best solution: create > the file under another name or in a separate directory, and when it's > completely written, rename it. > > ? /Richard > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From carlsson.richard@REDACTED Wed Mar 7 19:30:06 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 07 Mar 2012 19:30:06 +0100 Subject: [erlang-questions] Reading a file before it has been completely written In-Reply-To: <002c01ccfc8d$56adc9c0$04095d40$@com> References: <000001ccfc87$4e9d2dd0$ebd78970$@com> <4F57A0C0.9030009@gmail.com> <002c01ccfc8d$56adc9c0$04095d40$@com> Message-ID: <4F57A92E.2090402@gmail.com> On 03/07/2012 07:08 PM, David Mercer wrote: > On Wednesday, March 07, 2012, Richard Carlsson wrote: > >> Hey, a user! I haven't had any reports about this module before (and >> the >> fact that it's still in my development branch of eunit is more of a >> historical accident; it's not shipped with OTP). I don't know of any >> real issues with it though. > > It works fine. If you know of a better one, I'd be OK switching. This was > just the one that came up when I Googled. No, I think it's pretty good and I don't know any other portable implementation. I'd just like to add optional inotify support (and whatever it's called on Windows) on supported platforms. Right now it only works by polling. Which is actually good enough in a lot of cases. /Richard From per@REDACTED Wed Mar 7 19:39:25 2012 From: per@REDACTED (Per Hedeland) Date: Wed, 7 Mar 2012 19:39:25 +0100 (CET) Subject: [erlang-questions] New ssl vs Google Chrome In-Reply-To: Message-ID: <201203071839.q27IdPmv044280@pluto.hedeland.org> Ingela Andin wrote: > >We will have to look into this more, as a start see comments below: Thanks! >crypto:mpint/1 ofcourse is an overhad that could somehow be worked out >of the equation. The three calls in public_key:sign/3 add upp to about a millisecond on my slow test system. >I tried it with a big key on my machine but does not appear to make such a big >difference that it does for you. Right, you need a slow CPU.:-) On my workstation PC the same test gives something like 0.2 seconds CPU for "old" and 0.6 seconds CPU for "new", which would go unnoticed unless there were huge numbers of SSL connections. Unfortunately our customers tend to use low-powered PowerPC and the like, and still expect to have a usable SSL-webui (at least one session:-). For a relevant number, 'openssl speed rsa4096' gives 2.0 sign/sec on the PPC where I'm doing the test (62.7 on my PC - i.e. 16 ms per sign - vs 54 ms for public_key:sign/3). But the CPU time needed to do a RSA sign is probably not the major problem - using Firefox with "new" on the slow system works perfectly fine, because Firefox knows how to do SSL in a server-friendly way - in my test it's a single full SSL handshake and 5 additional connections with session reuse, all 6 used to carry application data. Chrome opens 26 connnections, starting handshake on all of them, 7 handshakes are actually completed (5 full and 2 reuse), and only 4 of the connections do anything useful. But somehow "old" managed to handle this stupidity with less CPU impact - actually it ended up with "only" 19 connections, 11 completed handshakes (a single full and 10 reuse), and 5 of them doing something useful. If you want I can send you ssldump output off-list... >Does your server do client verification? No. Thanks again! --Per From g@REDACTED Wed Mar 7 22:09:38 2012 From: g@REDACTED (Garrett Smith) Date: Wed, 7 Mar 2012 15:09:38 -0600 Subject: [erlang-questions] Serving PDF using Nitrogen/Inets In-Reply-To: <37BFB75479424820BA16E6B9EA531748@JohnsTablet2012> References: <37BFB75479424820BA16E6B9EA531748@JohnsTablet2012> Message-ID: On Wed, Mar 7, 2012 at 10:35 AM, John Hughes wrote: > I?m having trouble serving static pdf files using nitrogen (running on inets > for the time being). I think they?re being served with the wrong content > type: my browser treats them as text and displays the PDF source; moreover, > if I try to ?save the page? then a .txt is appended to the file name (I?m > running on Windows 7). This happens if I visit a URL such as > http://localhost:8000/file.pdf in my browser (with nitrogen running on the > same machine). When I use httpd/inets with modlib, I need to explicitly configure MIME types, even for images, css, etc. I wonder if Nitrogen does the same thing but has omitted pdf. You're seeing other mime types come through? E.g. css -> text/css? Garrett From gumm@REDACTED Wed Mar 7 22:41:38 2012 From: gumm@REDACTED (Jesse Gumm) Date: Wed, 7 Mar 2012 15:41:38 -0600 Subject: [erlang-questions] Serving PDF using Nitrogen/Inets In-Reply-To: References: <37BFB75479424820BA16E6B9EA531748@JohnsTablet2012> Message-ID: Hi John, In your nitrogen installation, check out /etc/inets_httpd.env, which defines a few common mimetypes for inets. https://github.com/nitrogen/nitrogen/blob/master/rel/overlay/inets/etc/inets_httpd.erlenv -Jesse On Wed, Mar 7, 2012 at 3:09 PM, Garrett Smith wrote: > On Wed, Mar 7, 2012 at 10:35 AM, John Hughes wrote: >> I?m having trouble serving static pdf files using nitrogen (running on inets >> for the time being). I think they?re being served with the wrong content >> type: my browser treats them as text and displays the PDF source; moreover, >> if I try to ?save the page? then a .txt is appended to the file name (I?m >> running on Windows 7). This happens if I visit a URL such as >> http://localhost:8000/file.pdf in my browser (with nitrogen running on the >> same machine). > > When I use httpd/inets with modlib, I need to explicitly configure > MIME types, even for images, css, etc. I wonder if Nitrogen does the > same thing but has omitted pdf. > > You're seeing other mime types come through? E.g. css -> text/css? > > Garrett > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm From dave@REDACTED Thu Mar 8 01:28:46 2012 From: dave@REDACTED (David Goehrig) Date: Wed, 7 Mar 2012 19:28:46 -0500 Subject: [erlang-questions] rebar vs. escript vs. service vs. init.d Message-ID: <196E766E-3143-44FE-A32C-D9604AE936F6@nexttolast.com> I ran into an interesting bug today where in the default init script generated by rebar was experiencing some distress. A word about my setup, all of the versions of my app are in /opt/[appname]/[version]/ and the script was altered to not pwd as it was trying to look for /etc/etc/vm.args as a result when started by the service script. Invoking it via full path worked, as did /etc/init.d/appname after fixing the auto-discovered path. Once the paths worked escript failed to exec when run by service and the runner failed due to HOME not being defined when run by the service script. But most interestingly, it returned status 0, which caused puppet to erroneously report as successful. The solution was to update sysconfig to fake a HOME. This whole curious enterprise ultimately derives from some interesting assumptions about how the scripts should discover their environment. Has anyone else run into similar issues? -=-=- dave@REDACTED -=-=- From jeraymond@REDACTED Thu Mar 8 02:15:44 2012 From: jeraymond@REDACTED (Jeremy Raymond) Date: Wed, 7 Mar 2012 20:15:44 -0500 Subject: [erlang-questions] rebar vs. escript vs. service vs. init.d In-Reply-To: <196E766E-3143-44FE-A32C-D9604AE936F6@nexttolast.com> References: <196E766E-3143-44FE-A32C-D9604AE936F6@nexttolast.com> Message-ID: <-3658942028456756888@unknownmsgid> I recall having to update the generated start script as well with the full path to get it to work as an init.d startup script. Same issues. -- Jeremy On Mar 7, 2012, at 7:29 PM, David Goehrig wrote: > I ran into an interesting bug today where in the default init script generated by rebar was experiencing some distress. > > A word about my setup, all of the versions of my app are in /opt/[appname]/[version]/ and the script was altered to not pwd as it was trying to look for /etc/etc/vm.args as a result when started by the service script. Invoking it via full path worked, as did /etc/init.d/appname after fixing the auto-discovered path. > > Once the paths worked escript failed to exec when run by service and the runner failed due to HOME not being defined when run by the service script. But most interestingly, it returned status 0, which caused puppet to erroneously report as successful. The solution was to update sysconfig to fake a HOME. > > This whole curious enterprise ultimately derives from some interesting assumptions about how the scripts should discover their environment. > > Has anyone else run into similar issues? > > -=-=- dave@REDACTED -=-=- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From daniel@REDACTED Thu Mar 8 06:13:33 2012 From: daniel@REDACTED (Daniel Luna) Date: Thu, 8 Mar 2012 00:13:33 -0500 Subject: [erlang-questions] Erlang Community Conference In-Reply-To: <4F54FE0F.6090700@berlin.ccc.de> References: <4F54FE0F.6090700@berlin.ccc.de> Message-ID: This sounds awesome. I'm sad that I won't be near Berlin when it happens. Good luck guys. On 5 March 2012 12:55, John-Paul Bader wrote: > Hey, > > > I want to announce that a bunch of people here in Berlin/Germany are about > to organize a community driven erlang conference. > > We know that there are quite a lot of erlang related conferences by erlang > solutions already and we really like them, however we believe it would be > great to have an independant, community driven event as well. So we don't > want to compete with the existing conferences but add another nice one to > the mix. > > We are still at a very early stage as we are just gathering together all the > details and people so please consider any information on this eventual > consistent. We don't even have a name yet. Current project title is > "Erlounge Conference". > > Yet we would really like to get some early feedback on this from the > community so we don't run into the wrong direction right from the start. > > This is the brief description of what we want to do: > > What is it? > =========== > > The erlounge conference is a small single track conference about Erlang and > other programming languages with similar concepts like Scala, Clojure and > Haskell. > > It is a non-profit conference which is organized by community effort. The > entrance fee will be as low as we can make it so that everybody has a chance > attending the conference. Think of a range between 40-80 Euro. > > The talks will be about real projects and real people, about hacking and > inspiraton for what can be done with those languages. It will have talks for > beginners as well as experienced developers and in addition there will be > lighting talks for showing other cool stuff. > > Where is it? > ============ > > The conference will take place in Berlin / Germany > But we have to find a proper venue yet which is suitable for 200-400 people. > > When is it? > =========== > > We will try to make it happen in fall 2012. Could be one or two days - > preferrably on a weekend. > > What is it about? > ================= > > erlang, other functional languages and functional programming in general, > building systems that never fail, writing backends for the next big web app, > advantages of erlang over other languages like ruby, python, javascript > > > Any feedback is appreciated! We will have a dedicated interface for tracking > who would be interested to come later on. > > > > Kind regards, John > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Thu Mar 8 06:42:20 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 8 Mar 2012 06:42:20 +0100 Subject: [erlang-questions] Erlang Community Conference In-Reply-To: References: <4F54FE0F.6090700@berlin.ccc.de> Message-ID: <1BB47624-65D5-4EDD-95CD-7B1E903635D5@gmail.com> Excellent. +1 Regards, Zabrane On Mar 8, 2012, at 6:13 AM, Daniel Luna wrote: > This sounds awesome. I'm sad that I won't be near Berlin when it happens. > > Good luck guys. > > On 5 March 2012 12:55, John-Paul Bader wrote: >> Hey, >> >> >> I want to announce that a bunch of people here in Berlin/Germany are about >> to organize a community driven erlang conference. >> >> We know that there are quite a lot of erlang related conferences by erlang >> solutions already and we really like them, however we believe it would be >> great to have an independant, community driven event as well. So we don't >> want to compete with the existing conferences but add another nice one to >> the mix. >> >> We are still at a very early stage as we are just gathering together all the >> details and people so please consider any information on this eventual >> consistent. We don't even have a name yet. Current project title is >> "Erlounge Conference". >> >> Yet we would really like to get some early feedback on this from the >> community so we don't run into the wrong direction right from the start. >> >> This is the brief description of what we want to do: >> >> What is it? >> =========== >> >> The erlounge conference is a small single track conference about Erlang and >> other programming languages with similar concepts like Scala, Clojure and >> Haskell. >> >> It is a non-profit conference which is organized by community effort. The >> entrance fee will be as low as we can make it so that everybody has a chance >> attending the conference. Think of a range between 40-80 Euro. >> >> The talks will be about real projects and real people, about hacking and >> inspiraton for what can be done with those languages. It will have talks for >> beginners as well as experienced developers and in addition there will be >> lighting talks for showing other cool stuff. >> >> Where is it? >> ============ >> >> The conference will take place in Berlin / Germany >> But we have to find a proper venue yet which is suitable for 200-400 people. >> >> When is it? >> =========== >> >> We will try to make it happen in fall 2012. Could be one or two days - >> preferrably on a weekend. >> >> What is it about? >> ================= >> >> erlang, other functional languages and functional programming in general, >> building systems that never fail, writing backends for the next big web app, >> advantages of erlang over other languages like ruby, python, javascript >> >> >> Any feedback is appreciated! We will have a dedicated interface for tracking >> who would be interested to come later on. >> >> >> >> Kind regards, John >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ebegumisa@REDACTED Thu Mar 8 08:52:15 2012 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Thu, 08 Mar 2012 18:52:15 +1100 Subject: [erlang-questions] models for replicated/distributed processors In-Reply-To: <4F54DD83.5030305@meetinghouse.net> References: <4F54DD83.5030305@meetinghouse.net> Message-ID: I spent some time trying to gather wisdom on this subject too... In regards to fault-tolerant design with Erlang, I found Joe's paper on the Bluetail Mail Robustifier to be worth a million dollars (probably my favorite Erlang paper, I wish I had read it first before any Erlang books or tutorials): "Increasing the reliability of email services" by Joe Armstrong. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.851&rep=rep1&type=pdf The following thread on processes and fault tolerance got me some great responses: http://erlang.org/pipermail/erlang-questions/2011-January/055531.html ... especially Joe's description of take-over/error-recovery (the "replicated, synchronized processes" you refer to)... http://erlang.org/pipermail/erlang-questions/2011-January/055525.html ... and also Ulf's description of hot/cold standby and choosing the right recover state: http://erlang.org/pipermail/erlang-questions/2011-January/055519.html Hope those help. - Edmond - On Tue, 06 Mar 2012 02:36:35 +1100, Miles Fidelman wrote: > Hi Folks, > > I'm trying to think through various approaches to fault-tolerance for an > actor-based system architecture - generally around the notion of > replicated copies of actors operating on different nodes. > > Two questions to the assembled wisdom: > > 1. Has anybody done any work with replicated, synchronized processes > spread across multiple erlang nodes? If so, can you share anything > about architectural concepts? (Pointers to papers or slide decks would > be much appreciated). > > 2. A more specific question: I notice that spawn-link has a from that > allows creating a linked process on a different node - spawn_link(Node, > Module, Function, Args) -> pid() - which seems like a good start on > building supervision trees across nodes. But... there doesn't seem to > be an equivalent form of spawn_monitor - seems like spawn_opt/5 can be > used instead, but sort of curious about the omission. Anybody know the > story? > > Thanks, > > Miles Fidelman > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From dangud@REDACTED Thu Mar 8 08:54:27 2012 From: dangud@REDACTED (Dan Gudmundsson) Date: Thu, 8 Mar 2012 08:54:27 +0100 Subject: [erlang-questions] wxStaticText resizing In-Reply-To: <44b0c067-39b0-441f-9461-bffe59f7954b@o16g2000yqg.googlegroups.com> References: <44b0c067-39b0-441f-9461-bffe59f7954b@o16g2000yqg.googlegroups.com> Message-ID: You want the actual font size to change? If so you will have to write that yourself there is no such functionality in wx that I'm aware of. Something like this: go() -> wx:new(), Parent = wxFrame:new(wx:null(), ?wxID_ANY, "Foobar"), %% wxFrame:connect(Parent, close_window), Panel = wxPanel:new(Parent, []), %% Setup sizers TextSizer = wxBoxSizer:new(?wxVERTICAL), ClockText = wxStaticText:new(Panel,1,"00 : 00 : 00",[{style,?wxALIGN_CENTER}]), %% Add to sizers wxSizer:add(TextSizer,ClockText,[{flag,?wxEXPAND},{proportion,0}]), DrawArea = wxPanel:new(Panel, [{style, ?wxFULL_REPAINT_ON_RESIZE}]), wxSizer:add(TextSizer,DrawArea,[{flag,?wxEXPAND},{proportion,1}]), Fonts = create_fonts(), wxPanel:connect(DrawArea, paint, [{callback, fun(Ev, _) -> draw(Ev, Fonts) end}]), wxPanel:setSizer(Panel,TextSizer), wxFrame:show(Parent). create_fonts() -> [wxFont:new(Sz, ?wxFONTFAMILY_DEFAULT, ?wxFONTSTYLE_NORMAL, ?wxFONTWEIGHT_NORMAL) || Sz <- lists:reverse([8, 10, 12, 14, 16, 18, 24, 28, 40, 48, 60])]. draw(#wx{obj=Panel}, Fonts) -> DC = wxPaintDC:new(Panel), Sz = {W,H} = wxWindow:getClientSize(Panel), wxDC:drawRectangle(DC, {3,3,W-6, H-6}), Str = "00 : 00 : 00", Pos = set_font(Fonts, Str, DC, Sz), wxDC:drawText(DC, Str, Pos), wxPaintDC:destroy(DC). set_font([Font|Fs], Str, DC, Sz = {MaxW, MaxH}) -> wxDC:setFont(DC, Font), case wxDC:getTextExtent(DC, Str) of {W,H} when W < MaxW, H < MaxH -> {(MaxW - W) div 2, (MaxH - H) div 2}; _ -> set_font(Fs, Str, DC, Sz) end; set_font([], _, _, _) -> {0,0}. /Dan On Wed, Mar 7, 2012 at 5:03 PM, eigenfunction wrote: > I have a static text centered in a panel. When i resize the panel, i > would like the text to resize as well. This is my code: > > ? ?wxFrame:connect(Parent, close_window), > ? ?Panel = wxPanel:new(Parent, []), > > ? ?%% Setup sizers > ? ?TextSizer = wxBoxSizer:new(?wxVERTICAL), > ? ?ClockText = wxStaticText:new(Panel,1,"00 : 00 : 00",[{style,? > wxALIGN_CENTER}]), > > ? ?%% Add to sizers > ? ?wxSizer:add(TextSizer,ClockText,[{flag,?wxEXPAND},{proportion, > 1}]), > > ? ?wxPanel:setSizer(Panel,TextSizer), > ? ?wxFrame:show(Parent) > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ingela.andin@REDACTED Thu Mar 8 11:09:04 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 8 Mar 2012 11:09:04 +0100 Subject: [erlang-questions] New ssl vs Google Chrome In-Reply-To: <201203071839.q27IdPmv044280@pluto.hedeland.org> References: <201203071839.q27IdPmv044280@pluto.hedeland.org> Message-ID: Hi Per! 2012/3/7, Per Hedeland : > Ingela Andin wrote: >> >>We will have to look into this more, as a start see comments below: > > Thanks! > >>crypto:mpint/1 ofcourse is an overhad that could somehow be worked out >>of the equation. > > The three calls in public_key:sign/3 add upp to about a millisecond on > my slow test system. Acctually it is public_key:encrypt_private that will be used as signing function for your cipher suite, only the latest version of TLS uses public_key:sign for RSA. >>I tried it with a big key on my machine but does not appear to make such a >> big >>difference that it does for you. > > Right, you need a slow CPU.:-) On my workstation PC the same test gives > something like 0.2 seconds CPU for "old" and 0.6 seconds CPU for "new", > which would go unnoticed unless there were huge numbers of SSL > connections. Unfortunately our customers tend to use low-powered PowerPC > and the like, and still expect to have a usable SSL-webui (at least one > session:-). For a relevant number, 'openssl speed rsa4096' gives 2.0 > sign/sec on the PPC where I'm doing the test (62.7 on my PC - i.e. 16 ms > per sign - vs 54 ms for public_key:sign/3). > > But the CPU time needed to do a RSA sign is probably not the major > problem - using Firefox with "new" on the slow system works perfectly > fine, because Firefox knows how to do SSL in a server-friendly way - in > my test it's a single full SSL handshake and 5 additional connections > with session reuse, all 6 used to carry application data. It would be interesting if you could test how much gain you would get on your slow machin if calling crypto:sign or crypto:rsa_private_encrypt with the mpint-values directly. > Chrome opens 26 connnections, starting handshake on all of them, 7 > handshakes are actually completed (5 full and 2 reuse), and only 4 of > the connections do anything useful. But somehow "old" managed to handle > this stupidity with less CPU impact - actually it ended up with "only" > 19 connections, 11 completed handshakes (a single full and 10 reuse), > and 5 of them doing something useful. Old ssl had a bottleneck-process that all connections talked to so maybe less parallel processing happened to be beneficial for chromes stupidity. > If you want I can send you ssldump output off-list. Pleas do. Regards Ingela Erlang/OTP team - Ericsson AB From dmercer@REDACTED Thu Mar 8 15:23:30 2012 From: dmercer@REDACTED (David Mercer) Date: Thu, 8 Mar 2012 08:23:30 -0600 Subject: [erlang-questions] Reading a file before it has been completely written In-Reply-To: <002c01ccfc8d$56adc9c0$04095d40$@com> References: <000001ccfc87$4e9d2dd0$ebd78970$@com> <4F57A0C0.9030009@gmail.com> <002c01ccfc8d$56adc9c0$04095d40$@com> Message-ID: <00ad01ccfd37$0a1c57c0$1e550740$@com> On Wednesday, March 07, 2012, I wrote: > On Wednesday, March 07, 2012, Richard Carlsson wrote: > > > Trying to fiddle with exclusive locks is probably always > going > > to have corner cases. The only techniques you can trust to > practically > > always work and be portable across file systems are directory > creation > > and file renaming. So what Tony suggested is likely to be the best > > solution: create the file under another name or in a separate > > directory, > > and when it's completely written, rename it. > > I might go back to my renaming approach, which also had no failures > during > testing. I just attempt to rename the file to itself. If it fails, I > try > again 5 seconds later. For closure here, I went back to my approach of attempting to rename the file to itself before reading it. I'll let y'all know if I encounter any more corner cases. Cheers, DBM From johannes.govaerts@REDACTED Thu Mar 8 16:23:09 2012 From: johannes.govaerts@REDACTED (Johannes Govaerts) Date: Thu, 8 Mar 2012 16:23:09 +0100 Subject: [erlang-questions] gen_sctp throughput mystery In-Reply-To: <20120306104634.GA13185@erix.ericsson.se> References: <201203041237.03563.als@iinet.net.au> <201203041243.49277.als@iinet.net.au> <201203041319.00947.als@iinet.net.au> <20120306104634.GA13185@erix.ericsson.se> Message-ID: <20120308162309.53c958a6@jgo-laptop> Hi, Today I stumbled also across this issue. It turned out that you get this behavior when not specifying the recbuf option at the server. The default recbuf is only 1500 bytes. So when the client sends a package he waits for acknowledgment before sending a another package; and at the other side the server receives only one package and will only acknowledge when sacktimeout is triggered because he never receives a second package... When using a higher recbuf value there is no throughput issue anymore. Regards, Johannes On Tue, 6 Mar 2012 11:46:34 +0100 ext Raimo Niskanen wrote: > On Sun, Mar 04, 2012 at 01:19:00PM +1100, Anthony Shipman wrote: > > On Sun, 4 Mar 2012 12:43:49 pm Anthony Shipman wrote: > > > > The now() value shows the packets being written to the socket > > > > at 11ms intervals on the client but they are received at strict > > > > 200ms intervals on the server. The server is just doing a > > > > simple receive loop. > > > > > > > > Is there something in SCTP that limits the rate on an SCTP > > > > stream? > > > > > > I forgot to mention that the behaviour is the same for > > > sctp_nodelay=true. > > > > > > > Replying to my own question. It's the net.sctp.sack_timeout = 200 > > kernel parameter. But I don't understand why it should limit the > > server to 5 packets per second over a localhost link. > > Great finding if true... Can you elaborate? What happens when you > tweak this parameter? Note that 1 / 200ms is exactly 5 per second. > > I found this link: > http://sandarenu.blogspot.com/2009/11/reducing-time-between-sctp-message.html > And a quote from it it says: > That 200ms is a configuration based on the SCTP RFC. According to > that an acknowledgment(SACK) should be generated for at least every > second packet received, and SHOULD be generated within 200 ms of the > arrival of any unacknowledged DATA chunk. > So, might it be so that if you send just one data chunk there will be > a 200 ms delay before an ack? > > I have focused on the SCTP socket API, not dug into the RFC:s... > > > > > -- > > Anthony Shipman Mamas don't let your babies > > als@REDACTED grow up to be outsourced. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > -- Johannes Govaerts Nokia Siemens Networks phone: +32 473 36 65 28 mailto: johannes.govaerts@REDACTED From vances@REDACTED Thu Mar 8 16:58:19 2012 From: vances@REDACTED (Vance Shipley) Date: Thu, 8 Mar 2012 21:28:19 +0530 Subject: [erlang-questions] backquotes in re:replace/3 Message-ID: <20120308155816.GI22798@aluminum.wavenet.lk> Is there a syntax supported which allows us to insert a back reference in a replacement followed directly by digits? I haven't found one yet. Eshell V5.9 (abort with ^G) 1> re:replace("012345", "0(...)45", "\\1bcde"). [[<<"123">>,<<"bcde">>]] 2> re:replace("012345", "0(...)45", "\\16789"). [[<<>>]] 3> re:replace("012345", "0(...)45", "\\g{1}6789"). [<<"g{1}6789">>] -- -Vance From raimo+erlang-questions@REDACTED Thu Mar 8 17:52:10 2012 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 8 Mar 2012 17:52:10 +0100 Subject: [erlang-questions] gen_sctp throughput mystery In-Reply-To: <20120308162309.53c958a6@jgo-laptop> References: <201203041237.03563.als@iinet.net.au> <201203041243.49277.als@iinet.net.au> <201203041319.00947.als@iinet.net.au> <20120306104634.GA13185@erix.ericsson.se> <20120308162309.53c958a6@jgo-laptop> Message-ID: <20120308165210.GA28408@erix.ericsson.se> On Thu, Mar 08, 2012 at 04:23:09PM +0100, Johannes Govaerts wrote: > Hi, > > Today I stumbled also across this issue. It turned out that you get > this behavior when not specifying the recbuf option at the server. The > default recbuf is only 1500 bytes. So when the client sends a package > he waits for acknowledgment before sending a another package; and at > the other side the server receives only one package and will only > acknowledge when sacktimeout is triggered because he never receives a > second package... > > When using a higher recbuf value there is no throughput issue anymore. > > Regards, > Johannes That is great news (that there is an explanation). So the combination of default recbuf size of 1500 bytes in gen_sctp, the (Linux ?) kernel default net.sctp.sack_timeout = 200 (RFC recommended), and sending packets larger than recbuf size choaks the throughput to 1000/net.sctp.scack_timeout packets per second... What a gotcha! > > On Tue, 6 Mar 2012 11:46:34 +0100 > ext Raimo Niskanen wrote: > > > On Sun, Mar 04, 2012 at 01:19:00PM +1100, Anthony Shipman wrote: > > > On Sun, 4 Mar 2012 12:43:49 pm Anthony Shipman wrote: > > > > > The now() value shows the packets being written to the socket > > > > > at 11ms intervals on the client but they are received at strict > > > > > 200ms intervals on the server. The server is just doing a > > > > > simple receive loop. > > > > > > > > > > Is there something in SCTP that limits the rate on an SCTP > > > > > stream? > > > > > > > > I forgot to mention that the behaviour is the same for > > > > sctp_nodelay=true. > > > > > > > > > > Replying to my own question. It's the net.sctp.sack_timeout = 200 > > > kernel parameter. But I don't understand why it should limit the > > > server to 5 packets per second over a localhost link. > > > > Great finding if true... Can you elaborate? What happens when you > > tweak this parameter? Note that 1 / 200ms is exactly 5 per second. > > > > I found this link: > > http://sandarenu.blogspot.com/2009/11/reducing-time-between-sctp-message.html > > And a quote from it it says: > > That 200ms is a configuration based on the SCTP RFC. According to > > that an acknowledgment(SACK) should be generated for at least every > > second packet received, and SHOULD be generated within 200 ms of the > > arrival of any unacknowledged DATA chunk. > > So, might it be so that if you send just one data chunk there will be > > a 200 ms delay before an ack? > > > > I have focused on the SCTP socket API, not dug into the RFC:s... > > > > > > > > -- > > > Anthony Shipman Mamas don't let your babies > > > als@REDACTED grow up to be outsourced. > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Johannes Govaerts > Nokia Siemens Networks > phone: +32 473 36 65 28 > mailto: johannes.govaerts@REDACTED > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From yoursurrogategod@REDACTED Thu Mar 8 18:16:44 2012 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Thu, 8 Mar 2012 12:16:44 -0500 Subject: [erlang-questions] Getting erlang and cygwin to play along Message-ID: I read this little thread that I found on bing. http://erlang.2086793.n4.nabble.com/Installing-on-cygwin-td2550612.html The last e-mail said that I need to do a little bit of 'fussing' to get everything just peachy... what 'fussing'? Not sure what to do here :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From gustav.simonsson@REDACTED Thu Mar 8 19:01:12 2012 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Thu, 08 Mar 2012 18:01:12 -0000 (GMT) Subject: [erlang-questions] backquotes in re:replace/3 In-Reply-To: <20120308155816.GI22798@aluminum.wavenet.lk> Message-ID: <178afdee-be35-4afc-9b5e-db8a5a60639b@knuth> Is this what you want todo? re:replace("012345", "0(...)45", "x\\1\1"). [[<<"x">>,<<"123">>,<<1>>]] Regards, Gustav Simonsson Sent from my PC ----- Original Message ----- > From: "Vance Shipley" > To: erlang-questions@REDACTED > Sent: Thursday, 8 March, 2012 4:58:19 PM > Subject: [erlang-questions] backquotes in re:replace/3 > > Is there a syntax supported which allows us to insert a back > reference in a replacement followed directly by digits? I > haven't found one yet. > > Eshell V5.9 (abort with ^G) > 1> re:replace("012345", "0(...)45", "\\1bcde"). > [[<<"123">>,<<"bcde">>]] > 2> re:replace("012345", "0(...)45", "\\16789"). > [[<<>>]] > 3> re:replace("012345", "0(...)45", "\\g{1}6789"). > [<<"g{1}6789">>] > > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From eriksoe@REDACTED Thu Mar 8 19:41:02 2012 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Thu, 08 Mar 2012 19:41:02 +0100 Subject: [erlang-questions] backquotes in re:replace/3 In-Reply-To: <178afdee-be35-4afc-9b5e-db8a5a60639b@knuth> References: <178afdee-be35-4afc-9b5e-db8a5a60639b@knuth> Message-ID: <4F58FD3E.20900@gmail.com> That's not a digit; that's an ASCII 1. For the digit "1", you'd need 4> re:replace("012345", "0(...)45", "x\\1\61"). which of course doesn't work either. "man pcresyntax" really does indicate that "\g{N}" would be the expected PCRE syntax for the job: BACKREFERENCES \n reference by number (can be ambiguous) \gn reference by number \g{n} reference by number \g{-n} relative reference by number but "\\g" does not appear to be supported in any form... And the workaround I can think of - using an empty quoted section as a separator - doesn't work either: 2> re:replace("012345", "0(...)45", "\\1\\Q\\E6789"). [[<<"123">>,<<"QE6789">>]] /Erik Den 08-03-2012 19:01, Gustav Simonsson skrev: > Is this what you want todo? > > re:replace("012345", "0(...)45", "x\\1\1"). > [[<<"x">>,<<"123">>,<<1>>]] > > Regards, > Gustav Simonsson > > Sent from my PC > > ----- Original Message ----- >> From: "Vance Shipley" >> To: erlang-questions@REDACTED >> Sent: Thursday, 8 March, 2012 4:58:19 PM >> Subject: [erlang-questions] backquotes in re:replace/3 >> >> Is there a syntax supported which allows us to insert a back >> reference in a replacement followed directly by digits? I >> haven't found one yet. >> >> Eshell V5.9 (abort with ^G) >> 1> re:replace("012345", "0(...)45", "\\1bcde"). >> [[<<"123">>,<<"bcde">>]] >> 2> re:replace("012345", "0(...)45", "\\16789"). >> [[<<>>]] >> 3> re:replace("012345", "0(...)45", "\\g{1}6789"). >> [<<"g{1}6789">>] >> >> >> -- >> -Vance >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From als@REDACTED Thu Mar 8 22:07:24 2012 From: als@REDACTED (Anthony Shipman) Date: Fri, 9 Mar 2012 08:07:24 +1100 Subject: [erlang-questions] gen_sctp throughput mystery In-Reply-To: <20120308165210.GA28408@erix.ericsson.se> References: <201203041237.03563.als@iinet.net.au> <20120308162309.53c958a6@jgo-laptop> <20120308165210.GA28408@erix.ericsson.se> Message-ID: <201203090807.24154.als@iinet.net.au> On Fri, 9 Mar 2012 03:52:10 am Raimo Niskanen wrote: > That is great news (that there is an explanation). > > So the combination of default recbuf size of 1500 bytes in gen_sctp, > the (Linux ?) kernel default net.sctp.sack_timeout = 200 (RFC recommended), > and sending packets larger than recbuf size choaks the throughput to > 1000/net.sctp.scack_timeout packets per second... > > What a gotcha! > In inet_int.hrl in R15B, the default for recbuf is 1024, while sndbuf is 65536. When I set recbuf to 65536 in my test server the delays disappear. I can get up to 50MB/s from client to server before the eagain problem hits. -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From mail@REDACTED Fri Mar 9 00:05:36 2012 From: mail@REDACTED (Tim Fletcher) Date: Thu, 8 Mar 2012 15:05:36 -0800 (PST) Subject: [erlang-questions] Erlang Community Conference In-Reply-To: <4F54FE0F.6090700@berlin.ccc.de> References: <4F54FE0F.6090700@berlin.ccc.de> Message-ID: > Any feedback is appreciated! We will have a dedicated interface for > tracking who would be interested to come later on. Sounds good. Website and/or twitter account that we can follow would be helpful :) Cheers, Tim From jeffm@REDACTED Fri Mar 9 00:40:46 2012 From: jeffm@REDACTED (jm) Date: Fri, 09 Mar 2012 10:40:46 +1100 Subject: [erlang-questions] [BULK] Re: SNMP get,set,walk, etc In-Reply-To: <20120306063446.GA29324@mail.ipv6.dp.ua> References: <4F55A416.9070202@ghostgun.com> <20120306063446.GA29324@mail.ipv6.dp.ua> Message-ID: <4F59437E.4040307@ghostgun.com> Thanks will take a look. Jeff. On 6/03/12 5:34 PM, Artem Teslenko wrote: > Hi, > > Look at the repo below, it's a simple implementation of 'snmp walk': > > git://github.com/ates/snmpcl.git > > On Tue, 06 Mar 2012, jm wrote: > >> Are there any decent guides anyone can point me to that deal with using >> Erlang to retrieve SNMP measurements from devices such as network >> routers and switches? The stuff I've been able to dig up in the past >> has dealt mostly with how to add a SNMP agent to your own applications. >> >> Jeff. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From freeakk@REDACTED Fri Mar 9 09:59:58 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Fri, 9 Mar 2012 11:59:58 +0300 Subject: [erlang-questions] [BULK] Re: SNMP get,set,walk, etc In-Reply-To: <4F59437E.4040307@ghostgun.com> References: <4F55A416.9070202@ghostgun.com> <20120306063446.GA29324@mail.ipv6.dp.ua> <4F59437E.4040307@ghostgun.com> Message-ID: Here is a manual: http://www.erlang.org/doc/apps/snmp/snmp.pdf -- Best regards, Uvarov Michael From john.hughes@REDACTED Fri Mar 9 10:49:27 2012 From: john.hughes@REDACTED (John Hughes) Date: Fri, 9 Mar 2012 10:49:27 +0100 Subject: [erlang-questions] Serving PDF using Nitrogen/Inets In-Reply-To: References: <37BFB75479424820BA16E6B9EA531748@JohnsTablet2012> Message-ID: Thank you! Problem solved. That's a useful file to know about! John -----Ursprungligt meddelande----- From: Jesse Gumm Sent: Wednesday, March 07, 2012 10:41 PM To: Garrett Smith Cc: John Hughes ; erlang-questions@REDACTED Subject: Re: [erlang-questions] Serving PDF using Nitrogen/Inets Hi John, In your nitrogen installation, check out /etc/inets_httpd.env, which defines a few common mimetypes for inets. https://github.com/nitrogen/nitrogen/blob/master/rel/overlay/inets/etc/inets_httpd.erlenv -Jesse On Wed, Mar 7, 2012 at 3:09 PM, Garrett Smith wrote: > On Wed, Mar 7, 2012 at 10:35 AM, John Hughes > wrote: >> I?m having trouble serving static pdf files using nitrogen (running on >> inets >> for the time being). I think they?re being served with the wrong content >> type: my browser treats them as text and displays the PDF source; >> moreover, >> if I try to ?save the page? then a .txt is appended to the file name (I?m >> running on Windows 7). This happens if I visit a URL such as >> http://localhost:8000/file.pdf in my browser (with nitrogen running on >> the >> same machine). > > When I use httpd/inets with modlib, I need to explicitly configure > MIME types, even for images, css, etc. I wonder if Nitrogen does the > same thing but has omitted pdf. > > You're seeing other mime types come through? E.g. css -> text/css? > > Garrett > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm From emeka_1978@REDACTED Fri Mar 9 12:44:05 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Fri, 9 Mar 2012 03:44:05 -0800 (PST) Subject: [erlang-questions] Getting erlang and cygwin to play along In-Reply-To: References: Message-ID: Erlang should not be a problem at all, as cygwin automatically adds Windows environment variables on its path. You might have some problems with building applications that need some gcc compiler for the build. Only then are you going to need to do some extra work. In my experience, plain old erlang worked out of the box on my cygwin installation. On 8 Mrz., 18:16, "Yves S. Garret" wrote: > I read this little thread that I found on bing. > > http://erlang.2086793.n4.nabble.com/Installing-on-cygwin-td2550612.html > > The last e-mail said that I need to do a little bit of 'fussing' to get > everything just peachy... what 'fussing'? ?Not sure what to do here :-) > > _______________________________________________ > erlang-questions mailing list > erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions From per@REDACTED Fri Mar 9 15:18:05 2012 From: per@REDACTED (Per Hedeland) Date: Fri, 9 Mar 2012 15:18:05 +0100 (CET) Subject: [erlang-questions] New ssl vs Google Chrome In-Reply-To: Message-ID: <201203091418.q29EI5M1092245@pluto.hedeland.org> Ingela Andin wrote: > >2012/3/7, Per Hedeland : >> >> The three calls in public_key:sign/3 add upp to about a millisecond on >> my slow test system. > >Acctually it is public_key:encrypt_private that will be used as >signing function for your cipher suite, only the latest version of TLS >uses public_key:sign for RSA. Ah, I didn't bother following the code, just grabbed the 'sign'. But it's essentially the same thing I guess, in any case it takes pretty much exactly as long to run. >It would be interesting if you could test how much gain you would get >on your slow machin if calling crypto:sign or >crypto:rsa_private_encrypt with the mpint-values directly. I did that for encrypt_private/rsa_private_encrypt, but there was no improvement - no surprise really, it's the same mpint calls as in 'sign', where I measured them to take ~ 1 ms total (out of 1400+). >Old ssl had a bottleneck-process that all connections talked to so >maybe less parallel processing happened to be beneficial for chromes >stupidity. Yes, I suspect it's something like that - "new" is too efficient.:-) Actually in our case serializing the processing is almost an advantage, we don't want a connection from chrome to "take over the box". But having not only seen but actually worked on the innards of ssl_esock, I certainly don't want it back... Finding a way to achieve the same "level of non-parallelism" in "new", i.e. serializing just the processing in the individual steps of the handshake, should be doable (and possibly useful for low-end systems) I guess, but I can't quite find my way in the state machine yet. Serializing the ssl_accept() invocations was trivial but not a good idea.:-) >> If you want I can send you ssldump output off-list. > >Pleas do. Coming right up... Thanks! --Per From fancle.zhang@REDACTED Fri Mar 9 15:33:01 2012 From: fancle.zhang@REDACTED (=?UTF-8?B?5byg5Lyf5bOw?=) Date: Fri, 9 Mar 2012 22:33:01 +0800 Subject: [erlang-questions] (no subject) Message-ID: -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Fri Mar 9 16:20:45 2012 From: bourinov@REDACTED (Max Bourinov) Date: Fri, 9 Mar 2012 18:20:45 +0300 Subject: [erlang-questions] Record field position number? Message-ID: Hi Erlangers, Is there is a way to find a position number of the field in the record: 1. At compile time (record_info)? 2. At runtime (I think not). Motivation: I use lists module and its functions like lists:keyfind/3 for dealing with records. I don't want to hardcode number to target certain record's field. I afraid that if I change order of fields in my records my code will be broken (It will be indeed). So, I do believe that it is possible to do something like: ... -record(my_rec { name, age, gender }). % ListOfRecords = [#my_rec{name = "John"}, #my_rec{name = "Peter"}, #my_rec{name = "Marry"}], Value = "Peter", % case lists:keyfind(Value, ?get_filed_number(name, my_rec), ListOfRecords) of ... I hope it is possible to define some sort of makro that will do the job... Any ideas? Best regards, Max -------------- next part -------------- An HTML attachment was scrubbed... URL: From demeshchuk@REDACTED Fri Mar 9 16:23:31 2012 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Fri, 9 Mar 2012 19:23:31 +0400 Subject: [erlang-questions] Record field position number? In-Reply-To: References: Message-ID: #recordname.fieldname On Fri, Mar 9, 2012 at 7:20 PM, Max Bourinov wrote: > Hi Erlangers, > > Is there is a way to find a position number of the field in the record: > > 1. At compile time (record_info)? > 2. At runtime (I think not). > > Motivation: > I use lists module and its functions like lists:keyfind/3 for dealing with > records. I don't want to hardcode number to target certain record's field. I > afraid that if I change order of?fields?in my records my code will be broken > (It will be indeed). So, I do believe that it is possible to do something > like: > > ... > -record(my_rec { > ? name, age, gender > }). > % > ListOfRecords = [#my_rec{name = "John"},?#my_rec{name = > "Peter"},?#my_rec{name = "Marry"}], > Value = "Peter", > % > case lists:keyfind(Value, ?get_filed_number(name,?my_rec),?ListOfRecords) of > ... > > I hope it is possible to define some sort of makro that will do the job... > > Any ideas? > > Best regards, > Max > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best regards, Dmitry Demeshchuk From bourinov@REDACTED Fri Mar 9 16:31:23 2012 From: bourinov@REDACTED (Max Bourinov) Date: Fri, 9 Mar 2012 18:31:23 +0300 Subject: [erlang-questions] Record field position number? In-Reply-To: References: Message-ID: Thanks. It was heavy Friday for me. Best regards, Max On Fri, Mar 9, 2012 at 6:23 PM, Dmitry Demeshchuk wrote: > #recordname.fieldname > > On Fri, Mar 9, 2012 at 7:20 PM, Max Bourinov wrote: > > Hi Erlangers, > > > > Is there is a way to find a position number of the field in the record: > > > > 1. At compile time (record_info)? > > 2. At runtime (I think not). > > > > Motivation: > > I use lists module and its functions like lists:keyfind/3 for dealing > with > > records. I don't want to hardcode number to target certain record's > field. I > > afraid that if I change order of fields in my records my code will be > broken > > (It will be indeed). So, I do believe that it is possible to do something > > like: > > > > ... > > -record(my_rec { > > name, age, gender > > }). > > % > > ListOfRecords = [#my_rec{name = "John"}, #my_rec{name = > > "Peter"}, #my_rec{name = "Marry"}], > > Value = "Peter", > > % > > case lists:keyfind(Value, > ?get_filed_number(name, my_rec), ListOfRecords) of > > ... > > > > I hope it is possible to define some sort of makro that will do the > job... > > > > Any ideas? > > > > Best regards, > > Max > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > Best regards, > Dmitry Demeshchuk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From juanjo@REDACTED Fri Mar 9 16:34:37 2012 From: juanjo@REDACTED (Juan Jose Comellas) Date: Fri, 9 Mar 2012 12:34:37 -0300 Subject: [erlang-questions] Record field position number? In-Reply-To: References: Message-ID: You could use the following module that contains a parse transform for what you need: https://github.com/jcomellas/mlapi/blob/master/src/dynarec.erl When you include it in a module it will automatically create the functions that you need to access the records and fields defined in your module by name. To include it in the module where you have defined the records you do: -compile([{parse_transform, dynarec}]). The functions that will now be available are: get_value(field_name, Record) -> Record#record_name.field_name. set_value(field_name, Value, Record) when is_record(Record, record_name) -> Record#record_name{field_name = Value}. records() -> [record_name1, record_name2, ...]. fields(record_name) -> [field_name1, field_name2, ...]. new_record(record_name) -> #record_name{}. Hope it helps, Juanjo On Fri, Mar 9, 2012 at 12:20 PM, Max Bourinov wrote: > Hi Erlangers, > > Is there is a way to find a position number of the field in the record: > > 1. At compile time (record_info)? > 2. At runtime (I think not). > > Motivation: > I use lists module and its functions like lists:keyfind/3 for dealing with > records. I don't want to hardcode number to target certain record's field. > I afraid that if I change order of fields in my records my code will be > broken (It will be indeed). So, I do believe that it is possible to do > something like: > > ... > -record(my_rec { > name, age, gender > }). > % > ListOfRecords = [#my_rec{name = "John"}, #my_rec{name = > "Peter"}, #my_rec{name = "Marry"}], > Value = "Peter", > % > case lists:keyfind(Value, ?get_filed_number(name, my_rec), ListOfRecords) > of > ... > > I hope it is possible to define some sort of makro that will do the job... > > Any ideas? > > Best regards, > Max > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Fri Mar 9 16:46:56 2012 From: gumm@REDACTED (Jesse Gumm) Date: Fri, 9 Mar 2012 09:46:56 -0600 Subject: [erlang-questions] Serving PDF using Nitrogen/Inets In-Reply-To: References: <37BFB75479424820BA16E6B9EA531748@JohnsTablet2012> Message-ID: Excellent! Glad that helped. Take care, -Jesse -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 www.sigma-star.com @jessegumm On Mar 9, 2012 4:25 AM, "John Hughes" wrote: > Thank you! Problem solved. That's a useful file to know about! > > John > > -----Ursprungligt meddelande----- From: Jesse Gumm > Sent: Wednesday, March 07, 2012 10:41 PM > To: Garrett Smith > Cc: John Hughes ; erlang-questions@REDACTED > Subject: Re: [erlang-questions] Serving PDF using Nitrogen/Inets > > Hi John, > > In your nitrogen installation, check out /etc/inets_httpd.env, which > defines a few common mimetypes for inets. > > https://github.com/nitrogen/**nitrogen/blob/master/rel/** > overlay/inets/etc/inets_httpd.**erlenv > > -Jesse > > On Wed, Mar 7, 2012 at 3:09 PM, Garrett Smith wrote: > >> On Wed, Mar 7, 2012 at 10:35 AM, John Hughes >> wrote: >> >>> I?m having trouble serving static pdf files using nitrogen (running on >>> inets >>> for the time being). I think they?re being served with the wrong content >>> type: my browser treats them as text and displays the PDF source; >>> moreover, >>> if I try to ?save the page? then a .txt is appended to the file name (I?m >>> running on Windows 7). This happens if I visit a URL such as >>> http://localhost:8000/file.pdf in my browser (with nitrogen running on >>> the >>> same machine). >>> >> >> When I use httpd/inets with modlib, I need to explicitly configure >> MIME types, even for images, css, etc. I wonder if Nitrogen does the >> same thing but has omitted pdf. >> >> You're seeing other mime types come through? E.g. css -> text/css? >> >> Garrett >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866 || sigma-star.com || @jessegumm > -------------- next part -------------- An HTML attachment was scrubbed... URL: From emeka_1978@REDACTED Fri Mar 9 22:16:15 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Fri, 9 Mar 2012 13:16:15 -0800 (PST) Subject: [erlang-questions] wxStaticText resizing In-Reply-To: References: <44b0c067-39b0-441f-9461-bffe59f7954b@o16g2000yqg.googlegroups.com> Message-ID: <13211425.1584.1331327775194.JavaMail.geo-discussion-forums@ynnk21> I am starting to fall in love with wxWerlang. Thanks a lot. Am Donnerstag, 8. M?rz 2012 08:54:27 UTC+1 schrieb Dan Gudmundsson: > > You want the actual font size to change? > > If so you will have to write that yourself there is no such > functionality in wx that I'm aware of. > > Something like this: > > go() -> > wx:new(), > Parent = wxFrame:new(wx:null(), ?wxID_ANY, "Foobar"), > %% wxFrame:connect(Parent, close_window), > Panel = wxPanel:new(Parent, []), > > %% Setup sizers > TextSizer = wxBoxSizer:new(?wxVERTICAL), > ClockText = wxStaticText:new(Panel,1,"00 : 00 : > 00",[{style,?wxALIGN_CENTER}]), > > %% Add to sizers > wxSizer:add(TextSizer,ClockText,[{flag,?wxEXPAND},{proportion,0}]), > DrawArea = wxPanel:new(Panel, [{style, ?wxFULL_REPAINT_ON_RESIZE}]), > wxSizer:add(TextSizer,DrawArea,[{flag,?wxEXPAND},{proportion,1}]), > Fonts = create_fonts(), > wxPanel:connect(DrawArea, paint, [{callback, fun(Ev, _) -> > draw(Ev, Fonts) end}]), > wxPanel:setSizer(Panel,TextSizer), > wxFrame:show(Parent). > > create_fonts() -> > [wxFont:new(Sz, ?wxFONTFAMILY_DEFAULT, ?wxFONTSTYLE_NORMAL, > ?wxFONTWEIGHT_NORMAL) || > Sz <- lists:reverse([8, 10, 12, 14, 16, 18, 24, 28, 40, 48, 60])]. > > draw(#wx{obj=Panel}, Fonts) -> > DC = wxPaintDC:new(Panel), > Sz = {W,H} = wxWindow:getClientSize(Panel), > wxDC:drawRectangle(DC, {3,3,W-6, H-6}), > Str = "00 : 00 : 00", > Pos = set_font(Fonts, Str, DC, Sz), > wxDC:drawText(DC, Str, Pos), > wxPaintDC:destroy(DC). > > set_font([Font|Fs], Str, DC, Sz = {MaxW, MaxH}) -> > wxDC:setFont(DC, Font), > case wxDC:getTextExtent(DC, Str) of > {W,H} when W < MaxW, H < MaxH -> > {(MaxW - W) div 2, (MaxH - H) div 2}; > _ -> > set_font(Fs, Str, DC, Sz) > end; > set_font([], _, _, _) -> > {0,0}. > > /Dan > > On Wed, Mar 7, 2012 at 5:03 PM, eigenfunction > wrote: > > I have a static text centered in a panel. When i resize the panel, i > > would like the text to resize as well. This is my code: > > > > wxFrame:connect(Parent, close_window), > > Panel = wxPanel:new(Parent, []), > > > > %% Setup sizers > > TextSizer = wxBoxSizer:new(?wxVERTICAL), > > ClockText = wxStaticText:new(Panel,1,"00 : 00 : 00",[{style,? > > wxALIGN_CENTER}]), > > > > %% Add to sizers > > wxSizer:add(TextSizer,ClockText,[{flag,?wxEXPAND},{proportion, > > 1}]), > > > > wxPanel:setSizer(Panel,TextSizer), > > wxFrame:show(Parent) > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Fri Mar 9 23:37:03 2012 From: comptekki@REDACTED (Wes James) Date: Fri, 9 Mar 2012 15:37:03 -0700 Subject: [erlang-questions] binary strings and interspersing erlang variables Message-ID: I'm working with binary data like this: <<" ">> I then set Port= <<"8080">>; and now to put Port in to the html code above: <<<<" ">>/binary>> Is there a better way to do this? Thanks, Wes From lukas@REDACTED Fri Mar 9 23:57:50 2012 From: lukas@REDACTED (Lukas Larsson) Date: Fri, 9 Mar 2012 23:57:50 +0100 Subject: [erlang-questions] binary strings and interspersing erlang variables In-Reply-To: References: Message-ID: You might want to consider creating a iolist, i.e. [<<"html">>,Port,<<"">>]. Most if not all output interfaces allow you to mix lists and binaries like this. This makes it more elegant to generate complex structures (imo) and possibly faster as well as you do not combine the binaries until just before sending it over the socket. Lukas On Fri, Mar 9, 2012 at 11:37 PM, Wes James wrote: > I'm working with binary data like this: > > <<" > > > > > > ">> > > I then set > > Port= <<"8080">>; > > and now to put Port in to the html code above: > > <<<<" > > > > > > ">>/binary>> > > Is there a better way to do this? > > Thanks, > > Wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From comptekki@REDACTED Sat Mar 10 00:05:16 2012 From: comptekki@REDACTED (Wes James) Date: Fri, 9 Mar 2012 16:05:16 -0700 Subject: [erlang-questions] binary strings and interspersing erlang variables In-Reply-To: References: Message-ID: Ok, I'll try that. I'm testing cowboy and trying to put some variables in the binary string. Thanks, Wes ps. I just tried it and it worked. Thanks! On Fri, Mar 9, 2012 at 3:57 PM, Lukas Larsson wrote: > You might want to consider creating a iolist, i.e. > [<<"html">>,Port,<<"">>]. > > Most if not all output interfaces allow you to mix lists and binaries > like this. This makes it more elegant to generate complex structures > (imo) and possibly faster as well as you do not combine the binaries > until just before sending it over the socket. > > Lukas > > On Fri, Mar 9, 2012 at 11:37 PM, Wes James wrote: >> I'm working with binary data like this: >> >> <<" >> >> >> >> >> >> ">> >> >> I then set >> >> Port= <<"8080">>; >> >> and now to put Port in to the html code above: >> >> <<<<" >> >> >> >> >> >> ">>/binary>> >> >> Is there a better way to do this? >> >> Thanks, >> >> Wes >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From eriksoe@REDACTED Sat Mar 10 10:08:01 2012 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Sat, 10 Mar 2012 10:08:01 +0100 Subject: [erlang-questions] binary strings and interspersing erlang variables In-Reply-To: References: Message-ID: For completeness, I'll provide an answer which is closer to the original - and constructs a binary: The strings need not be marked as binaries; you can just put the string literals directly inside a binary constructor: <<" ">> Den 9. mar. 2012 23.37 skrev Wes James : > I'm working with binary data like this: > > <<" > > > > > > ">> > > I then set > > Port= <<"8080">>; > > and now to put Port in to the html code above: > > <<<<" > > > > > > ">>/binary>> > > Is there a better way to do this? > > Thanks, > > Wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tilman.holschuh@REDACTED Sat Mar 10 20:20:04 2012 From: tilman.holschuh@REDACTED (Tilman Holschuh) Date: Sat, 10 Mar 2012 11:20:04 -0800 Subject: [erlang-questions] Erlando monad performance comparsion Message-ID: Hi Guys, I was wondering if anybody has done some experiments on performance of erlando monads and would be willing to share their results. I'm interested in how to create such a performance test. How would a comparison of erlando monads vs. nested case statements vs. chained function calls look like? Which monads that come with erlando are worth considering? Cheers - Tilman From comptekki@REDACTED Sun Mar 11 00:16:38 2012 From: comptekki@REDACTED (Wes James) Date: Sat, 10 Mar 2012 16:16:38 -0700 Subject: [erlang-questions] binary strings and interspersing erlang variables In-Reply-To: References: Message-ID: On Sat, Mar 10, 2012 at 2:08 AM, Erik S?e S?rensen wrote: > For completeness, I'll provide an answer which is closer to the original - > and constructs a binary: > The strings need not be marked as binaries; you can just put the string > literals directly inside a binary constructor: > > <<" > > > > > > ">> Thanks - learn something new all the time :) I tried it and it work great! -wes From jws@REDACTED Sun Mar 11 02:03:06 2012 From: jws@REDACTED (Jeff Schultz) Date: Sun, 11 Mar 2012 12:03:06 +1100 Subject: [erlang-questions] object vs. actor (Re: Massive Numbers of Actors vs. Massive Numbers of Objects vs. ????) In-Reply-To: <4F4FD12A.6000202@meetinghouse.net> References: <4F4EB0BB.8000201@meetinghouse.net> <4F4FA98D.20503@meetinghouse.net> <4F4FD12A.6000202@meetinghouse.net> Message-ID: <20120311010306.GA15464@mulga.csse.unimelb.edu.au> On Thu, Mar 01, 2012 at 02:42:34PM -0500, Miles Fidelman wrote: > Garrett Smith wrote: >> Erlang processes aren't persisted, but their state can be, quite easily. >> A trivial example, in an Erlang shell: >> 1> Add = fun(X, Y) -> X + Y end. >> 2> Add(10, 10). >> 100 >> 3> file:write_file("addfun", term_to_binary(Add)). >> 4> init:stop(). >> And restart Erlang shell: >> 1> {ok, Bin} = file:read_file("addfun"). >> 2> Add = binary_to_term(Bin). >> 3> Add(10, 10). >> 100 > That's not persisting a process' state, though. It's not so easy to say: > 1. spawn, say, 10,000 gen_fsms, > 2. run for a while > 3. save the internal state of those 10,000 state machines > 4. stop the shell (really the VM) > 5. restart > 6. restore those state machines, complete with their original PIDs It's a small part of the overall application you're building, but doing almost all of that is pretty easy, and rather fun. Here's a simple example: -module(per). -export([start/1]). loop(State) -> io:format("Looping from ~p~n", [State]), receive {inc, X} -> io:format("Incrementing ~p by ~p~n", [State, X]), loop(State + X); suspend -> io:format("Suspending with state ~p~n", [State]), fun() -> loop(State) end end. start(InitialValue) -> outer(loop(InitialValue)). outer(Computation) -> outer(Computation()). 38> Pid = spawn(per, start, [10]). Looping from 10 <0.121.0> 39> Pid ! {inc, 5}. Incrementing 10 by 5 {inc,5} Looping from 15 40> Pid ! suspend. Suspending with state 15 suspend Looping from 15 41> Pid ! {inc, 3}. Incrementing 15 by 3 {inc,3} Looping from 18 42> Pid ! suspend. Suspending with state 18 suspend Looping from 18 Note that the fun returned from loop/1 on receipt of the suspend message captures the current State. This doesn't do your point 6. The process running loop/1 gets a new PID each time it's restarted, but if you're depending on Erlang PIDs to find a *persistent* resource, you're going to have trouble the first time something goes down anyway. Jeff Schultz > Fairly straightforward to: > 1. spawn 10,000 gen_fsms > 2. add their PIDs to a name-to-PID lookup table > 3. include logic such that an inactive fsm > - times out > - writes it's state to a file > - updates the name-to-PID table to point the file > - kill itself > 4. write a listener that takes incoming messages, identifies the associated > fsm by name and either > - passes the message to the file, or > - loads and restarts the process, then passes the messages > What I've really done is rewrite erlang's hibernation functionality. From jws@REDACTED Sun Mar 11 02:58:31 2012 From: jws@REDACTED (Jeff Schultz) Date: Sun, 11 Mar 2012 12:58:31 +1100 Subject: [erlang-questions] object vs. actor (Re: Massive Numbers of Actors vs. Massive Numbers of Objects vs. ????) In-Reply-To: <20120311010306.GA15464@mulga.csse.unimelb.edu.au> References: <4F4EB0BB.8000201@meetinghouse.net> <4F4FA98D.20503@meetinghouse.net> <4F4FD12A.6000202@meetinghouse.net> <20120311010306.GA15464@mulga.csse.unimelb.edu.au> Message-ID: <20120311015831.GA18978@mulga.csse.unimelb.edu.au> On Sun, Mar 11, 2012 at 12:03:06PM +1100, Jeff Schultz wrote: > Note that the fun returned from loop/1 on receipt of the suspend > message captures the current State. Of course, tricks like this are very brittle. The serialised version of the data produced by term_to_binary/1 won't reliably unserialise if you change Erlang release or update any of your program's code that the fun used. Jeff Schultz From james@REDACTED Sun Mar 11 05:07:56 2012 From: james@REDACTED (james) Date: Sun, 11 Mar 2012 04:07:56 +0000 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: <4F4FA9A4.9050909@gmail.com> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> Message-ID: <4F5C251C.7090803@mansionfamily.plus.com> >I guess we will just agree to disagree on this. What is there to 'agree to disagree' about? It is entirely clear that you can sell GPL code. You just have to give the sources to your customers, and you can't stop any of your customers from giving away (or selling) either modified or unmodified forms. In practice you are arguably right - you need to add value through services. But what you said is simply incorrect. I suspect I agree with you in terms of how practical GPL code is and I hardly ever use a GPL library (and hence would never contribute to one). But your statement is, nevertheless, plainy wrong as it stands. I suspect that in practice there ARE ways around it. It does appear to me (and no I'm not a lwayer), for example, that you can combine GPL and non-GPL code and run it for your own use. The 'problem' occurs when you give someone else a compiled binary that includes both - then you are obligated to give them your sources under GPL. Remember that GPL is a copyright licence. It applies when there is copying. But if you sell them your sources under non-GPL, and they combine with GPL components for their own use (and do not distribute the resut: doing so would be in violation of GPL or your terms) then it is certainly unclear whether there is any problem at all. Certainly, my reading is that the GPL gives rights to a person (or entity) that you copy the code to in some form. But I don't think it gives rights to someone else that you didn't copy the code to (including, for example, the author of the GPL code or some interested third party). Has to be said, I have argued with GPL code authors about local combination and running. One claimed that the act of linking and then running the code in the computer was a form of copying, and that my code would become GPL'd. But I think the licence says that I have to give a copy of my code to the recipient of the copy of the GPL'd code; and not anyone else. Also, I can relicence back to non-GPL at will (on code I own). So its not at all clear that the fact that there was a version of my code that the computer ran under GPL is actually an issue, because the computer isn't an entity that can receive the right to further distribute the code that it ran. And no-one else has received a right to demand a copy of the code I copied to the coputer - *it* didn't copy it to *them* after all. I think the GPL is rather poor. Too many terms are woolly, and I'm not sure that the apparent objectives are met by mere copyright (ie by copying, rather than by usage). Certainly some people using it seem to want to impact on usage, and that alone suggests a problem. Its also not clear who I can demand the sources from for GPL code on the DVD coverdisk on a Linux fan-mag that I buy in a newsagent. The newsagent - a corner store maybe, or a general chain? The distribution chain? The DVD duplication company? The magazine publisher? None of these businesses is generally capable of fulfilling an obligation to provide the sources in the mandated 'preferred form' unless its all on the disk already; and yet it is arguably in no-one's interest that they actually be accountable (well, certainly not Mr Singh the cornershop newsagent that stocks a magazine I want to buy - I like being able to pick these things up on spec). James From michael.eugene.turner@REDACTED Sun Mar 11 05:40:56 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sun, 11 Mar 2012 13:40:56 +0900 Subject: [erlang-questions] object vs. actor (Re: Massive Numbers of Actors vs. Massive Numbers of Objects vs. ????) In-Reply-To: <20120311015831.GA18978@mulga.csse.unimelb.edu.au> References: <4F4EB0BB.8000201@meetinghouse.net> <4F4FA98D.20503@meetinghouse.net> <4F4FD12A.6000202@meetinghouse.net> <20120311010306.GA15464@mulga.csse.unimelb.edu.au> <20120311015831.GA18978@mulga.csse.unimelb.edu.au> Message-ID: > .... The serialised version > of the data produced by term_to_binary/1 won't reliably unserialise if > you change Erlang release or update any of your program's code that the > fun used. Does term_to_binary/1 on a fun produce BEAM code? I don't think so, and that's the only reason (besides using deprecated functions in the fun) that I can think of, for why it would break after an Erlang release or two. And let's say you updated other code in your application. If the fun accesses the changed module, it'll call the new code -- the only reason it would break, I think, is if you violated some basic encapsulation discipline in the module, or the module's interface changed significantly. The idea of persisting Erlang funs in this way makes me queasy. Then again, anything like non-plain-text code embedded in data make my skin crawl. (I think there's something in the Old Testament about how it's a sin in the sight of God.) The conditions for making it work reliably in Erlang don't seem to be difficult to meet, however. -michael turner On Sun, Mar 11, 2012 at 10:58 AM, Jeff Schultz wrote: > On Sun, Mar 11, 2012 at 12:03:06PM +1100, Jeff Schultz wrote: >> Note that the fun returned from loop/1 on receipt of the suspend >> message captures the current State. > > Of course, tricks like this are very brittle. ?The serialised version > of the data produced by term_to_binary/1 won't reliably unserialise if > you change Erlang release or update any of your program's code that the > fun used. > > > ? ?Jeff Schultz > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mjtruog@REDACTED Sun Mar 11 08:56:02 2012 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 10 Mar 2012 23:56:02 -0800 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: <4F5C251C.7090803@mansionfamily.plus.com> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> Message-ID: <4F5C5A92.6020505@gmail.com> On 03/10/2012 08:07 PM, james wrote: > >I guess we will just agree to disagree on this. > > What is there to 'agree to disagree' about? > There are two sides to every story. A good link to some discussion of pro-GPL versus pro-BSD can be found here: http://news.slashdot.org/story/99/06/23/1313224/featuregpl-vs-bsd I am sure you can find many other links that show similar discussions about why someone might dislike using the GPL. A good summary, done by an anonymous coward (at the previously mentioned link) is below: The GPL license is conducive to liberating software. The BSD license is conducive to liberating people. With the GPL license, the software maintains more of the freedom than the programmers who work on it. With the BSD licenses, the programmers maintain more of the freedom with what they are allowed to do with derivative code. I am not a lawyer and this is not legal advice. From max.lapshin@REDACTED Sun Mar 11 09:12:48 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 11 Mar 2012 11:12:48 +0300 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: <4F5C5A92.6020505@gmail.com> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> Message-ID: It is very easy about GPL. There is absolutely no sense to use GPL for infrastructure and easy-repeatable libraries like marshalling or logging, because it will block their development. But it is a very good idea to use restrictive GPL or AGPL for complicated libraries with very expensive business logic. Best example is libx264. It is one of the best encoders available and there are almost no commercial replacements for it than can compare. And it is GPL. If you want to create a hardware box using this library and sell it, so go to Jason and pay money to receive the same source under non-GPL. It is great, because there are no replacements and these guys have shared this excelent library with community for free. So let business pay for it. So, when I hear "I will not use GPL because I can't use other's work for free" I always ask myself: "hey, what have _you_ done to others? You are going to earn money with others people work. What are you sharing with others?". Usually there are no pull-requests from such people, no commitment. And such people are usually too lazy just to write email and ask about conditions for non-GPL distribution. From mjtruog@REDACTED Sun Mar 11 11:40:52 2012 From: mjtruog@REDACTED (Michael Truog) Date: Sun, 11 Mar 2012 03:40:52 -0700 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> Message-ID: <4F5C8134.10908@gmail.com> On 03/11/2012 12:12 AM, Max Lapshin wrote: > So, when I hear "I will not use GPL because I can't use other's work > for free" I always ask myself: "hey, what have _you_ done to others? > You are going to earn money with others people work. What are you > sharing with others?". Usually there are no pull-requests from such > people, no commitment. And such people are usually too lazy just to > write email and ask about conditions for non-GPL distribution You are bringing up a separate subject, which I don't believe was mentioned much. I believe this is the simplicity of how BSD code can be put into GPL codebases, but the complication with going the opposite route, from GPL code to a BSD codebase. When you judge this transition based on the simplicity of each license, it is obvious the GPL is much more verbose with the various declarations of software freedom. I guess I would be more concerned about the limits to software reuse, which the GPL is meant to limit. If the author doesn't believe a GPL license is proper for their project, I have complete confidence they would publicly change the license. I have had communications with developers of GPL code in the past about changing their license to a BSD, however, it is very common for people to have a more religious attachment to one, or the other. So, perhaps my experience has been different from yours. However, I would be interested to know what GPL software you think people would want as BSD, since making that need public might help promote its change publicly. From thomas.elsgaard@REDACTED Sun Mar 11 17:52:15 2012 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Sun, 11 Mar 2012 17:52:15 +0100 Subject: [erlang-questions] Erlang social meetings in Copenhagen ? Message-ID: Hello! Is anybody interested in doing a monthly meeting in the Copenhagen area and play around with Erlang and drink some coke and have fun? It could be at my employer's office facilities in ?restaden, just next to Fields, we have a decent coffee ;-) Let me know if anybody is interested, and i will try arrange something. Best Regards Thomas Elsgaard From shahrdad1@REDACTED Sun Mar 11 18:09:37 2012 From: shahrdad1@REDACTED (Shahrdad Shadab) Date: Sun, 11 Mar 2012 13:09:37 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers Message-ID: When I was learning Erlang and understanding its capabilities I really cannot find a satisfactory answer to the question that why in North America companies like former BEA, former Sun, Oracle , ... use Java to build commercial application servers instead of Erlang? >From technical perspective such decision doesn't make any sense to me for following reasons: _Java is not a fault tolerant. _Java performance is nowhere near Erlang. _Concurrent programming in Java is a pain. _J2ee Technology introduced as add on to java to make communication cross servers possible (i.e web services XML SCHEMA, WSDL) is unreasonably and grotesquely complicated. (This complication is dictated by the technology and not by the problem domain) _Java is not distributed language (No asynch communication is possible without JMS, also RMI stub solution is more complicated than it should be). and many more reasons I can list here. Thanks in advance Shahrdad -- Software Architect & Computer Scientist -------------- next part -------------- An HTML attachment was scrubbed... URL: From sunwood360@REDACTED Sun Mar 11 21:43:45 2012 From: sunwood360@REDACTED (envelopes envelopes) Date: Sun, 11 Mar 2012 13:43:45 -0700 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: Message-ID: simple answer: because erlang was not invented in U. S. thought it is a great language. On Mar 11, 2012 10:09 AM, "Shahrdad Shadab" wrote: > When I was learning Erlang and understanding its capabilities I really > cannot find a satisfactory answer to the question that > why in North America companies like former BEA, former Sun, Oracle , ... > use Java to build commercial application servers instead of Erlang? > From technical perspective such decision doesn't make any sense to me for > following reasons: > > _Java is not a fault tolerant. > _Java performance is nowhere near Erlang. > _Concurrent programming in Java is a pain. > _J2ee Technology introduced as add on to java to make communication cross > servers possible (i.e web services XML SCHEMA, WSDL) is unreasonably and > grotesquely complicated. (This complication is dictated by the technology > and not by the problem domain) > _Java is not distributed language (No asynch communication is possible > without JMS, also RMI stub solution is more complicated than it should be). > > and many more reasons I can list here. > > Thanks in advance > Shahrdad > -- > Software Architect & Computer Scientist > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Sun Mar 11 22:04:33 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Sun, 11 Mar 2012 22:04:33 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: Message-ID: <4F5D1361.9060604@gmail.com> Nobody has been fired for choosing Java. See Mike Williams slides from the London Erlang Factory 2011: http://www.erlang-factory.com/conference/London2011/speakers/MikeWilliams Digging a bit deeper it comes down to risk management and most big companies has a strong dislike for anything new and different since that reeks risk to them. Take a look at all the start-ups in the US that is using Erlang and asking for Erlangers. There Erlang has been chosen since it is the right fit for the problem at hand. For a big company with a lot of legacy Java code and people trained in Java it is far from obvious that a switch to Erlang will be the right choice. I would actually get a bit nervous if management accepted such a shift without a thorough investigation and even if that investigation gave a go-ahead to do a shift I would be nervous since such a fundamental break with the past only happens when a company is staring into the abyss of a pending bankruptcy!! I was one of the two guys behind a product made in Erlang in Motorola. One of the main reasons for getting the go-ahead to that project was that we were building a new product and did not have to throw anything out. Eventually we shipped the product as a beta to a single customer, but the reluctance to bet on something as strange as Erlang for a "real" contract is still around and I will actually bet on the final product being written from scratch in C or Java since that is what the managers and old school architects feel most at ease with! The history of technology is full of this kind of stories... and one day some new technology will come along and overturn whatever kingdom Erlang might have build. The circle of life continues. Cheers, Torben On 11/3/12 18:09 , Shahrdad Shadab wrote: > When I was learning Erlang and understanding its capabilities I really > cannot find a satisfactory answer to the question that > why in North America companies like former BEA, former Sun, Oracle , > ... use Java to build commercial application servers instead of Erlang? > From technical perspective such decision doesn't make any sense to me > for following reasons: > > _Java is not a fault tolerant. > _Java performance is nowhere near Erlang. > _Concurrent programming in Java is a pain. > _J2ee Technology introduced as add on to java to make communication > cross servers possible (i.e web services XML SCHEMA, WSDL) is > unreasonably and grotesquely complicated. (This complication is > dictated by the technology and not by the problem domain) > _Java is not distributed language (No asynch communication is possible > without JMS, also RMI stub solution is more complicated than it should > be). > > and many more reasons I can list here. > > Thanks in advance > Shahrdad > -- > Software Architect & Computer Scientist > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From james@REDACTED Sun Mar 11 22:27:02 2012 From: james@REDACTED (james) Date: Sun, 11 Mar 2012 21:27:02 +0000 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: <4F5C5A92.6020505@gmail.com> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> Message-ID: <4F5D18A6.3030106@mansionfamily.plus.com> > There are two sides to every story. A good link to some discussion of pro-GPL versus pro-BSD can be found here: >The GPL license is conducive to liberating software. >The BSD license is conducive to liberating people. How is this relevant? I don't like the GPL. So what? What you said is simply wrong. What you said is: > This is a little misleading. You can never sell anything that contains GPL software Whether BSD or GPL is better in some way (or even in every way) is not relevant at all to whether you have materially misrepresented what GPL requires, and is simply diversion. Which clause of the GPL justifies your statement? They are numbered, it should be straightforward for you to identify the one you think has this effect. I think its bad to let a clearly erroneous statement go unchallenged and fester malignantly in the archives for posterity. James From paperless@REDACTED Sun Mar 11 23:45:37 2012 From: paperless@REDACTED (Tim McNamara) Date: Mon, 12 Mar 2012 11:45:37 +1300 Subject: [erlang-questions] Thoughts of someone new to Erlang Message-ID: Hello all, I am coming to terms with Erlang after lots of experience with Python and other dynamic languages. Just because you only have first impressions once, and it's often quite useful to get outsiders' views. ### Suggestions for improvement ## Dense console output I would really prefer if the console added spaces between terms, after commas. I find the following quite hard to digest when terms get larger than this toy example. 1> [{weather, "sunny"}, {city, "Sydney"}]. [{weather,"sunny"},{city,"Sydney"}] ## Monotone console I spend most of my time in Python in iPython, which adds syntax highlighting and a few other conveniences. It would be wonderful if some level of colour could be introduced to the interactive shell. ## Clumsy/non-standard JSON handling This reflects my web-bias and perhaps my reliance on a large standard library, however there seem to be many ways of handling JSON data. Each library seems to do things slightly differently. I don't know which is standard. I want to make an HTTP GET call, pull in JSON and have it appear as a prop list. [edit: I think I have found this https://github.com/lambder/jsonerl] ## No hash map implementation (?) Does Erlang have a hash map implementation that works in constant time? Perhaps I should just push through the culture shock and learn to love the list! ## Ubuntu/Debian packaging It took me a while to figure out that the package `erlang-manpages` is separate from `erlang-docs`. It would be great if an official PPA could be built. They're wonderful. [From memory, I remember someone else on the mailing list saying that PPAs shouldn't be relied upon. However, as they're GPG signed, I think that this is a non-issue.] ### Things that I really enjoy ## OTP I had no idea that OTP was so powerful. In fact, when I first read about Erlang a few years ago, OTP seemed like a huge hurdle that made Erlang more difficult to manage as a whole. However, after taking some time, It seems like a phenomenal way to built software. I'm sure that it's going to be great that most applications have a fairly similar structure and so forth. ## rebar, other tools There are some amazing tools that support software development. I haven't built any large projects, but it seems like deployment has really been something that people have been focusing on. ## Helpful community I've found everyone on IRC to be really helpful. Erlang is certainly smaller, but this feels fairly familiar to me as a New Zealander anyway. We get used to the fact that other places, such as the USA & Europe, have bigger events. I love the fact that Erlang Solutions and others have taken the effort to record conference talks. They have been a huge help over the last few weeks of coming to terms with the language. Kind regards, Tim McNamara paperlessprojects.com \\ ?@timClicks? \\?timmcnamara.co.nz From mjtruog@REDACTED Mon Mar 12 00:27:21 2012 From: mjtruog@REDACTED (Michael Truog) Date: Sun, 11 Mar 2012 16:27:21 -0700 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: <4F5D18A6.3030106@mansionfamily.plus.com> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> Message-ID: <4F5D34D9.40009@gmail.com> On 03/11/2012 02:27 PM, james wrote: > > There are two sides to every story. A good link to some discussion of pro-GPL versus pro-BSD can be found here: > >The GPL license is conducive to liberating software. > >The BSD license is conducive to liberating people. > > How is this relevant? I don't like the GPL. So what? > > What you said is simply wrong. What you said is: > > > This is a little misleading. You can never sell anything that contains GPL software > > Whether BSD or GPL is better in some way (or even in every way) is not relevant at all to whether you have materially misrepresented what GPL requires, and is simply diversion. > > Which clause of the GPL justifies your statement? They are numbered, it should be straightforward for you to identify the one you think has this effect. > > I think its bad to let a clearly erroneous statement go unchallenged and fester malignantly in the archives for posterity. > > James > I can understand that this subject is a tough subject to reach agreement on, due to our own viewpoints of what open source software is, free software, freedom to reuse software, etc.. So, I don't expect us to agree on the specifics. When looking at the newest version, v3 (http://www.gnu.org/copyleft/gpl.html), the relevant numbers are: #5 and #6. You may notice the paragraph hiding at the bottom of the page with the first sentence that says "The GNU General Public License does not permit incorporating your program into proprietary programs.", which is a useful summary of the problem. Any modifications of GPL software retains the GPL license and must be distributed with the source code. This is good for making sure that modifications always feed back into the open source community. However, this is bad for any proprietary changes that give a company a competitive advantage. I don't regard my previous statements as erroneous, just simply more abstract. - Michael From mattevans123@REDACTED Mon Mar 12 01:00:25 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Sun, 11 Mar 2012 20:00:25 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F5D1361.9060604@gmail.com> References: , <4F5D1361.9060604@gmail.com> Message-ID: Pretty much this. I've recently been introduced to Java, and unfortunately what most of the pointy-haired managers don't realize is there is a huge difference between Java the language (which isn't that bad), and Java the application environment (which is complex and terrible). Once you get Java+Spring+Hibernate+20 other frameworks and libraries you are left with a VERY hard to maintain and complex application. In the end it often only has a small amount of real Java - the rest is pages of impossible to maintain XML. See this blog: http://www.jillesvangurp.com/2009/10/07/server-side-development-sucks/ The other problem is one of control. Even if the managers, and the rest of a development team don't know Java, it looks very familiar to C++ and the C family of languages. If you propose Erlang, they see that the language looks very different, and that scares people. Suddenly they are not in control anymore, feel less valuable to the organization; it makes them feel threatened. We as a community can help to remove many of the negative perceptions people have about Erlang. It's perceived to be slow, although in many cases that's caused by lousy code from newbies (JIT would be nice though ;-). It's also hard as a newbie to write good code since the concepts are very different. You can knock up a simple Erlang application quite quickly, but it might not perform that well. What you end up with are newbies producing poor code, getting frustrated and giving up. I've read many stories on Reddit and similar places of companies starting an Erlang project, getting bad results and throwing it out. It give a very bad impression on the language. More often than not someone will come along later, refactor the code, and get stellar performance - but by then it's too late. I think tidying up some modules, and getting the word out there on how to write proper Erlang apps would help a lot. Things like E2 http://e2project.org/ are a good start along this path. Projects like rebar make app. packaging simpler. Basically we need more consistency across the board. Also having more languages running on BEAM could make the take-up easier (as well as Erjang on the JVM). Finally I would say this. As Torben mentioned, no one gets fired for choosing Java. More to the point, no one gets fired for choosing Oracle. I think Ericsson is doing a more than stellar job of maintaining Erlang. But is this also a problem? Sometimes I wonder if it would be better if Erlang was "owned" by Ericsson, Klarna, Basho, Trifork, Erlang Solutions, Springsource/VMware (RabbitMQ). Basically have it owned by a consortium of companies. Would that make managers more comfortable taking on Erlang? I could go on, but that's my 2 cents for now. Matt Date: Sun, 11 Mar 2012 22:04:33 +0100 From: torben.lehoff@REDACTED To: shahrdad1@REDACTED CC: erlang-questions@REDACTED Subject: Re: [erlang-questions] Erlang is the best choice for building commercial application servers Nobody has been fired for choosing Java. See Mike Williams slides from the London Erlang Factory 2011: http://www.erlang-factory.com/conference/London2011/speakers/MikeWilliams Digging a bit deeper it comes down to risk management and most big companies has a strong dislike for anything new and different since that reeks risk to them. Take a look at all the start-ups in the US that is using Erlang and asking for Erlangers. There Erlang has been chosen since it is the right fit for the problem at hand. For a big company with a lot of legacy Java code and people trained in Java it is far from obvious that a switch to Erlang will be the right choice. I would actually get a bit nervous if management accepted such a shift without a thorough investigation and even if that investigation gave a go-ahead to do a shift I would be nervous since such a fundamental break with the past only happens when a company is staring into the abyss of a pending bankruptcy!! I was one of the two guys behind a product made in Erlang in Motorola. One of the main reasons for getting the go-ahead to that project was that we were building a new product and did not have to throw anything out. Eventually we shipped the product as a beta to a single customer, but the reluctance to bet on something as strange as Erlang for a "real" contract is still around and I will actually bet on the final product being written from scratch in C or Java since that is what the managers and old school architects feel most at ease with! The history of technology is full of this kind of stories... and one day some new technology will come along and overturn whatever kingdom Erlang might have build. The circle of life continues. Cheers, Torben On 11/3/12 18:09 , Shahrdad Shadab wrote: When I was learning Erlang and understanding its capabilities I really cannot find a satisfactory answer to the question that why in North America companies like former BEA, former Sun, Oracle , ... use Java to build commercial application servers instead of Erlang? From technical perspective such decision doesn't make any sense to me for following reasons: _Java is not a fault tolerant. _Java performance is nowhere near Erlang. _Concurrent programming in Java is a pain. _J2ee Technology introduced as add on to java to make communication cross servers possible (i.e web services XML SCHEMA, WSDL) is unreasonably and grotesquely complicated. (This complication is dictated by the technology and not by the problem domain) _Java is not distributed language (No asynch communication is possible without JMS, also RMI stub solution is more complicated than it should be). and many more reasons I can list here. Thanks in advance Shahrdad -- Software Architect & Computer Scientist _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -- http://www.linkedin.com/in/torbenhoffmann _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From sunwood360@REDACTED Mon Mar 12 02:27:41 2012 From: sunwood360@REDACTED (envelopes envelopes) Date: Sun, 11 Mar 2012 18:27:41 -0700 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> Message-ID: scala has the potential to successor java and also integrate all legacy java applications. Though many of its features are copied from Erlang. On Mar 11, 2012 5:01 PM, "Matthew Evans" wrote: > Pretty much this. I've recently been introduced to Java, and > unfortunately what most of the pointy-haired managers don't realize is > there is a huge difference between Java the language (which isn't that > bad), and Java the application environment (which is complex and terrible). > Once you get Java+Spring+Hibernate+20 other frameworks and libraries you > are left with a VERY hard to maintain and complex application. In the end > it often only has a small amount of real Java - the rest is pages of > impossible to maintain XML. > > See this blog: > http://www.jillesvangurp.com/2009/10/07/server-side-development-sucks/ > > The other problem is one of control. Even if the managers, and the rest of > a development team don't know Java, it looks very familiar to C++ and the C > family of languages. If you propose Erlang, they see that the language > looks very different, and that scares people. Suddenly they are not in > control anymore, feel less valuable to the organization; it makes them feel > threatened. > > We as a community can help to remove many of the negative perceptions > people have about Erlang. It's perceived to be slow, although in many cases > that's caused by lousy code from newbies (JIT would be nice though ;-). > It's also hard as a newbie to write good code since the concepts are very > different. You can knock up a simple Erlang application quite quickly, but > it might not perform that well. What you end up with are newbies producing > poor code, getting frustrated and giving up. I've read many stories on > Reddit and similar places of companies starting an Erlang project, getting > bad results and throwing it out. It give a very bad impression on the > language. More often than not someone will come along later, refactor the > code, and get stellar performance - but by then it's too late. > > I think tidying up some modules, and getting the word out there on how to > write proper Erlang apps would help a lot. Things like E2 > http://e2project.org/ are a good start along this path. Projects like > rebar make app. packaging simpler. Basically we need > more consistency across the board. Also having more languages running on > BEAM could make the take-up easier (as well as Erjang on the JVM). > > Finally I would say this. As Torben mentioned, no one gets fired for > choosing Java. More to the point, no one gets fired for choosing Oracle. I > think Ericsson is doing a more than stellar job of maintaining Erlang. But > is this also a problem? Sometimes I wonder if it would be better if Erlang > was "owned" by Ericsson, Klarna, Basho, Trifork, Erlang Solutions, > Springsource/VMware (RabbitMQ). Basically have it owned by a consortium of > companies. Would that make managers more comfortable taking on Erlang? > > I could go on, but that's my 2 cents for now. > > Matt > > ------------------------------ > Date: Sun, 11 Mar 2012 22:04:33 +0100 > From: torben.lehoff@REDACTED > To: shahrdad1@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Erlang is the best choice for building > commercial application servers > > Nobody has been fired for choosing Java. > > See Mike Williams slides from the London Erlang Factory 2011: > http://www.erlang-factory.com/conference/London2011/speakers/MikeWilliams > > Digging a bit deeper it comes down to risk management and most big > companies has a strong dislike for anything new and different since that > reeks risk to them. > Take a look at all the start-ups in the US that is using Erlang and asking > for Erlangers. There Erlang has been chosen since it is the right fit for > the problem at hand. > > For a big company with a lot of legacy Java code and people trained in > Java it is far from obvious that a switch to Erlang will be the right > choice. I would actually get a bit nervous if management accepted such a > shift without a thorough investigation and even if that investigation gave > a go-ahead to do a shift I would be nervous since such a fundamental break > with the past only happens when a company is staring into the abyss of a > pending bankruptcy!! > > I was one of the two guys behind a product made in Erlang in Motorola. One > of the main reasons for getting the go-ahead to that project was that we > were building a new product and did not have to throw anything out. > Eventually we shipped the product as a beta to a single customer, but the > reluctance to bet on something as strange as Erlang for a "real" contract > is still around and I will actually bet on the final product being written > from scratch in C or Java since that is what the managers and old school > architects feel most at ease with! > > The history of technology is full of this kind of stories... and one day > some new technology will come along and overturn whatever kingdom Erlang > might have build. The circle of life continues. > > Cheers, > Torben > > On 11/3/12 18:09 , Shahrdad Shadab wrote: > > When I was learning Erlang and understanding its capabilities I really > cannot find a satisfactory answer to the question that > why in North America companies like former BEA, former Sun, Oracle , ... > use Java to build commercial application servers instead of Erlang? > From technical perspective such decision doesn't make any sense to me for > following reasons: > > _Java is not a fault tolerant. > _Java performance is nowhere near Erlang. > _Concurrent programming in Java is a pain. > _J2ee Technology introduced as add on to java to make communication cross > servers possible (i.e web services XML SCHEMA, WSDL) is unreasonably and > grotesquely complicated. (This complication is dictated by the technology > and not by the problem domain) > _Java is not distributed language (No asynch communication is possible > without JMS, also RMI stub solution is more complicated than it should be). > > and many more reasons I can list here. > > Thanks in advance > Shahrdad > -- > Software Architect & Computer Scientist > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > -- http://www.linkedin.com/in/torbenhoffmann > > > _______________________________________________ erlang-questions mailing > list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jared.nance@REDACTED Mon Mar 12 02:34:04 2012 From: jared.nance@REDACTED (Jared Kofron) Date: Sun, 11 Mar 2012 18:34:04 -0700 Subject: [erlang-questions] Erlang meets physics Message-ID: Hi All, I've been using Erlang at work for a few years now, and I thought I'd throw my experience out there, as my application is a little different than what you usually see on the list - I am a graduate student at the Center for Nuclear Physics and Astrophysics at the University of Washington, and use Erlang extensively in my work. In my experience, something that Erlang is really great at but doesn't receive much attention for these days is managing and interacting with hardware. In any physics experiment of even modest sizes, you wind up having to keep track of the state of various pieces of equipment, often modify that state, and constantly interrogate particular values. For example, we might want to change the current in a magnetic trap, turn that trap off altogether, or simply read back the voltage drop across our superconducting magnet. So far, I have deployed Erlang in this zone for two separate experiments (SNO+, a large particle physics experiment in Canada) and Project 8 (a small nuclear physics experiment here in Seattle). Both times have been great successes, and I have found the reception of Erlang in this market to be great. In general, what I have done is wrap a hardware management layer with some kind of outside world interface. For SNO+, we used Webmachine and RESTful control, and for Project 8 we actually conduct all communication by using CouchDB as a message passing interface. Physicists are suspicious creatures, but once you demonstrate the feature set that you get for practically free with OTP, they see the advantage pretty quickly. On top of that, the development cycle for sophisticated applications can be greatly reduced - more than once it made my group float to the top in terms of meeting goals. In short, as far as I am concerned, Erlang has found a new niche in the world of Physics, and I intend to spread the word as much as I can! Jared Kofron From daniel.goertzen@REDACTED Mon Mar 12 03:44:29 2012 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Sun, 11 Mar 2012 21:44:29 -0500 Subject: [erlang-questions] enif_alloc_resource and enif_release_resource Message-ID: Am I allowed to call enif_release_resource() immediately after calling enif_alloc_resource(). The documentation suggests that I should be calling enif_make_resource() in between, but I don't want to do that. Furthermore, on a resource that I just allocated with enif_alloc_resource, is there a way to release it without invoking the destructor? I am working on a C++11 wrapper for resources. In C++, if construction of a dynamically allocated object throws an exception, the memory gets deallocated without invoking the destructor. I'm trying to reproduce that in a nif resource. Thanks, Dan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From shahrdad1@REDACTED Mon Mar 12 04:00:45 2012 From: shahrdad1@REDACTED (Shahrdad Shadab) Date: Sun, 11 Mar 2012 23:00:45 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> Message-ID: On Sun, Mar 11, 2012 at 9:27 PM, envelopes envelopes wrote: > scala has the potential to successor java and also integrate all legacy > java applications. Though many of its features are copied from Erlang. > On Mar 11, 2012 5:01 PM, "Matthew Evans" wrote: > >> Pretty much this. I've recently been introduced to Java, and >> unfortunately what most of the pointy-haired managers don't realize is >> there is a huge difference between Java the language (which isn't that >> bad), and Java the application environment (which is complex and terrible). >> Once you get Java+Spring+Hibernate+20 other frameworks and libraries you >> are left with a VERY hard to maintain and complex application. In the end >> it often only has a small amount of real Java - the rest is pages of >> impossible to maintain XML. >> >> See this blog: >> http://www.jillesvangurp.com/2009/10/07/server-side-development-sucks/ >> >> The other problem is one of control. Even if the managers, and the rest >> of a development team don't know Java, it looks very familiar to C++ and >> the C family of languages. If you propose Erlang, they see that the >> language looks very different, and that scares people. Suddenly they are >> not in control anymore, feel less valuable to the organization; it makes >> them feel threatened. >> >> We as a community can help to remove many of the negative perceptions >> people have about Erlang. It's perceived to be slow, although in many cases >> that's caused by lousy code from newbies (JIT would be nice though ;-). >> It's also hard as a newbie to write good code since the concepts are very >> different. You can knock up a simple Erlang application quite quickly, but >> it might not perform that well. What you end up with are newbies producing >> poor code, getting frustrated and giving up. I've read many stories on >> Reddit and similar places of companies starting an Erlang project, getting >> bad results and throwing it out. It give a very bad impression on the >> language. More often than not someone will come along later, refactor the >> code, and get stellar performance - but by then it's too late. >> >> I think tidying up some modules, and getting the word out there on how to >> write proper Erlang apps would help a lot. Things like E2 >> http://e2project.org/ are a good start along this path. Projects like >> rebar make app. packaging simpler. Basically we need >> more consistency across the board. Also having more languages running on >> BEAM could make the take-up easier (as well as Erjang on the JVM). >> >> Finally I would say this. As Torben mentioned, no one gets fired for >> choosing Java. More to the point, no one gets fired for choosing Oracle. I >> think Ericsson is doing a more than stellar job of maintaining Erlang. But >> is this also a problem? Sometimes I wonder if it would be better if Erlang >> was "owned" by Ericsson, Klarna, Basho, Trifork, Erlang Solutions, >> Springsource/VMware (RabbitMQ). Basically have it owned by a consortium of >> companies. Would that make managers more comfortable taking on Erlang? >> >> I could go on, but that's my 2 cents for now. >> >> Matt >> >> ------------------------------ >> Date: Sun, 11 Mar 2012 22:04:33 +0100 >> From: torben.lehoff@REDACTED >> To: shahrdad1@REDACTED >> CC: erlang-questions@REDACTED >> Subject: Re: [erlang-questions] Erlang is the best choice for building >> commercial application servers >> >> Nobody has been fired for choosing Java. >> >> See Mike Williams slides from the London Erlang Factory 2011: >> http://www.erlang-factory.com/conference/London2011/speakers/MikeWilliams >> >> Digging a bit deeper it comes down to risk management and most big >> companies has a strong dislike for anything new and different since that >> reeks risk to them. >> Take a look at all the start-ups in the US that is using Erlang and >> asking for Erlangers. There Erlang has been chosen since it is the right >> fit for the problem at hand. >> >> For a big company with a lot of legacy Java code and people trained in >> Java it is far from obvious that a switch to Erlang will be the right >> choice. I would actually get a bit nervous if management accepted such a >> shift without a thorough investigation and even if that investigation gave >> a go-ahead to do a shift I would be nervous since such a fundamental break >> with the past only happens when a company is staring into the abyss of a >> pending bankruptcy!! >> >> I was one of the two guys behind a product made in Erlang in Motorola. >> One of the main reasons for getting the go-ahead to that project was that >> we were building a new product and did not have to throw anything out. >> Eventually we shipped the product as a beta to a single customer, but the >> reluctance to bet on something as strange as Erlang for a "real" contract >> is still around and I will actually bet on the final product being written >> from scratch in C or Java since that is what the managers and old school >> architects feel most at ease with! >> >> The history of technology is full of this kind of stories... and one day >> some new technology will come along and overturn whatever kingdom Erlang >> might have build. The circle of life continues. >> >> Cheers, >> Torben >> >> On 11/3/12 18:09 , Shahrdad Shadab wrote: >> >> When I was learning Erlang and understanding its capabilities I really >> cannot find a satisfactory answer to the question that >> why in North America companies like former BEA, former Sun, Oracle , ... >> use Java to build commercial application servers instead of Erlang? >> From technical perspective such decision doesn't make any sense to me for >> following reasons: >> >> _Java is not a fault tolerant. >> _Java performance is nowhere near Erlang. >> _Concurrent programming in Java is a pain. >> _J2ee Technology introduced as add on to java to make communication cross >> servers possible (i.e web services XML SCHEMA, WSDL) is unreasonably and >> grotesquely complicated. (This complication is dictated by the technology >> and not by the problem domain) >> _Java is not distributed language (No asynch communication is possible >> without JMS, also RMI stub solution is more complicated than it should be). >> >> and many more reasons I can list here. >> >> Thanks in advance >> Shahrdad >> -- >> Software Architect & Computer Scientist >> >> >> _______________________________________________ >> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >> >> >> -- http://www.linkedin.com/in/torbenhoffmann >> >> >> _______________________________________________ erlang-questions mailing >> list erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> Thanks everybody for your valuable comments, What I got from your comments is basically comes down to the very point that the decision of picking java/j2ee over Erlang is most business driven than computer science / technology driven. The similar reason is behind creation of C# and .net which is a close copy of java and J2ee (Just because of competition between Microsoft and Sun). If I want to go a little bit deeper to the heart of the problem (at least in North America) big companies like Microsoft or Oracle kind of derailed IT from its normal path. You would never see such negative influence of money and power in other fields like applied math or physics because the result of innovation in these fields isn't directly applicable in business problems. Here we are dealing with business man's decision override right technical decisions.This is just trouble. This causes many practitioners use their mind and time to understand and maintane systems that are implemented in technologies which were not the right solution since the day one. Even business is currently paying for such bad decisions that they make in this filed by spending too much money for maintenance of the systems that are implemented with non-suitable technologies. Currently I don't see any solution to this problem unless IT's point of view respected by line of business and business decision makers don't cross their red lines and invade IT realm. Best Regards Shahrdad -- Software Architect & Computer Scientist -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Mar 12 04:05:26 2012 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 12 Mar 2012 16:05:26 +1300 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F5D1361.9060604@gmail.com> References: <4F5D1361.9060604@gmail.com> Message-ID: <722D2DCD-DE46-4115-A006-A8F6DE4B3349@cs.otago.ac.nz> I'm reminded of someone who was writing a program for a US Government contract. The contract required the program to be written in Ada. He wanted to write it in Prolog. Oh dear, what to do? He wrote a Prolog interpreter in Ada, told them the Ada program was the program they had paid for, and this other file with all the :- lines in it was an initialisation data file... I just wish I could remember his name so that I could be sure not to mention it (and not get him in retrospective trouble). Of course, if anyone else had had to maintain the program... From ok@REDACTED Mon Mar 12 04:11:38 2012 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 12 Mar 2012 16:11:38 +1300 Subject: [erlang-questions] Thoughts of someone new to Erlang In-Reply-To: References: Message-ID: <952406B8-D4EA-404D-9D2E-8ED9E92D8661@cs.otago.ac.nz> On 12/03/2012, at 11:45 AM, Tim McNamara wrote: > ## No hash map implementation (?) > > Does Erlang have a hash map implementation that works in constant > time? *Nobody* does. Python's aren't, Smalltalk's aren't, Java's aren't. However, Erlang has several dictionary-like structures, including - dict - gb_trees - ets (The process dictionary is in fact a per-process hash table, but that is no more 'constant time' than anyone else's.) > Perhaps I should just push through the culture shock and learn > to love the list! Lists have their uses, but representing possibly large dictionaries is not one of them. From mjtruog@REDACTED Mon Mar 12 04:32:53 2012 From: mjtruog@REDACTED (Michael Truog) Date: Sun, 11 Mar 2012 20:32:53 -0700 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> Message-ID: <4F5D6E65.90800@gmail.com> On 03/11/2012 06:59 PM, Robert Melton wrote: > On Sun, Mar 11, 2012 at 7:27 PM, Michael Truog wrote: >> On 03/11/2012 02:27 PM, james wrote: >>>> There are two sides to every story. A good link to some discussion of pro-GPL versus pro-BSD can be found here: >>>> The GPL license is conducive to liberating software. >>>> The BSD license is conducive to liberating people. >>> How is this relevant? I don't like the GPL. So what? >>> >>> What you said is simply wrong. What you said is: >>> >>>> This is a little misleading. You can never sell anything that contains GPL software >>> Whether BSD or GPL is better in some way (or even in every way) is not relevant at all to whether you have materially misrepresented what GPL requires, and is simply diversion. >>> >>> Which clause of the GPL justifies your statement? They are numbered, it should be straightforward for you to identify the one you think has this effect. >>> >>> I think its bad to let a clearly erroneous statement go unchallenged and fester malignantly in the archives for posterity. >>> >>> James >>> >> I can understand that this subject is a tough subject to reach agreement on, due to our own viewpoints of what open source software is, free software, freedom to reuse software, etc.. So, I don't expect us to agree on the specifics. When looking at the newest version, v3 (http://www.gnu.org/copyleft/gpl.html), the relevant numbers are: #5 and #6. You may notice the paragraph hiding at the bottom of the page with the first sentence that says "The GNU General Public License does not permit incorporating your program into proprietary programs.", which is a useful summary of the problem. Any modifications of GPL software retains the GPL license and must be distributed with the source code. This is good for making sure that modifications always feed back into the open source community. However, this is bad for any proprietary changes that give a company a competitive advantage. I don't regard my previous statements as erroneous, just simply more abstract. > You keep trying to use wordplay to make this seem far more complicated > than it is. Your exact words were "You can never sell anything that > contains GPL software" ... this is outright false, as multiple people > on this list have been kind enough to tell you, link to you the GPL > FAQ. Now, you try to relabel your position as "abstract" while your > statement was anything but. You made a clear, obviously ERRONEOUS > statement. > > I can go online, download a piece of GPL software, put a $20,000 > price-tag on it and sell it tomorrow, the only catch is -- the > customer who gets it now has all the same abilities I did, ability to > redistribute, copies of the source, etc. > > It might not be a good value, or a good investment for the buyer, but > I can absolutely sell it. Don't confuse the ABILITY TO SELL SOMETHING > with the intelligence of BUYING it. The two are NOT RELATED. See > garage sales as proof. > > Beyond that, if I made changes to the GPL software I downloaded, maybe > a customization for a specific customer. I don't have to share those > changes with anyone until I distribute a copy, which means I could > easily and directly charge for at least the first copy. > > What you said was -- erroneous. Only if you don't believe in a free market. I believe that prices adjust and value is immediately lost in what you describe. That is why I don't consider that approach profitable. It is much simpler to charge for a service that uses GPL code, since then you don't have these problems with distributing the source code with custom changes. I still believe my viewpoint is valid, because: 1) People will not pay $20,000 for a product available for free, unless you are manipulating the situation in a way I would consider contrary to the GPL license purpose (i.e., making the downloadable harder to find, download speeds slower, etc.)... that may be more of an ethical dilemma 2) Your charge for the modifications to the source code only makes money for the first modification, unless you are manipulating the situation to somehow suppress the new code changes, again contrary to the GPL license purpose So, charging money for a service that uses GPL source code, or for support related to using the GPL source code makes much more sense. I have not had the pleasure of watching someone argue for selling a product as GPL source code, as you have done, because it is uncommon to promote the idea of disposable intellectual property. I say that because the work for custom modifications is then only paid for once. That would only encourage writing the same code modifications in N ways for N clients, all implementations being different, to receive separate charges for all custom implementations. So, you would knowingly be forced to keep re-inventing the wheel just to maintain a consistent flow of profits, while ignoring any need for innovation. I see that as counter-productive and a type of commercial suicide, since the business becomes stagnant, is not able to grow its inherent value, and dies with roughly the same product after a competitor has been able to replace the functionality with a better solution. That is a sad end all due to a GPL license for a commercial product. From mfidelman@REDACTED Mon Mar 12 04:43:56 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Sun, 11 Mar 2012 23:43:56 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> Message-ID: <4F5D70FC.50004@meetinghouse.net> Shahrdad Shadab wrote: > > Thanks everybody for your valuable comments, What I got from your > comments is basically comes down to the very point > that the decision of picking java/j2ee over Erlang is most business > driven than computer science / technology driven. > > Currently I don't see any solution to this problem unless IT's point > of view respected by line of business and business decision makers > don't cross their red lines and invade IT realm. Let me preface this by noting that I'm about to commit to Erlang for a major project. But... in fairness, picking a language is NOT just a computer science or technology decision. When one is contemplating investing millions of dollars in developing software that has to be maintained over a decade or more, the decision is far more an operational one than a technical one. Picking the "best" technology is far less important than considerations such as: Will anyone be supporting this language in 10 years? What kind of tools are available? Can I hire programmers and systems administrators who are familiar with the language and run-time environment? In short, one is evaluating the ecosystem surrounding the language, far more than the language itself. Betting on IBM ("you can't lose your job for going with IBM") has proven a pretty good strategy for decades. DEC and Wang are long gone, IBM is still around. I expect the same will be true of Microsoft. Who knows with Apple - they're likely to be around, but will they be a computer vendor, or an entertainment company in 15 years? Where languages are concerned, Fortran and COBOL had LONG periods in the sun. For a long time, it looked like PL/1 and Ada would be their successors - they certainly had strong backing from both major vendors and the world's largest customer. And, while LISP certainly has its fans (and still does), C seems to have become dominant for an awfully long time (in my opinion, it's a horridly useless language, but good tooling, a lot of coders, and tremendous cross-platform capabilities seem to have won the day). For that matter, if you're writing code for the iPhone or iPad, your language choice is dictated for you. Ten years ago, if I were betting millions of dollars on a new system, Java sure would have looked a lot safer than Erlang (and probably still does). At the time (I may be off by a few years), Ericsson was all set to kill Erlang - it would have been crazy to bet any kind of major system on it. It was a positively brilliant move on Joe Armstrong's part to open source Erlang and build a larger community around it. That makes it a much safer choice today - but the existence of a robust ecosystem is relatively recent, and it's still a LOT easier to hire Java coders (or C, or PHP, or even Ruby coders) than to find experienced Erlang developers. Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From mfidelman@REDACTED Mon Mar 12 04:54:18 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Sun, 11 Mar 2012 23:54:18 -0400 Subject: [erlang-questions] [OT] Re: GPL vs. whatever [was: Erlang UUID] In-Reply-To: <4F5D6E65.90800@gmail.com> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> Message-ID: <4F5D736A.8070901@meetinghouse.net> Just to throw in a different aspect of the GPL vs. BSD discussion: IMHO, GPL is a far better license than BSD for a developer that intends to commercialize a product. The initial developer (copyright holder) always has the option to release code under a dual license - GPL, BSD, or whatever for an open source release, something more restrictive for the commercial product (potentially with proprietary extensions). With GPL, you pretty much eliminate any competition - anybody else who extends the code is faced with copyleft considerations, they CAN'T take your code, combine it with their own code, and slap a proprietary license around the assemblage. With BSD, or Apache, (or LGPL for that matter), they can. Of course, if you dual-license your code under GPL and a proprietary license, things can come back to haunt you if you want to incorporate community-generated extensions into your upstream code base. In that case the GPL and copyleft apply to you. -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From max.lapshin@REDACTED Mon Mar 12 06:22:00 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 12 Mar 2012 08:22:00 +0300 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: <4F5D6E65.90800@gmail.com> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> Message-ID: > > As I have already told: contact author if you want some cool library under > non-GPL -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Mon Mar 12 09:51:45 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Mon, 12 Mar 2012 17:51:45 +0900 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <722D2DCD-DE46-4115-A006-A8F6DE4B3349@cs.otago.ac.nz> References: <4F5D1361.9060604@gmail.com> <722D2DCD-DE46-4115-A006-A8F6DE4B3349@cs.otago.ac.nz> Message-ID: Erlang's main problem might be that simply outlining its virtues *objectively* only falls on the ears of managers as all-too-typical geek techno-zealotry. So: perhaps some strategic modesty is in order? Let me suggest some internal "guerrilla marketing" approaches along those lines. - Pitch: "Erlang mainly excels when you use it for a little orchestration; other languages are usually better for all the nitty-gritty of choreography, where the rubber really hits the road." (Yeah, you just mixed at least two metaphors. Guess what? Nobody noticed.) OK, team, it's decided: Erlang will just be this relatively thin layer of relatively substitutable orchestration logic. Very little risk there, right? Then, as other parts of the project start to run a bit late .... - Pitch: "Erlang's kind of slow and piggish for choreography, but look: with OTP and all, you can throw together *prototypes* of the choreography very quickly; later, go back and plug in choreography crafted in the optimal language for it." OK, team, let's triage: what do we do in Erlang, what do we do in the originally contemplated languages, and what do we skip (or fake) in this phase? Then, when the client loves the demo prototype, and confuses it with a 90% finished product ..." - Fever Pitch: "Oh crapski, we're running out calendar time on our schedule, and the customer thinks we're almost done, and we're *much* more short of resources than we expected to be, at this point in the project. [Gosh, that *never* happens in software, does it?] That means we'll have to write a whole lot of distinctly suboptimal code really fast. It'll have to be concise to a fault, it'll have to draw on all kinds of libraries, it'll need pretty good tool support ... hmm, I wonder if ...." I think this approach works best if the internal champion gives a strategically-modest presentation on some aspect of Erlang development about once a week, perhaps in brownbag-lunch style so that managers don't complain about resources being sucked away during the work day. At that frequency, he's keeping everybody's brain warmed up for Erlang, always gaining a little mindshare in the process. Clearly, this approach also works best if the internal champion is both an excellent Erlang programmer and very good at coaching others. But how do you get to that place, except through enthusiasm? And being careful to *curb* (outward) enthusiasm can be harder than it sounds .... -michael turner On Mon, Mar 12, 2012 at 12:05 PM, Richard O'Keefe wrote: > I'm reminded of someone who was writing a program for a US Government > contract. ?The contract required the program to be written in Ada. > He wanted to write it in Prolog. ?Oh dear, what to do? > > He wrote a Prolog interpreter in Ada, told them the Ada program was > the program they had paid for, and this other file with all the :- > lines in it was an initialisation data file... > > I just wish I could remember his name so that I could be sure not to > mention it (and not get him in retrospective trouble). > > Of course, if anyone else had had to maintain the program... > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From gleber.p@REDACTED Mon Mar 12 10:02:44 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 12 Mar 2012 10:02:44 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> <722D2DCD-DE46-4115-A006-A8F6DE4B3349@cs.otago.ac.nz> Message-ID: On Mon, Mar 12, 2012 at 09:51, Michael Turner wrote: > Erlang's main problem might be that simply outlining its virtues > *objectively* only falls on the ears of managers as all-too-typical > geek techno-zealotry. So: perhaps some strategic modesty is in order? > Let me suggest some internal "guerrilla marketing" approaches along > those lines. > > - Pitch: "Erlang mainly excels when you use it for a little > orchestration; other languages are usually better for all the > nitty-gritty of choreography, where the rubber really hits the road." > (Yeah, you just mixed at least two metaphors. Guess what? Nobody > noticed.) > > OK, team, it's decided: Erlang will just be this relatively thin layer > of relatively substitutable orchestration logic. Very little risk > there, right? > > Then, as other parts of the project start to run a bit late .... > > - Pitch: "Erlang's kind of slow and piggish for choreography, but > look: with OTP and all, you can throw together *prototypes* of the > choreography very quickly; later, go back and plug in choreography > crafted in the optimal language for it." > > OK, team, let's triage: what do we do in Erlang, what do we do in the > originally contemplated languages, and what do we skip (or fake) in > this phase? > > Then, when the client loves the demo prototype, and confuses it with a > 90% finished product ..." > > - Fever Pitch: "Oh crapski, we're running out calendar time on our > schedule, and the customer thinks we're almost done, and we're *much* > more short of resources than we expected to be, at this point in the > project. [Gosh, that *never* happens in software, does it?] That means > we'll have to write a whole lot of distinctly suboptimal code really > fast. It'll have to be concise to a fault, it'll have to draw on all > kinds of libraries, it'll need pretty good tool support ... hmm, I > wonder if ...." > > I think this approach works best if the internal champion gives a > strategically-modest presentation on some aspect of Erlang development > about once a week, perhaps in brownbag-lunch style so that managers > don't complain about resources being sucked away during the work day. > At that frequency, he's keeping everybody's brain warmed up for > Erlang, always gaining a little mindshare in the process. Clearly, > this approach also works best if the internal champion is both an > excellent Erlang programmer and very good at coaching others. But how > do you get to that place, except through enthusiasm? And being careful > to *curb* (outward) enthusiasm can be harder than it sounds .... > > -michael turner On the afterparty of latest Erlang Factory Lite in Krakow one of participants told a story, where once after successful use of Erlang in a project it was presented to the management as a "thin library for C" so they will agree to use it :D From watson.timothy@REDACTED Mon Mar 12 12:02:51 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 12 Mar 2012 11:02:51 +0000 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> Message-ID: <7D4F7621-53E1-47B7-A635-A18CE16F17DC@gmail.com> On 12 Mar 2012, at 03:00, Shahrdad Shadab wrote: > > Thanks everybody for your valuable comments, What I got from your comments is basically comes down to the very point > that the decision of picking java/j2ee over Erlang is most business driven than computer science / technology driven. > The similar reason is behind creation of C# and .net which is a close copy of java and J2ee (Just because of competition between Microsoft and Sun). > If I want to go a little bit deeper to the heart of the problem (at least in North America) big companies like Microsoft or Oracle kind of > derailed IT from its normal path. You would never see such negative influence of money and power in other fields like applied math or physics because the result of innovation in these fields isn't directly applicable in business problems. Here we are dealing with business man's decision override right technical decisions.This is just trouble. This causes many practitioners use their mind and time to understand and maintane systems that are implemented in technologies which were not the right solution since the day one. > Even business is currently paying for such bad decisions that they make in this filed by spending too much money for maintenance of the systems that are implemented with non-suitable technologies. > Currently I don't see any solution to this problem unless IT's point of view respected by line of business and business decision makers don't cross their red lines and invade IT realm. > There have been some interesting perspectives here and I have a few observations to share. Firstly, there are plenty of areas where computing science isn't constrained to the 'big corporate platforms' quite so much, for example in bioinformatics and other areas of science, niche areas (like telecoms, life critical systems, etc) and these are to name but a few. If you think that money isn't an influence in those other human endeavours though - the sciences are really no exception to this - you are quite mistaken. Even academics have to earn a living and competition over grants, certainly in the UK, is not for the faint hearted from what I hear. My second observation comes from spending considerable effort helping a rather *large* IT unit (whose technical staff runs into tens of thousands) with solutions to varying challenges that rally under the banner of 'governance', many of which are about processes and people as well as technology. One thing is very clear to me from this experience. In the modern business technical landscape, Dijkstra's call for simplicity and elegance is pretty much lost in a sea of voices clamouring for cost/waste reduction, increased throughput, reduced turn around times and so on. Especially when financial hard times hit, the order of the day is to do 'more' with fewer people, less money, and in less time. And naturally nobody wants the quality to go down either. Frankly in the midst of that kind of storm, picking the right tool for the job has to account for a lot more than just the requisite technical merits of one platform versus another. The comments that have been made here about risk are absolutely spot on, and in practise this 'risk aversion' often has little or nothing to do with the fact that some unknown thing like Erlang is 'unusual' or 'unfamiliar'. The risk is far more practical than that, and it boils down to the exact questions you ask yourself in an architect or solution designer's role every time you have this kind of technology choice to make: 1. will Technology-X interface with any existing tech we already have on the ground? 2. is Technology-X sufficiently battle proven in the wild and with notable companies/organisations? 3. will I be able to hire developers to continue working on Technology-X if Joe and Rob (who are so keen to use it) decide to quit half way through the project? 4. if I do have to hire new developers, will they cost me twice as much because Technology-X represents a niche skill set? 5. will I ever be able to find an affordable second and/or third line support team for this? It can be harder to do things at scale with java, distribution doesn't come so easily and getting concurrency right requires skill, experience and good analysis and measurement tools. However for most of points 1..5, java scores very highly. Erlang does great on 1 and 2. Being able to find developers is getting better, but it's still incomparably more difficult to find a good Erlang developer who is available for hire than it is to get one with Java and/or .NET experience. True, a really good developer will be able to pick up almost any language, and will even be able to adapt to new and unfamiliar programming paradigms. However 'really good' developers who can pick things up like that, actually costs a lot of money, and it's not particularly easy to find ready made teams of them that you can outsource to. An IT department with tens of thousands of people that supports a business of hundreds of thousands, simply *has* to outsource some of its engineering work. And as for finding an affordable ASG solution, just forget it. Go and compare the cost of setting up a second or third line support contract for some bog standard Java web application or Oracle suite or Sharepoint application or whatever. There's no comparison. Despite the fact that in Erlang, doing those things that are 'hard work' in java is a lot easier (and more pleasant for the developer), it is not impossibly hard to do these things in java either. I have implemented java server applications that handle a very large amount of throughput, have to deal with issues of redundancy and failover correctly when problems occur - these were 'big' projects and it was hard work, especially to test. I am also able to write concurrent programs in java (and .NET for that matter) and whilst it is not my preference to do this, it is very much possible. I am not an 'average' programmer, but I'm not a superstar or a genius either. Ironically, Erlang was developed to make it easier for 'average' programmers to build safe and correct programs, yet because it is perceived as being a niche skill, this (simplicity making it easy to do the right thing) is lost of most people who're at decision making level in a company with > 10k IT staff. This isn't to even start to mention industry standards, which again are a contributing factor. Sometimes the 'big players' have an advantage because they've ploughed millions into getting ahead of the game in some areas. Is it big conspiracy between businesses and Microsoft/Oracle do we think? No, it's much simpler than that. The large vendors get in the door and sell their story to the beleaguered IT managers, who sign on the dotted line whilst desperately hoping that with this new product/platform in place, they might at last get their weekends back. Now you know what's influencing those decisions - it's absolutely not about business decision makers versus IT. It is IT who makes those decisions, and IT does tend to choose the popular, well known and 'safe' platforms and tool sets for these reasons (and many others besides). These are not bad decisions at all. An IT department does not exist in a vacuum separately from the business, and if we're talking about IT departments whose job it is to support and enable the business, doing what's *right* for the business means getting it right on the finances, accounting for business continuity and so on. So the point about all this is that Erlang might be the right tool for the job, but that doesn't necessarily equal the right thing for the business. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker@REDACTED Mon Mar 12 12:15:26 2012 From: sverker@REDACTED (Sverker Eriksson) Date: Mon, 12 Mar 2012 12:15:26 +0100 Subject: [erlang-questions] enif_alloc_resource and enif_release_resource In-Reply-To: References: Message-ID: <4F5DDACE.4040506@erix.ericsson.se> Daniel Goertzen wrote: > Am I allowed to call enif_release_resource() immediately after > calling enif_alloc_resource(). The documentation suggests that I should be > calling enif_make_resource() in between, but I don't want to do that. > > Calling enif_release_resource() immediately after enif_alloc_resource() is fine and will destruct the resource. This is very common in error conditions. IF you want to create a "resource-term" then you must do that by calling enif_make_resource() _before_ enif_release_resource(). > Furthermore, on a resource that I just allocated with enif_alloc_resource, > is there a way to release it without invoking the destructor? > > No. The destructor will always be called if the resource type has one. You can of course have your own "non-initialized" state that makes your destructor do nothing. /Sverker, Erlang/OTP Ericsson From fenix.serega@REDACTED Mon Mar 12 13:14:42 2012 From: fenix.serega@REDACTED (fenix.serega@REDACTED) Date: Mon, 12 Mar 2012 14:14:42 +0200 Subject: [erlang-questions] erlang redis client Message-ID: Hi I'm trying to use erlang redis client https://github.com/cstar/erldis/blob/master/src/erldis.erl Module (test) code is: query() -> {ok, Client} = erldis:connect("localhost", 6379), erlang:display(Client), erldis:blpop(Client, "KEY1", 600). as result: test.query(). [{error,<<"ERR timeout is not an integer or out of range">>}] What i'm doing wrong !? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Mar 12 13:34:47 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 12 Mar 2012 13:34:47 +0100 Subject: [erlang-questions] Erlang social meetings in Copenhagen ? In-Reply-To: References: Message-ID: <4F5DED67.3020408@erlang-solutions.com> On 3/11/12 5:52 PM, Thomas Elsgaard wrote: > Hello! > > Is anybody interested in doing a monthly meeting in the Copenhagen > area and play around with Erlang and drink some coke and have fun? > > It could be at my employer's office facilities in ?restaden, just next > to Fields, we have a decent coffee ;-) > > Let me know if anybody is interested, and i will try arrange something. > I'm naturally interested! Please keep me in the loop if possible :) -- Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen, DK From torben.lehoff@REDACTED Mon Mar 12 13:36:13 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Mon, 12 Mar 2012 13:36:13 +0100 Subject: [erlang-questions] Erlang social meetings in Copenhagen ? In-Reply-To: References: Message-ID: <4F5DEDBD.7080602@gmail.com> +1 Cheers, Torben On 11/3/12 17:52 , Thomas Elsgaard wrote: > Hello! > > Is anybody interested in doing a monthly meeting in the Copenhagen > area and play around with Erlang and drink some coke and have fun? > > It could be at my employer's office facilities in ?restaden, just next > to Fields, we have a decent coffee ;-) > > Let me know if anybody is interested, and i will try arrange something. > > Best Regards > > Thomas Elsgaard > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- http://www.linkedin.com/in/torbenhoffmann From spawn.think@REDACTED Mon Mar 12 14:03:59 2012 From: spawn.think@REDACTED (Ahmed Omar) Date: Mon, 12 Mar 2012 14:03:59 +0100 Subject: [erlang-questions] erlang redis client In-Reply-To: References: Message-ID: One note, I guess u should change the call to erldis:blpop(Client, ["KEY1"], 600). Not sure if that will fix your problem On Mon, Mar 12, 2012 at 1:14 PM, wrote: > Hi > > I'm trying to use erlang redis client > https://github.com/cstar/erldis/blob/master/src/erldis.erl > > Module (test) code is: > > query() -> > {ok, Client} = erldis:connect("localhost", 6379), > erlang:display(Client), > > erldis:blpop(Client, "KEY1", 600). > > as result: > > test.query(). > > > [{error,<<"ERR timeout is not an integer or out of range">>}] > > What i'm doing wrong !? > > Thanks > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From psvensson@REDACTED Mon Mar 12 14:08:29 2012 From: psvensson@REDACTED (Peter Svensson) Date: Mon, 12 Mar 2012 14:08:29 +0100 Subject: [erlang-questions] Dyncon 2012 in Stockholm Message-ID: Hi Everyone, I would just like to invite anyone nearby to the second Dyncon ever the 21-22/4 2012, a conference focusing on dynamic languages only. This year we welcome back Robert Virding from Elrang Solutions for both talk and workshops! http://swdc.se/dyncon2012/ We also have speakers on MongoDB, Smalltalk, Node.js, JavaScript build systems and more! Cheers, PS -- Escape from /dev/null; http://swdc.se/devnull Code your way with your pals through the computer of an alien spaceship, drink beer and win prizes. http://se.linkedin.com/in/petersvensson (LinkedIn) https://plus.google.com/u/0/109826099655556298824 (Google+) http://twitter.com/psvensson (@psvensson) -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattevans123@REDACTED Mon Mar 12 14:30:37 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Mon, 12 Mar 2012 09:30:37 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F5D70FC.50004@meetinghouse.net> References: , <4F5D1361.9060604@gmail.com> , , , <4F5D70FC.50004@meetinghouse.net> Message-ID: Yes, this is very true. But as I mentioned before, most large Java applications aren't "just Java". They include Java SE or EE, Spring or some similar framework, Hibernate or some other database access library, a DB itself, Dozer to do object to object mapping, GWT to do any kind of GUI and so on. Then once this is done, you now need Maven to bring it all together and Eclipse because it's so darned complex there is no way that Vim or Emacs will work. On top of this all of this stuff is glued together by pages of XML. You have a massive learning curve for all of this, worse still any 2 companies are probably using a different combination of said libraries/frameworks. My own company is using some of the above, and even the expert on the topic struggles from time to time. Although Java will certainly be around for 10 years, there is a good chance that on any project with a lengthy shelf-life some of these libraries would've died or been replaced with something more "shiny". Finding developers in 10 years who know Spring version X, and Hibernate version Y will also be hard. So you certainly have the same issues with Java apps that you do with Erlang. I would actually say Erlang is much better here since, although there are lots of 3rd party libraries out there, 99% of what most people need is all included out of the box. Effectively with Erlang the learning curve is a one off effort. Matt > Date: Sun, 11 Mar 2012 23:43:56 -0400 > From: mfidelman@REDACTED > To: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Erlang is the best choice for building commercial application servers > > Shahrdad Shadab wrote: > > > > Thanks everybody for your valuable comments, What I got from your > > comments is basically comes down to the very point > > that the decision of picking java/j2ee over Erlang is most business > > driven than computer science / technology driven. > > > > > Currently I don't see any solution to this problem unless IT's point > > of view respected by line of business and business decision makers > > don't cross their red lines and invade IT realm. > > Let me preface this by noting that I'm about to commit to Erlang for a > major project. > > But... in fairness, picking a language is NOT just a computer science or > technology decision. When one is contemplating investing millions of > dollars in developing software that has to be maintained over a decade > or more, the decision is far more an operational one than a technical > one. Picking the "best" technology is far less important than > considerations such as: Will anyone be supporting this language in 10 > years? What kind of tools are available? Can I hire programmers and > systems administrators who are familiar with the language and run-time > environment? In short, one is evaluating the ecosystem surrounding the > language, far more than the language itself. > > Betting on IBM ("you can't lose your job for going with IBM") has proven > a pretty good strategy for decades. DEC and Wang are long gone, IBM is > still around. I expect the same will be true of Microsoft. Who knows > with Apple - they're likely to be around, but will they be a computer > vendor, or an entertainment company in 15 years? > > Where languages are concerned, Fortran and COBOL had LONG periods in the > sun. For a long time, it looked like PL/1 and Ada would be their > successors - they certainly had strong backing from both major vendors > and the world's largest customer. And, while LISP certainly has its > fans (and still does), C seems to have become dominant for an awfully > long time (in my opinion, it's a horridly useless language, but good > tooling, a lot of coders, and tremendous cross-platform capabilities > seem to have won the day). For that matter, if you're writing code for > the iPhone or iPad, your language choice is dictated for you. > > Ten years ago, if I were betting millions of dollars on a new system, > Java sure would have looked a lot safer than Erlang (and probably still > does). At the time (I may be off by a few years), Ericsson was all set > to kill Erlang - it would have been crazy to bet any kind of major > system on it. It was a positively brilliant move on Joe Armstrong's > part to open source Erlang and build a larger community around it. That > makes it a much safer choice today - but the existence of a robust > ecosystem is relatively recent, and it's still a LOT easier to hire Java > coders (or C, or PHP, or even Ruby coders) than to find experienced > Erlang developers. > > Miles Fidelman > > > > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon Mar 12 15:40:30 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 12 Mar 2012 14:40:30 +0000 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5D70FC.50004@meetinghouse.net> Message-ID: On 12 March 2012 13:30, Matthew Evans wrote: > Yes, this is very true. > > But as I mentioned before, most large Java applications aren't "just Java". > They include Java SE or EE, Spring or some similar framework, Hibernate or > some other database access library, a DB itself, Dozer to do object to > object mapping, GWT to do any kind of GUI and so on. Then once this is done, > you now need Maven to bring it all together and Eclipse because it's so > darned complex there is no way that Vim or Emacs will work. On top of this > all of this stuff is glued together by pages of XML. You have a massive > learning curve for all of this, worse still any 2 companies are probably > using a different combination of said libraries/frameworks. My own company > is using some of the above, and even the expert on the topic struggles from > time to time. > > Although Java will certainly be around for 10 years, there is a good chance > that on any project with a lengthy shelf-life some of these libraries > would've died or been replaced with something more "shiny". Finding > developers in 10 years who know Spring version X, and Hibernate version Y > will also be hard. So you certainly have the same issues with Java apps that > you do with Erlang. I would actually say Erlang is much better here since, > although there are lots of 3rd party libraries out there, 99% of what most > people need is all included out of the box. Effectively with Erlang the > learning curve is a one off effort. > Personally I do agree with this, but sadly it don't find it to be a particularly prevalent view in the wider world. From fenix.serega@REDACTED Mon Mar 12 16:11:31 2012 From: fenix.serega@REDACTED (fenix.serega@REDACTED) Date: Mon, 12 Mar 2012 17:11:31 +0200 Subject: [erlang-questions] erlang redis client In-Reply-To: References: Message-ID: Solved. My fault. Source of problem - old version, from forked project. erlidis_client.erl where: server_timeout(V) when is_number(V) -> V / 1000. and it makes server_timeout - float, that is why [{error,<<"ERR timeout is not an integer or out of range">>}] just go to https://github.com/cstar/erldis for latest correct version. Thank's to Ahmed Omar, for help. 2012/3/12 > Hi > > I'm trying to use erlang redis client > https://github.com/cstar/erldis/blob/master/src/erldis.erl > > Module (test) code is: > > query() -> > {ok, Client} = erldis:connect("localhost", 6379), > erlang:display(Client), > > erldis:blpop(Client, "KEY1", 600). > > as result: > > test.query(). > > > [{error,<<"ERR timeout is not an integer or out of range">>}] > > What i'm doing wrong !? > > Thanks > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ebegumisa@REDACTED Mon Mar 12 16:38:35 2012 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Tue, 13 Mar 2012 02:38:35 +1100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> Message-ID: My impression is that it boils down to marketing. .Net gained acceptance in the business world fairly quickly by *both* developers and managers. If Erricson were in the business of selling software (which they are not), and put together a group of spin doctors with a fat budget, things would be different. I remember a debate I had with a Microsoftie who was convinced VMs were evil (and thus hated Java and all interpreted languages). A couple of year later when .Net came out and I "reminded" him that he's supposed to hate it, he rebuffed by claiming "No, you see .Net is 'managed' code. Just like in a business you don't want to have unmanaged employees running around doing as they please -- that's dangerous. You want to only have 'managed' employees!" This from an otherwise excellent programmer. Developers are not above falling for spin, much as we like to think we are. - Edmond - On Mon, 12 Mar 2012 14:00:45 +1100, Shahrdad Shadab wrote: > On Sun, Mar 11, 2012 at 9:27 PM, envelopes envelopes > wrote: > >> scala has the potential to successor java and also integrate all legacy >> java applications. Though many of its features are copied from Erlang. >> On Mar 11, 2012 5:01 PM, "Matthew Evans" >> wrote: >> >>> Pretty much this. I've recently been introduced to Java, and >>> unfortunately what most of the pointy-haired managers don't realize is >>> there is a huge difference between Java the language (which isn't that >>> bad), and Java the application environment (which is complex and >>> terrible). >>> Once you get Java+Spring+Hibernate+20 other frameworks and libraries >>> you >>> are left with a VERY hard to maintain and complex application. In the >>> end >>> it often only has a small amount of real Java - the rest is pages of >>> impossible to maintain XML. >>> >>> See this blog: >>> http://www.jillesvangurp.com/2009/10/07/server-side-development-sucks/ >>> >>> The other problem is one of control. Even if the managers, and the rest >>> of a development team don't know Java, it looks very familiar to C++ >>> and >>> the C family of languages. If you propose Erlang, they see that the >>> language looks very different, and that scares people. Suddenly they >>> are >>> not in control anymore, feel less valuable to the organization; it >>> makes >>> them feel threatened. >>> >>> We as a community can help to remove many of the negative perceptions >>> people have about Erlang. It's perceived to be slow, although in many >>> cases >>> that's caused by lousy code from newbies (JIT would be nice though ;-). >>> It's also hard as a newbie to write good code since the concepts are >>> very >>> different. You can knock up a simple Erlang application quite quickly, >>> but >>> it might not perform that well. What you end up with are newbies >>> producing >>> poor code, getting frustrated and giving up. I've read many stories on >>> Reddit and similar places of companies starting an Erlang project, >>> getting >>> bad results and throwing it out. It give a very bad impression on the >>> language. More often than not someone will come along later, refactor >>> the >>> code, and get stellar performance - but by then it's too late. >>> >>> I think tidying up some modules, and getting the word out there on how >>> to >>> write proper Erlang apps would help a lot. Things like E2 >>> http://e2project.org/ are a good start along this path. Projects like >>> rebar make app. packaging simpler. Basically we need >>> more consistency across the board. Also having more languages running >>> on >>> BEAM could make the take-up easier (as well as Erjang on the JVM). >>> >>> Finally I would say this. As Torben mentioned, no one gets fired for >>> choosing Java. More to the point, no one gets fired for choosing >>> Oracle. I >>> think Ericsson is doing a more than stellar job of maintaining Erlang. >>> But >>> is this also a problem? Sometimes I wonder if it would be better if >>> Erlang >>> was "owned" by Ericsson, Klarna, Basho, Trifork, Erlang Solutions, >>> Springsource/VMware (RabbitMQ). Basically have it owned by a >>> consortium of >>> companies. Would that make managers more comfortable taking on Erlang? >>> >>> I could go on, but that's my 2 cents for now. >>> >>> Matt >>> >>> ------------------------------ >>> Date: Sun, 11 Mar 2012 22:04:33 +0100 >>> From: torben.lehoff@REDACTED >>> To: shahrdad1@REDACTED >>> CC: erlang-questions@REDACTED >>> Subject: Re: [erlang-questions] Erlang is the best choice for building >>> commercial application servers >>> >>> Nobody has been fired for choosing Java. >>> >>> See Mike Williams slides from the London Erlang Factory 2011: >>> http://www.erlang-factory.com/conference/London2011/speakers/MikeWilliams >>> >>> Digging a bit deeper it comes down to risk management and most big >>> companies has a strong dislike for anything new and different since >>> that >>> reeks risk to them. >>> Take a look at all the start-ups in the US that is using Erlang and >>> asking for Erlangers. There Erlang has been chosen since it is the >>> right >>> fit for the problem at hand. >>> >>> For a big company with a lot of legacy Java code and people trained in >>> Java it is far from obvious that a switch to Erlang will be the right >>> choice. I would actually get a bit nervous if management accepted such >>> a >>> shift without a thorough investigation and even if that investigation >>> gave >>> a go-ahead to do a shift I would be nervous since such a fundamental >>> break >>> with the past only happens when a company is staring into the abyss of >>> a >>> pending bankruptcy!! >>> >>> I was one of the two guys behind a product made in Erlang in Motorola. >>> One of the main reasons for getting the go-ahead to that project was >>> that >>> we were building a new product and did not have to throw anything out. >>> Eventually we shipped the product as a beta to a single customer, but >>> the >>> reluctance to bet on something as strange as Erlang for a "real" >>> contract >>> is still around and I will actually bet on the final product being >>> written >>> from scratch in C or Java since that is what the managers and old >>> school >>> architects feel most at ease with! >>> >>> The history of technology is full of this kind of stories... and one >>> day >>> some new technology will come along and overturn whatever kingdom >>> Erlang >>> might have build. The circle of life continues. >>> >>> Cheers, >>> Torben >>> >>> On 11/3/12 18:09 , Shahrdad Shadab wrote: >>> >>> When I was learning Erlang and understanding its capabilities I really >>> cannot find a satisfactory answer to the question that >>> why in North America companies like former BEA, former Sun, Oracle , >>> ... >>> use Java to build commercial application servers instead of Erlang? >>> From technical perspective such decision doesn't make any sense to me >>> for >>> following reasons: >>> >>> _Java is not a fault tolerant. >>> _Java performance is nowhere near Erlang. >>> _Concurrent programming in Java is a pain. >>> _J2ee Technology introduced as add on to java to make communication >>> cross >>> servers possible (i.e web services XML SCHEMA, WSDL) is unreasonably >>> and >>> grotesquely complicated. (This complication is dictated by the >>> technology >>> and not by the problem domain) >>> _Java is not distributed language (No asynch communication is possible >>> without JMS, also RMI stub solution is more complicated than it should >>> be). >>> >>> and many more reasons I can list here. >>> >>> Thanks in advance >>> Shahrdad >>> -- >>> Software Architect & Computer Scientist >>> >>> >>> _______________________________________________ >>> erlang-questions mailing >>> listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> -- http://www.linkedin.com/in/torbenhoffmann >>> >>> >>> _______________________________________________ erlang-questions >>> mailing >>> list erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> > Thanks everybody for your valuable comments, What I got from your > comments > is basically comes down to the very point > that the decision of picking java/j2ee over Erlang is most business > driven > than computer science / technology driven. > The similar reason is behind creation of C# and .net which is a close > copy > of java and J2ee (Just because of competition between Microsoft and Sun). > If I want to go a little bit deeper to the heart of the problem (at least > in North America) big companies like Microsoft or Oracle kind of > derailed IT from its normal path. You would never see such negative > influence of money and power in other fields like applied math or physics > because the result of innovation in these fields isn't directly > applicable > in business problems. Here we are dealing with business man's decision > override right technical decisions.This is just trouble. This causes many > practitioners use their mind and time to understand and maintane systems > that are implemented in technologies which were not the right solution > since the day one. > Even business is currently paying for such bad decisions that they make > in > this filed by spending too much money for maintenance of the systems that > are implemented with non-suitable technologies. > Currently I don't see any solution to this problem unless IT's point of > view respected by line of business and business decision makers don't > cross > their red lines and invade IT realm. > > Best Regards > Shahrdad > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From mfidelman@REDACTED Mon Mar 12 17:10:00 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Mon, 12 Mar 2012 12:10:00 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> Message-ID: <4F5E1FD8.4060306@meetinghouse.net> Edmond Begumisa wrote: > My impression is that it boils down to marketing. > > .Net gained acceptance in the business world fairly quickly by *both* > developers and managers. > > If Erricson were in the business of selling software (which they are > not), and put together a group of spin doctors with a fat budget, > things would be different. The important part is "in the business of selling software." The critical business consideration, when selecting infrastructure, is the level of continued support you can expect. IBM is going to be around for a while, so is Microsoft. If you prefer vendor independence, you can be pretty confident that C is going to be supported by someone, so is Apache. My sense is that the Erlang/OTP ecosystem has become pretty strong in recent years - enough so that, if Ericsson bailed out, Erlang would continue. A few years back, that might not have been quite as good a gamble. (Comments?) -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From mattevans123@REDACTED Mon Mar 12 18:05:53 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Mon, 12 Mar 2012 13:05:53 -0400 Subject: [erlang-questions] Data in state records Message-ID: Hi, Many of my processes (gen_servers) need to store some kind of indexed data in the state record. Usually this involves creating a set/dict/ets table in the init/1 function, assigning it to an element in the state record and using the data on each handle_call/cast/info. My usual strategy is anything that is 100% private to my gen_server and will contain less than 50 or so records I will use sets/gb_trees or similar. In all other cases I would use an ETS table. Is there an official recommendation? Thanks Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From hukl@REDACTED Mon Mar 12 18:23:01 2012 From: hukl@REDACTED (John-Paul Bader) Date: Mon, 12 Mar 2012 18:23:01 +0100 Subject: [erlang-questions] Erlang Community Conference In-Reply-To: References: <4F54FE0F.6090700@berlin.ccc.de> Message-ID: <4F5E30F5.6080802@berlin.ccc.de> Hey again, so after a lot of discussions and even more bad puns about the conference name we have settled for ?Erlconf? I know - its not the most inspirational name ever but we will make it up to you with an awesome logo and design. If you like, you can follow @erlconf on twitter for all the updates we have. The corresponding domains are also registered but there is nothing to see yet. Right now we are still looking for a venue and once we have that we also have a date. ~ John Tim Fletcher wrote: >> Any feedback is appreciated! We will have a dedicated interface for >> tracking who would be interested to come later on. > > Sounds good. Website and/or twitter account that we can follow would > be helpful :) > > Cheers, > Tim > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From fhamilton@REDACTED Mon Mar 12 19:14:08 2012 From: fhamilton@REDACTED (Felix Hamilton) Date: Mon, 12 Mar 2012 11:14:08 -0700 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F5E1FD8.4060306@meetinghouse.net> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> Message-ID: An interesting data point that might be relevant here is that I have seen Erlang/OTP getting more traction in the 'big data' world recently. One really nice architecture seems to be to use Erlang for 3C stuff (command, control, and communications), something like 0mq to handle messaging, and Python (or something with similar user friendliness, readability, and popularity) to do user facing stuff and provide user scripting capabilities. This seems like a *really* powerful combination to me, and one that has arisen spontaneously in very disparate areas with completely unconnected developers. As a user of both Erlang and Python, there really doesnt seem to be a lot of overlap between the two, and using both buys pretty comprehensive problem space coverage cheaply and cleanly. The disco project ( http://discoproject.org/ ) is a great example most folks might be familiar with, but many others seem to be mostly closed source or 'software as a service' deals so far. ;-) Anyhow, that was a pretty interesting data point for me. You might not have to convince folks to let you write everything in Erlang if you can let all the user facing stuff use something they are more comfortable with and focus on using Erlang for 3C (where it really shines in my opinion) ... On a completely unrelated tangent, are there any Erlang modules that interact with javascript visualization libraries like protovis ( http://mbostock.github.com/protovis/ ) or d3.js ( http://mbostock.github.com/d3/ )? That might be interesting. Or are there any nice open source Erlang based visualization libraries that generate web embeddable, interactive svg graphics with all the standard charts and graphs as well as good flexibility? I havent run across anything myself, but good Erlang based data visualization tools would be a nice selling point as well. Some very interesting discussion has been going on here about interacting with the W3C DOM API directly in Erlang (mainly CSS and HTML) to control browser rendering, but no one has mentioned doing so for data visualization purposes (i.e. SVG elements) as far as I know ... /Felix On Mon, Mar 12, 2012 at 9:10 AM, Miles Fidelman wrote: > Edmond Begumisa wrote: > >> My impression is that it boils down to marketing. >> >> .Net gained acceptance in the business world fairly quickly by *both* >> developers and managers. >> >> If Erricson were in the business of selling software (which they are >> not), and put together a group of spin doctors with a fat budget, things >> would be different. >> > > The important part is "in the business of selling software." The critical > business consideration, when selecting infrastructure, is the level of > continued support you can expect. IBM is going to be around for a while, > so is Microsoft. If you prefer vendor independence, you can be pretty > confident that C is going to be supported by someone, so is Apache. > > My sense is that the Erlang/OTP ecosystem has become pretty strong in > recent years - enough so that, if Ericsson bailed out, Erlang would > continue. A few years back, that might not have been quite as good a > gamble. (Comments?) > > > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ebegumisa@REDACTED Mon Mar 12 19:30:45 2012 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Tue, 13 Mar 2012 05:30:45 +1100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F5E1FD8.4060306@meetinghouse.net> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> Message-ID: On Tue, 13 Mar 2012 03:10:00 +1100, Miles Fidelman wrote: > Edmond Begumisa wrote: >> My impression is that it boils down to marketing. >> >> .Net gained acceptance in the business world fairly quickly by *both* >> developers and managers. >> >> If Erricson were in the business of selling software (which they are >> not), and put together a group of spin doctors with a fat budget, >> things would be different. > > The important part is "in the business of selling software." The > critical business consideration, when selecting infrastructure, is the > level of continued support you can expect. IBM is going to be around > for a while, so is Microsoft. I used to believe that but not anymore. I'll give example: There are many businesses that invested heavily in the previous iteration of MS development infrastructure (COM-driven Visual Studio 6 and related tools), and then suddenly had the rug pulled from underneath them in 2002 when .Net appeared and they were expected to rewrite/migrate much of their code (I worked for such a victim, and gathered many now-worthless skills). It became clear to me that choosing development infrastructure because the vendor will "be around" guarantees you zilch continued support. Your business is at the mercy of their business decisions. Yet strangely, I watched as many businesses with years of complex COM-driven code, grumbled for a while, then eventually engaged in expensive time-consuming migrations to .Net or started new projects in .Net! "Why would a business do this?", I pondered, "this vendor has just illustrated that they can drop you without thinking twice! Why not just go to someone else entirely? And why would a programmer learn C# which may well be worthless at some random point in the near future?" The only sane answer I could come up with was successful marketing and PR. Microsoft succeeded in skillfully convincing people in this sector that re-writing their code in this new environment would allow their software to fly to the moon (managed code, reduced TCO and all that hoopla), despite having just illustrated that they could well decide to drop .Net anytime and move to something entirely different. Suddenly there were lots of .Net projects, .Net jobs and lots of .Net developers after those jobs. I concluded that marketing and PR affect this sector a lot more than people realise. People think they are making decisions based on engineering and business sense like "continued support" but there's a lot of evidence that this is not the case. Many of the "decisions" are being fed to them. > If you prefer vendor independence, you can be pretty confident that C is > going to be supported by someone, so is Apache. > Yes, I've come to the realisation that open and open-source development tools/infrastructure are the only ones that can guarantee you continuity, even if your business has to provide that continuity itself. Counter-intuitively, big commercial vendor names are actually riskier. > My sense is that the Erlang/OTP ecosystem has become pretty strong in > recent years - enough so that, if Ericsson bailed out, Erlang would > continue. A few years back, that might not have been quite as good a > gamble. (Comments?) I agree. I'm not big on "industry standards" (i.e. popularity) these days. But many businesses are, and not entirely for the reasons they think they are. Getting those on board with Erlang I think is largely a matter of playing the game the way other vendors play it -- lots of marketing, lots of PR, lots of spin all requiring lots of money. But Ericsson's in an entirely different game. - Edmond - > > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From watson.timothy@REDACTED Mon Mar 12 19:38:16 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 12 Mar 2012 18:38:16 +0000 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> Message-ID: <1C895455-47E2-440C-B7C8-DFF88531B1D8@gmail.com> On 12 Mar 2012, at 18:30, Edmond Begumisa wrote: > > >> If you prefer vendor independence, you can be pretty confident that C is going to be supported by someone, so is Apache. >> > > Yes, I've come to the realisation that open and open-source development tools/infrastructure are the only ones that can guarantee you continuity, even if your business has to provide that continuity itself. Counter-intuitively, big commercial vendor names are actually riskier. > In practise you're absolutely right about this Edmond. But in practise, not many decision makers seem realise it. >> My sense is that the Erlang/OTP ecosystem has become pretty strong in recent years - enough so that, if Ericsson bailed out, Erlang would continue. A few years back, that might not have been quite as good a gamble. (Comments?) > > I agree. I'm not big on "industry standards" (i.e. popularity) these days. But many businesses are, and not entirely for the reasons they think they are. Getting those on board with Erlang I think is largely a matter of playing the game the way other vendors play it -- lots of marketing, lots of PR, lots of spin all requiring lots of money. But Ericsson's in an entirely different game. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Mon Mar 12 20:03:48 2012 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 12 Mar 2012 12:03:48 -0700 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> Message-ID: <4F5E4894.3020505@gmail.com> On 03/12/2012 11:14 AM, Felix Hamilton wrote: > An interesting data point that might be relevant here is that I have seen Erlang/OTP getting more traction in the 'big data' world recently. One really nice architecture seems to be to use Erlang for 3C stuff (command, control, and communications), something like 0mq to handle messaging, and Python (or something with similar user friendliness, readability, and popularity) to do user facing stuff and provide user scripting capabilities. This seems like a *really* powerful combination to me, and one that has arisen spontaneously in very disparate areas with completely unconnected developers. As a user of both Erlang and Python, there really doesnt seem to be a lot of overlap between the two, and using both buys pretty comprehensive problem space coverage cheaply and cleanly. The disco project ( http://discoproject.org/ ) is a great example most folks might be familiar with, but many others seem to be mostly closed source or 'software as a service' deals so far. ;-) > > Anyhow, that was a pretty interesting data point for me. You might not have to convince folks to let you write everything in Erlang if you can let all the user facing stuff use something they are more comfortable with and focus on using Erlang for 3C (where it really shines in my opinion) ... > CloudI (http://cloudi.org) hasn't been mentioned yet, but is directly relevant since it acts as an application server and is implemented with Erlang. CloudI provides ZeroMQ integration along with Python, Ruby, Java, and C/C++ integration. The project is currently alpha and will be moving to beta soon. The project provides a nice way of wedging Erlang into a stack so that you don't necessarily have the problems previously mentioned when introducing Erlang to people. You can just take external code, convert it to using the CloudI API for major data messaging, and use the configuration to control the amount of processes/threads for scaling the service. Then, it becomes much simpler to show the benefits of Erlang, since you can replace parts of the system with Erlang, using CloudI services that take over various functions (advertised based on the service name) and improve upon the codebase without forcing anyone into a major rewrite. That reduces any development/business risk when developing with Erlang. The approach would be for people making a production system, not just for prototyping, though it can also help facilitate easier prototyping with diverse libraries that need to be used in a fault-tolerant, scalable way (with a consistent interface). - Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Mar 12 22:55:42 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 13 Mar 2012 10:55:42 +1300 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F5E1FD8.4060306@meetinghouse.net> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> Message-ID: <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> On 13/03/2012, at 5:10 AM, Miles Fidelman wrote: > The important part is "in the business of selling software." The critical business consideration, when selecting infrastructure, is the level of continued support you can expect. IBM is going to be around for a while, so is Microsoft. If you prefer vendor independence, you can be pretty confident that C is going to be supported by someone, so is Apache. "Company is going to be around for a while" is misleading. IBM is still around, but they abandoned their Pascal compiler a long time ago, long before other companies did. IBM is still around, but Lisp/370 (or Lisp/VM) isn't. IBM is still around, but IBM Prolog doesn't seem to be. IBM is still around, but they sold off their Smalltalk unit. HP is still around, but Compaq Visual Fortran for Windows is not. Microsoft is still around, but they dropped their Fortran compiler a long time ago. Microsoft is still around, but they dropped Visual Basic for Applications from the Macintosh port of Office several years ago, and while it had been pretty bad having to use VBA, it was even worse NOT being able to use it. Microsoft is still around, but they sold MS-Test to Rational who renamed it to Rational Visual Test; they got bought by IBM; I _think_ it's now SQABasic One of my colleagues is really keen on Rational Purify. It's still around, but if you look up the Windows support, it's "Intel IA-32" only, even under Windows 7. No 64-bit Windows. 64-bit Linux, yes. But if you are a Windows developer, Purify has moved away from you by standing still. From jws@REDACTED Tue Mar 13 03:00:49 2012 From: jws@REDACTED (Jeff Schultz) Date: Tue, 13 Mar 2012 13:00:49 +1100 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: <4F5D34D9.40009@gmail.com> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> Message-ID: <20120313020049.GA16224@mulga.csse.unimelb.edu.au> On Sun, Mar 11, 2012 at 04:27:21PM -0700, Michael Truog wrote: > numbers are: #5 and #6. You may notice the paragraph hiding at the > bottom of the page with the first sentence that says "The GNU General > Public License does not permit incorporating your program into > proprietary programs.", which is a useful summary of the problem. Any > modifications of GPL software retains the GPL license and must be > distributed with the source code. This is good for making sure that > modifications always feed back into the open source community. > However, this is bad for any proprietary changes that give a company a > competitive advantage. I don't regard my previous statements as > erroneous, just simply more abstract. The difference of viewpoint is simply that yours is "What rent can I collect by reselling software someone else has released with a GPL license?" The "someone else", who wrote the software in the first place, usually picked the GPL because their viewpoint was "What rent can I collect by releasing my software so that others will use and improve it?" You're partially correct in that it can be harder for someone with your viewpoint to make money, though it doesn't seem to be hurting Apple,* Google or RedHat all that much. Each has a different business model for dealing with the masses of GPL code essential to their businesses. Jeff Schultz ------------------------------------------------------ *Apple tried the embrace-extend-extinguish business model it learned from Microsoft on some GPL software, and failed, so it's working to replace as much of it as it can. It loaths GPL, but the people who put their work under it seem pretty happy with the profits from *their* business model. From michael.eugene.turner@REDACTED Tue Mar 13 04:45:04 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 13 Mar 2012 12:45:04 +0900 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: I have to concur with Richard here, at least insofar as he's saying "Big Company X is still around" can't be the overwhelming factor in language success. It is neither necessary nor sufficient. Pascal used to have enormous momentum. It was the de facto standard undergraduate language for computer science instruction -- if you had a CS degree, you knew Pascal, period. It had standards committees behind it. It had a de facto standard VM (UCSD pCode) for platform independence. It had support from big software vendors. Macpaint, the program that sold a million customers on the Mac, was prototyped in Pascal on the Apple Lisa. Then, in a few short years, almost everybody (Macintosh developers included) had pretty much switched to C. Borland keeps Pascal alive now (Delphi), but "Borland supports it" is not the magic incantation it was. How did C and the C standard library win? It wasn't because AT&T was brilliant at marketing, or because it was a huge company. In marketing terms, it was downright awful, shooting itself in the foot repeatedly. And it wasn't because C was the core programming tool for Unix and Unix was winning -- neither DOS nor the Mac were Unix-based. Unix (in some form) didn't really reach the desktop at any significant scale until OS X, I'd say. At which point, Apple was never going to become the dominant desktop player. C/stdlib won because it already had fitness for purpose in a burgeoning market. Microcomputers were basically recapitulating the technological evolution of minicomputers, in fast-forward and at price points an order of magnitude lower, where mass-market consumers could reach them. And C had been initially targeted for a minicomputer hardly as powerful as the IBM PC/AT. The flexibility of C won out over the relative safety of Pascal -- to get some things done in Pascal, you needed extensions and every Pascal vendor extended in a different, non-standard way. C could not only do more, it could run within a factor of three or so as fast as hand-tweaked assembler. This mattered at a time when microcomputer software developers would exult when the z80 went from 2MHz to 4Mhz. In software, time-to-market is *almost* everything. Early adopters of new software tended to have faster machines with more RAM, so even if the C version of what was then ordinarily written in z80 or 8086 assembler was a little bulky and slow on the standard desktop in terms of demands on the hardware, the end-user performance standard was changing so fast anyway that it hardly mattered by the time a new software product appeared on the radar of mainstream users -- many of whom were buying a computer for the first time. What's happening now is true utility computing -- the mass-market end user is finally moving toward client-server distributed computing in a big way, and the servers are increasingly multi-core and "cloudy". Many people are backing up their hard drives to the Internet -- for the first time. I'm writing this on Gmail, then I will go back to working on the chapter of a book hosted in Google Docs, and edited through a web interface. Such paradigm shifts play to Erlang's strengths. I've never been in the Java enterprise software milieu. However, everything I hear about it reminds me of the cover of The Mythical Man Month: those beasts struggling vainly to escape from the tar pit. Even if they can still escape, the real question is: how fast can they move toward a new opportunity as conditions change? -michael turner On Tue, Mar 13, 2012 at 6:55 AM, Richard O'Keefe wrote: > > On 13/03/2012, at 5:10 AM, Miles Fidelman wrote: >> The important part is "in the business of selling software." ?The critical business consideration, when selecting infrastructure, is the level of continued support you can expect. ?IBM is going to be around for a while, so is Microsoft. ?If you prefer vendor independence, you can be pretty confident that C is going to be supported by someone, so is Apache. > > "Company is going to be around for a while" is misleading. > > IBM is still around, but they abandoned their Pascal compiler a long time ago, > long before other companies did. > IBM is still around, but Lisp/370 (or Lisp/VM) isn't. > IBM is still around, but IBM Prolog doesn't seem to be. > IBM is still around, but they sold off their Smalltalk unit. > > HP is still around, but Compaq Visual Fortran for Windows is not. > > Microsoft is still around, but they dropped their Fortran compiler a long time ago. > Microsoft is still around, but they dropped Visual Basic for Applications from > ?the Macintosh port of Office several years ago, and while it had been pretty bad > ?having to use VBA, it was even worse NOT being able to use it. > Microsoft is still around, but they sold MS-Test to Rational who renamed it to > ?Rational Visual Test; they got bought by IBM; I _think_ it's now SQABasic > > One of my colleagues is really keen on Rational Purify. ?It's still around, > but if you look up the Windows support, it's "Intel IA-32" only, even under > Windows 7. ?No 64-bit Windows. ?64-bit Linux, yes. ?But if you are a > Windows developer, Purify has moved away from you by standing still. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Tue Mar 13 07:09:17 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 13 Mar 2012 07:09:17 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: Hi guys, I'd like to know if there's a good and portable method to check if a directory is empty (or not)? In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. I've tested the following working calls: a. file:list_dir/1 b. filelib:wildcard/2 The problem with them is they both return a list of filenames. I'd like to avoid getting this list back to be able to check if it's empty or not. Advices please? Regards Zabrane From rtrlists@REDACTED Tue Mar 13 07:42:34 2012 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 13 Mar 2012 06:42:34 +0000 Subject: [erlang-questions] Data in state records In-Reply-To: References: Message-ID: On Mon, Mar 12, 2012 at 5:05 PM, Matthew Evans wrote: > Hi, > > Many of my processes (gen_servers) need to store some kind of indexed data > in the state record. Usually this involves creating a set/dict/ets table in > the init/1 function, assigning it to an element in the state record and > using the data on each handle_call/cast/info. > > My usual strategy is anything that is 100% private to my gen_server and > will contain less than 50 or so records I will use sets/gb_trees or > similar. In all other cases I would use an ETS table. Is there an > official recommendation? > > Thanks > > Matt > > It depends a lot on your usage patterns. But for anything less than 50 to 100, I tend to simply use a proplist or keylist, as they "print" nicely in debugging situations. A set is good for when you need a set. The other data structures have characteristics that map well onto certain types of usage patterns. But my code hasn't so far needed anything more complicated. I view ets as a good stepping point to dets, if I know that I may want to persist stuff, I might opt for ets first. But even that has turned out to be premature optimisation. It might be nice to have an overview summary of the various standard data structures with examples of when each of them is most appropriate. But in summary, I'd recommend sticking with the simplest data structure, and only increase complexity if you find that you need it. I'm one of those people who have no fear of rewriting code over and over, though. So YMMV. Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Mar 13 08:12:30 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 13 Mar 2012 20:12:30 +1300 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: <9CCBD7CD-27B3-4407-BD61-EA9EF89278D4@cs.otago.ac.nz> On 13/03/2012, at 7:09 PM, Zabrane Mickael wrote: > Hi guys, > > I'd like to know if there's a good and portable method to check if a directory is empty (or not)? > In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. > > I've tested the following working calls: > a. file:list_dir/1 > b. filelib:wildcard/2 I don't know about filelib, but file:list_dir/1 used to work by going through several layers and finally sending a command to the 'efile' driver. You might have to extend the driver. For POSIX systems you could always run an outboard command. The Windows 'SUA' kit should be POSIX enough for this to work. From rtrlists@REDACTED Tue Mar 13 08:35:28 2012 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 13 Mar 2012 07:35:28 +0000 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: I have slowly come to the realisation, that the whole Enterprise Java world is the result of business computing slowly moving away from Cobol. I have never had much dealings with business computing (e.g., banks, ticket systems, IT governance) until 5 years ago or so. Up until that point, I was perplexed by the rise of J2EE. It seemed overly complex to my eyes. But up until that point I worked first in compilers, then in stats/visualisation. In hindsight, those areas are pretty niche and allow for quite a lot of freedom in choice of technology. The world of business computing, with it's heavy use of VM architectures (not the new ones, mind you, the old IBM ones, CICS regions on mainframes), huge databases, communication architectures (CORBA), and business oriented languages (Cobol, PL/I, and, amazingly, APL) appears to be coming to the tail end of a transition from IBM/Cobol/CORBA to VMware/Java/J2EE. There's still enormous amounts of stuff in the old tech there, though. So, this transition, even after 15 odd years, is not yet done. One way of getting Erlang into a project, is by using it for prototyping. But if you do this, you must also be willing to throw away the prototype afterwards and do it the approved way. Prototypes have a nasty way of becoming products, though. So, this is one way in. And some of the libraries that Erlang supplies (the CORBA and SNMP stuff, for example, and third party things like RabbitMQ) can show applicability in your domain (if it's not moved fully to J2EE yet). A rather dramatic trick is to take the risk onto yourself. If you're reasonably certain that your solution (or prototype :-) will be successful, and you're willing to take the chance of failing at what you're about to do, then you always have the option of "just doing it". If it fails and you're found out to have been using the "wrong" technology, then you're the culprit. The risk was with yourself. There's nothing that focuses your attention more than getting yourself into a situation, where "failure is not an option" :-D This is obviously only possible on a smallish project with just you and maybe one or two other likeminded individuals. Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: From rapsey@REDACTED Tue Mar 13 08:37:01 2012 From: rapsey@REDACTED (Rapsey) Date: Tue, 13 Mar 2012 08:37:01 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: <9CCBD7CD-27B3-4407-BD61-EA9EF89278D4@cs.otago.ac.nz> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <9CCBD7CD-27B3-4407-BD61-EA9EF89278D4@cs.otago.ac.nz> Message-ID: os:cmd isn't really portable but easy to use and all it takes is a case statement for windows. Sergej On Tue, Mar 13, 2012 at 8:12 AM, Richard O'Keefe wrote: > > On 13/03/2012, at 7:09 PM, Zabrane Mickael wrote: > > > Hi guys, > > > > I'd like to know if there's a good and portable method to check if a > directory is empty (or not)? > > In my case, the directory can contains huge number (more than 150K in > average) of files if it is not empty. > > > > I've tested the following working calls: > > a. file:list_dir/1 > > b. filelib:wildcard/2 > > I don't know about filelib, but file:list_dir/1 used to work by going > through several > layers and finally sending a command to the 'efile' driver. You might > have to extend > the driver. > > For POSIX systems you could always run an outboard command. The Windows > 'SUA' kit should > be POSIX enough for this to work. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve@REDACTED Tue Mar 13 08:41:37 2012 From: steve@REDACTED (Steve Strong) Date: Tue, 13 Mar 2012 08:41:37 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: Not a good solution if other apps are using the directory, but if it's just your app you could call file:del_dir() - if it's empty, it will return ok if not you'll get an {error,eexist} error. No idea on the performance characteristics, but suspect it's pretty good... -- Steve Strong @srstrong Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Tuesday, 13 March 2012 at 07:09, Zabrane Mickael wrote: > Hi guys, > > I'd like to know if there's a good and portable method to check if a directory is empty (or not)? > In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. > > I've tested the following working calls: > a. file:list_dir/1 > b. filelib:wildcard/2 > > The problem with them is they both return a list of filenames. > I'd like to avoid getting this list back to be able to check if it's empty or not. > > Advices please? > > Regards > Zabrane > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Tue Mar 13 08:49:32 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 13 Mar 2012 08:49:32 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: <8AE138BB-DACA-4B88-A7EF-03489CB44A32@gmail.com> I already tried the file:del_dir/1 trick. The only downside is when this directory is really empty ... then it will be deleted. My app will not work anymore as the existence of the directory is mandatory. Regards, Zabrane On Mar 13, 2012, at 8:41 AM, Steve Strong wrote: > Not a good solution if other apps are using the directory, but if it's just your app you could call file:del_dir() - if it's empty, it will return ok if not you'll get an {error,eexist} error. No idea on the performance characteristics, but suspect it's pretty good... > > -- > Steve Strong > @srstrong > > Sent with Sparrow > > On Tuesday, 13 March 2012 at 07:09, Zabrane Mickael wrote: > >> Hi guys, >> >> I'd like to know if there's a good and portable method to check if a directory is empty (or not)? >> In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. >> >> I've tested the following working calls: >> a. file:list_dir/1 >> b. filelib:wildcard/2 >> >> The problem with them is they both return a list of filenames. >> I'd like to avoid getting this list back to be able to check if it's empty or not. >> >> Advices please? >> >> Regards >> Zabrane >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve@REDACTED Tue Mar 13 09:03:29 2012 From: steve@REDACTED (Steve Strong) Date: Tue, 13 Mar 2012 09:03:29 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: <8AE138BB-DACA-4B88-A7EF-03489CB44A32@gmail.com> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <8AE138BB-DACA-4B88-A7EF-03489CB44A32@gmail.com> Message-ID: <2E65DCDD-0611-4950-8F09-708C20592289@srstrong.com> Yeah, if there are other processes (erlang or os) then that would be a problem :) - obviously easy to recreate the directory if it did get deleted but you would have that time when it wasn't there. Sent from my iPhone On 13 Mar 2012, at 08:49, Zabrane Mickael wrote: > I already tried the file:del_dir/1 trick. > > The only downside is when this directory is really empty ... then it will be deleted. > My app will not work anymore as the existence of the directory is mandatory. > > Regards, > Zabrane > > > On Mar 13, 2012, at 8:41 AM, Steve Strong wrote: > >> Not a good solution if other apps are using the directory, but if it's just your app you could call file:del_dir() - if it's empty, it will return ok if not you'll get an {error,eexist} error. No idea on the performance characteristics, but suspect it's pretty good... >> >> -- >> Steve Strong >> @srstrong >> >> Sent with Sparrow >> >> On Tuesday, 13 March 2012 at 07:09, Zabrane Mickael wrote: >> >>> Hi guys, >>> >>> I'd like to know if there's a good and portable method to check if a directory is empty (or not)? >>> In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. >>> >>> I've tested the following working calls: >>> a. file:list_dir/1 >>> b. filelib:wildcard/2 >>> >>> The problem with them is they both return a list of filenames. >>> I'd like to avoid getting this list back to be able to check if it's empty or not. >>> >>> Advices please? >>> >>> Regards >>> Zabrane >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Tue Mar 13 09:24:25 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 13 Mar 2012 09:24:25 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: <2E65DCDD-0611-4950-8F09-708C20592289@srstrong.com> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <8AE138BB-DACA-4B88-A7EF-03489CB44A32@gmail.com> <2E65DCDD-0611-4950-8F09-708C20592289@srstrong.com> Message-ID: <4AC1A82E-9E6F-42C6-B3DA-A8884D955D4C@gmail.com> Yep. Any other advices/hints? Regards, Zabrane On Mar 13, 2012, at 9:03 AM, Steve Strong wrote: > Yeah, if there are other processes (erlang or os) then that would be a problem :) - obviously easy to recreate the directory if it did get deleted but you would have that time when it wasn't there. > > Sent from my iPhone > > On 13 Mar 2012, at 08:49, Zabrane Mickael wrote: > >> I already tried the file:del_dir/1 trick. >> >> The only downside is when this directory is really empty ... then it will be deleted. >> My app will not work anymore as the existence of the directory is mandatory. >> >> Regards, >> Zabrane >> >> >> On Mar 13, 2012, at 8:41 AM, Steve Strong wrote: >> >>> Not a good solution if other apps are using the directory, but if it's just your app you could call file:del_dir() - if it's empty, it will return ok if not you'll get an {error,eexist} error. No idea on the performance characteristics, but suspect it's pretty good... >>> >>> -- >>> Steve Strong >>> @srstrong >>> >>> Sent with Sparrow >>> >>> On Tuesday, 13 March 2012 at 07:09, Zabrane Mickael wrote: >>> >>>> Hi guys, >>>> >>>> I'd like to know if there's a good and portable method to check if a directory is empty (or not)? >>>> In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. >>>> >>>> I've tested the following working calls: >>>> a. file:list_dir/1 >>>> b. filelib:wildcard/2 >>>> >>>> The problem with them is they both return a list of filenames. >>>> I'd like to avoid getting this list back to be able to check if it's empty or not. >>>> >>>> Advices please? >>>> >>>> Regards >>>> Zabrane >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Tue Mar 13 09:51:30 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 13 Mar 2012 09:51:30 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: <4F5F0A92.8010006@gmail.com> On 03/13/2012 07:09 AM, Zabrane Mickael wrote: > Hi guys, > > I'd like to know if there's a good and portable method to check if a directory is empty (or not)? > In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. > > I've tested the following working calls: > a. file:list_dir/1 > b. filelib:wildcard/2 > > The problem with them is they both return a list of filenames. > I'd like to avoid getting this list back to be able to check if it's empty or not. I don't think it can be done. E.g., looking at the POSIX interface towards directories, a directory is treated as a sequence of names and corresponding inode numbers. There is no function for getting the number of entries of a directory, and I assume that in many file systems the directory size is simply never stored explicitly. The only portable method is to get the list of entries and count them yourself, i.e., using file:list_dir/1. In my experience, file systems may _handle_ 100K+ files in directories, and opening individual files may work fine when you know their exact names, but reading the whole directory list can be slow, and if you need to traverse all the inodes to check time stamps or similar, you might as well go for lunch. /Richard From zerthurd@REDACTED Tue Mar 13 10:30:02 2012 From: zerthurd@REDACTED (Maxim Treskin) Date: Tue, 13 Mar 2012 15:30:02 +0600 Subject: [erlang-questions] Rebar, common_test and cover issue Message-ID: Hello I have strange behaviour of rebar+common_test+cover. There is two files in ./test/: my_module1_SUITE.erl my_module2_SUITE.erl where ct suites defined. When I run `rebar ct verbose=1` I see: ``` ... Cover compiling 'myapp' (16 files) - this may take some time... done ... Cover analysing... WARNING: Analysis failed for my_module1. Reason: {error,{not_cover_compiled,my_module1}} WARNING: Analysis failed for my_module2. Reason: {error,{not_cover_compiled,my_module2}} ``` Why I see no any error before at cover compilation step? Moreover, if I just replace my_module1_SUITE:all/0 to return just empty list [], warning "Analysis failed for my_module1" disappears, and cover for this file appears in cover.html log. Can you explain me this strange issues? Is this any bug or expected behaviour? Thank you. -- Maxim Treskin http://metachord.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Tue Mar 13 10:38:42 2012 From: erlangsiri@REDACTED (Siri Hansen) Date: Tue, 13 Mar 2012 10:38:42 +0100 Subject: [erlang-questions] Rebar, common_test and cover issue In-Reply-To: References: Message-ID: This would typically happen if your test loads my_module1 and my_module2. Could that be the case? /siri Den 10:30 13. mars 2012 skrev Maxim Treskin f?lgende: > Hello > > I have strange behaviour of rebar+common_test+cover. > There is two files in ./test/: > > my_module1_SUITE.erl > my_module2_SUITE.erl > > where ct suites defined. When I run `rebar ct verbose=1` I see: > > ``` > ... > Cover compiling 'myapp' (16 files) - this may take some time... done > ... > Cover analysing... > WARNING: Analysis failed for my_module1. Reason: > {error,{not_cover_compiled,my_module1}} > WARNING: Analysis failed for my_module2. Reason: > {error,{not_cover_compiled,my_module2}} > ``` > > Why I see no any error before at cover compilation step? > > Moreover, if I just replace my_module1_SUITE:all/0 to return just empty > list [], warning "Analysis failed for my_module1" disappears, and cover for > this file appears in cover.html log. > > Can you explain me this strange issues? Is this any bug or expected > behaviour? > > Thank you. > > -- > Maxim Treskin > http://metachord.com > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zerthurd@REDACTED Tue Mar 13 10:55:53 2012 From: zerthurd@REDACTED (Maxim Treskin) Date: Tue, 13 Mar 2012 16:55:53 +0700 Subject: [erlang-questions] Rebar, common_test and cover issue In-Reply-To: References: Message-ID: Yes, this modules loaded in tests. How I can fix this? Rename SUIT modules? On 13 March 2012 16:38, Siri Hansen wrote: > This would typically happen if your test loads my_module1 and my_module2. > Could that be the case? > /siri > > Den 10:30 13. mars 2012 skrev Maxim Treskin f?lgende: > >> Hello >> >> I have strange behaviour of rebar+common_test+cover. >> There is two files in ./test/: >> >> my_module1_SUITE.erl >> my_module2_SUITE.erl >> >> where ct suites defined. When I run `rebar ct verbose=1` I see: >> >> ``` >> ... >> Cover compiling 'myapp' (16 files) - this may take some time... done >> ... >> Cover analysing... >> WARNING: Analysis failed for my_module1. Reason: >> {error,{not_cover_compiled,my_module1}} >> WARNING: Analysis failed for my_module2. Reason: >> {error,{not_cover_compiled,my_module2}} >> ``` >> >> Why I see no any error before at cover compilation step? >> >> Moreover, if I just replace my_module1_SUITE:all/0 to return just empty >> list [], warning "Analysis failed for my_module1" disappears, and cover for >> this file appears in cover.html log. >> >> Can you explain me this strange issues? Is this any bug or expected >> behaviour? >> >> Thank you. >> >> -- >> Maxim Treskin >> http://metachord.com >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- Maxim Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Tue Mar 13 10:58:46 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 13 Mar 2012 10:58:46 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: 2012/3/13 Zabrane Mickael : > Hi guys, > > I'd like to know if there's a good and portable method to check if a directory is empty (or not)? > In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. > > I've tested the following working calls: > a. file:list_dir/1 > b. filelib:wildcard/2 > > The problem with them is they both return a list of filenames. > I'd like to avoid getting this list back to be able to check if it's empty or not. > > Advices please? Maybe call opendir, then readdir once (through NIFs). If it returns NULL, the directory is empty. I'm not sure about the portability, though. > Regards > Zabrane > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Tue Mar 13 11:18:57 2012 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 13 Mar 2012 11:18:57 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: Message-ID: Hang on - java is not just a programming language it's an entire industry. Erlang is not an industry it's a programming language. I hope software Darwinism will prevail. All other things being equal a company that bets on Erlang will get to market before a company that bets on Java. The former will gain commerical advantage by doing do. Of course, all things are not equal so this may of may not happen in practice. On average companies that bet on the wrong technology should eventually go bankrupt, company that bet right should prosper. There seems to be a lot more Erlang work going on under-the-radar than is commonly realized but since this involves a real commercial advantage to the guys using Erlang they are often reluctant to talk about it. Companies talk openly about the stuff that everybody knows - they don't talk about the stuff that gives them a commercial advantage. In the Erlang world wo things happen: - drip drip drip - splash drip drip drip is the steady relentless addition of new programmers who "get it" - gains are made one-at-a-time- but year on year the numbers grow and grow and grow. Splash happens every few years - it's mochiweb, its couchDB, it's klarna. In the years when there is no splash - we think "oh dear nothing is happening" then comes splash and we get all excited. But something is happening and we don't notice, it's drip drip drip. You don't notice a bucket that's slowly filling with water, but you do notice it when it spills over. I get excited when I sigh a copy of my book and meet some young guy who is starting off from zero - and I think - "this guy might use Erlang to invent something really fantastic .." now that is way cool - that is the the drip drip drip that excites me. Changing how people think will eventually change how they behave, but it may take a while. Computer science grows in drips and splashes - connecting hypertext and networking was obvious "after the event" after Tim Berners Lee made the connection, but not before. So my philosophy is to get my watering can out and water my plants in the hope that they will become big and strong and provide us with wholesome food ... Cheers /Joe On Sun, Mar 11, 2012 at 6:09 PM, Shahrdad Shadab wrote: > When I was learning Erlang and understanding its capabilities I really > cannot find a satisfactory answer to the question that > why in North America companies like former BEA, former Sun, Oracle , ... use > Java to build commercial application servers instead of Erlang? > From technical perspective such decision doesn't make any sense to me for > following reasons: > > _Java is not a fault tolerant. > _Java performance is nowhere near Erlang. > _Concurrent programming in Java is a pain. > _J2ee Technology introduced as add on to java to make communication cross > servers possible (i.e web services? XML SCHEMA, WSDL) is unreasonably and > grotesquely complicated. (This complication? is dictated by the technology > and not by the problem domain) > _Java is not distributed language (No asynch communication is possible > without JMS, also RMI stub solution is more complicated than it should be). > > and many more reasons I can list here. > > Thanks in advance > Shahrdad > -- > Software Architect & Computer Scientist > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From erlang@REDACTED Tue Mar 13 11:34:33 2012 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 13 Mar 2012 11:34:33 +0100 Subject: [erlang-questions] Erlang meets physics In-Reply-To: References: Message-ID: Great news - spread the word ! Just for the record Erlang programmers numbers 1 and 2 (ie myself and Robert Virding) are both ex physicists. When I lecture I often point out the similarity between causality and message reception. You don't know that something has happened until you get a message telling that it has happened. (In physics it's a ray of light, or a photon, or something - forgetting entanglement for the moment) In computing it's the reception of a message. As a ex physicist I know that we can't say anything about simultaneous events occurring at different places in space-time - turn this into computer science and the same arguments apply to things like making sure replicated data is consistent on remote sites - well you can't - at least if you want to change it - Brewer's CAP theorem applies - which for a physicist makes perfect sense. Also as an ex physicist I realize that things do actually happen in parallel in the real world, so modelling them in a sequential programming language (if I wanted to do that) is big time crazy - just describe the parallel stuff in a concurrent language and the program writes itself. Wait a few years till we have million core computers and the parallel problems can be solved 1:1 on parallel computers - and programming simulations and so on will be really easy - but don't even think about doing it in a sequential language... Cheers /Joe On Mon, Mar 12, 2012 at 2:34 AM, Jared Kofron wrote: > Hi All, > I've been using Erlang at work for a few years now, and I thought I'd throw my experience out there, as > my application is a little different than what you usually see on the list - I am a graduate student at the > Center for Nuclear Physics and Astrophysics at the University of Washington, and use Erlang extensively > in my work. > > In my experience, something that Erlang is really great at but doesn't receive much attention for these days > is managing and interacting with hardware. ?In any physics experiment of even modest sizes, you wind up > having to keep track of the state of various pieces of equipment, often modify that state, and constantly > interrogate particular values. ?For example, we might want to change the current in a magnetic trap, turn > that trap off altogether, or simply read back the voltage drop across our superconducting magnet. > > So far, I have deployed Erlang in this zone for two separate experiments (SNO+, a large particle physics > experiment in Canada) and Project 8 (a small nuclear physics experiment here in Seattle). ?Both times have > been great successes, and I have found the reception of Erlang in this market to be great. ?In general, what > I have done is wrap a hardware management layer with some kind of outside world interface. For SNO+, we > used Webmachine and RESTful control, and for Project 8 we actually conduct all communication > by using CouchDB as a message passing interface. > > Physicists are suspicious creatures, but once you demonstrate the feature set that you get for practically > free with OTP, they see the advantage pretty quickly. ?On top of that, the development cycle for sophisticated > applications can be greatly reduced - more than once it made my group float to the top in terms of meeting > goals. > > In short, as far as I am concerned, Erlang has found a new niche in the world of Physics, and I intend to > spread the word as much as I can! > > Jared Kofron > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From edward.yu.wang@REDACTED Tue Mar 13 12:32:53 2012 From: edward.yu.wang@REDACTED (Edward Wang) Date: Tue, 13 Mar 2012 19:32:53 +0800 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: There might be an obvious solution. That is, if certain file is bound to be present out of that 150k files, then you can just check the presence of that particular file. On Mar 13, 2012 2:10 PM, "Zabrane Mickael" wrote: > Hi guys, > > I'd like to know if there's a good and portable method to check if a > directory is empty (or not)? > In my case, the directory can contains huge number (more than 150K in > average) of files if it is not empty. > > I've tested the following working calls: > a. file:list_dir/1 > b. filelib:wildcard/2 > > The problem with them is they both return a list of filenames. > I'd like to avoid getting this list back to be able to check if it's empty > or not. > > Advices please? > > Regards > Zabrane > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lambdadmitry@REDACTED Tue Mar 13 13:38:08 2012 From: lambdadmitry@REDACTED (Dmitry Groshev) Date: Tue, 13 Mar 2012 15:38:08 +0300 Subject: [erlang-questions] Polymorphic opaque types and dialyzer Message-ID: Dialyzer can't handle polymorphic opaque type: $ echo "-module(opaque)." >> opaque.erl $ echo "-opaque test(A) :: {A, A}." >> opaque.erl $ dialyzer --src opaque.erl Checking whether the PLT /home/si14/work/repos/saelmon/.saelmon_dialyzer.plt is up-to-date... yes Proceeding with analysis... =ERROR REPORT==== 13-Mar-2012::16:32:50 === Error in process <0.30.0> with exit value: {{nocatch,{error,"Polymorphic opaque types not supported yet"}},[{erl_types,'-t_opaque_from_records/1-anonymous-2-',3,[{file,"erl_types.erl"},{line,564}]},{dict,map_bucket,2,[{file,"dict.erl"},{line,459}]},{dict,map_bkt_list,2,... dialyzer: Analysis failed with error: {{nocatch,{error,"Polymorphic opaque types not supported yet"}}, [{erl_types,'-t_opaque_from_records/1-anonymous-2-',3, [{file,"erl_types.erl"},{line,564}]}, {dict,map_bucket,2,[{file,[...]},{line,...}]}, {dict,map_bkt_list,2,[{file,...},{...}]}, {dict,map_bkt_list,2,[{...}|...]}, {dict,map_seg_list,2,[...]}, {dict,map_dict,2,...}, {erl_types,t_opaque_from_records,...}, {dialyzer_typesig,...}]} Last messages in the log cache: Reading files and computing callgraph... done in 0.06 secs Removing edges... done in 0.01 secs Typesig analysis for SCC: [{opaque,module_info,0}] Is there any solution to this? Will it be fixed in upcoming releases of Erlang? From erlangsiri@REDACTED Tue Mar 13 13:58:34 2012 From: erlangsiri@REDACTED (Siri Hansen) Date: Tue, 13 Mar 2012 13:58:34 +0100 Subject: [erlang-questions] Rebar, common_test and cover issue In-Reply-To: References: Message-ID: This is not related to the name of the SUITEs. What happens is that the modules (my_module1 and my_module2 - and possible other modules as derived from your cover spec file) are cover compiled and loaded by common_test before the test starts. The result of the cover compilation exists only in memory so if the test code explicitly loads these modules again the original beam files will be used and the cover compiled version disappears. The only way to overcome this is to avoid loading of code in your tests. /siri Den 10:55 13. mars 2012 skrev Maxim Treskin f?lgende: > Yes, this modules loaded in tests. How I can fix this? Rename SUIT modules? > > > On 13 March 2012 16:38, Siri Hansen wrote: > >> This would typically happen if your test loads my_module1 and my_module2. >> Could that be the case? >> /siri >> >> Den 10:30 13. mars 2012 skrev Maxim Treskin f?lgende: >> >>> Hello >>> >>> I have strange behaviour of rebar+common_test+cover. >>> There is two files in ./test/: >>> >>> my_module1_SUITE.erl >>> my_module2_SUITE.erl >>> >>> where ct suites defined. When I run `rebar ct verbose=1` I see: >>> >>> ``` >>> ... >>> Cover compiling 'myapp' (16 files) - this may take some time... done >>> ... >>> Cover analysing... >>> WARNING: Analysis failed for my_module1. Reason: >>> {error,{not_cover_compiled,my_module1}} >>> WARNING: Analysis failed for my_module2. Reason: >>> {error,{not_cover_compiled,my_module2}} >>> ``` >>> >>> Why I see no any error before at cover compilation step? >>> >>> Moreover, if I just replace my_module1_SUITE:all/0 to return just empty >>> list [], warning "Analysis failed for my_module1" disappears, and cover for >>> this file appears in cover.html log. >>> >>> Can you explain me this strange issues? Is this any bug or expected >>> behaviour? >>> >>> Thank you. >>> >>> -- >>> Maxim Treskin >>> http://metachord.com >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > > -- > Maxim Treskin > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Tue Mar 13 14:03:11 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 13 Mar 2012 22:03:11 +0900 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: If it's any consolation, Ruby hackers are also stumped. http://www.ruby-forum.com/topic/84762 -michael turner On Tue, Mar 13, 2012 at 8:32 PM, Edward Wang wrote: > There might be an obvious solution. That is, if certain file is bound to be > present out of that 150k files, then you can just check the presence of that > particular file. > > On Mar 13, 2012 2:10 PM, "Zabrane Mickael" wrote: >> >> Hi guys, >> >> I'd like to know if there's a good and portable method to check if a >> directory is empty (or not)? >> In my case, the directory can contains huge number (more than 150K in >> average) of files if it is not empty. >> >> I've tested the following working calls: >> a. file:list_dir/1 >> b. filelib:wildcard/2 >> >> The problem with them is they both return a list of filenames. >> I'd like to avoid getting this list back to be able to check if it's empty >> or not. >> >> Advices please? >> >> Regards >> Zabrane >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From comptekki@REDACTED Tue Mar 13 15:33:35 2012 From: comptekki@REDACTED (Wes James) Date: Tue, 13 Mar 2012 08:33:35 -0600 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: On Tue, Mar 13, 2012 at 12:09 AM, Zabrane Mickael wrote: > Hi guys, > > I'd like to know if there's a good and portable method to check if a directory is empty (or not)? > In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. One possibility: os:cmd("find folder -d -empty"). returns [] on false returns ["folder\n"] if true if you need this on windows, cygwin might work -wes From mattevans123@REDACTED Tue Mar 13 15:56:01 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Tue, 13 Mar 2012 10:56:01 -0400 Subject: [erlang-questions] Erlang Pre-Processor (epp) issue Message-ID: Hi, Not sure if this should be a "bugs" or "questions" issue, since technically it isn't a bug. Perhaps it's a change request? We are using Apache Thrift to get Java, Erlang and a small Python test app to communicate. We all have access to the Thrift files, even non-Erlang developers. Unfortunately one particular set of these files has a nasty include hierarchy causing an "include too deep" error from epp. Personally I'd rather ensure that the include depth wasn't so bad, but I can't guarantee what Thrift does and I certainly can't guarantee what the Java developers do (since Java doesn't have include files it isn't a problem for them). I have changed epp.erl in our system to change the depth check from 8 to 10, and it gets around the problem for now. What I was wondering is if this should be a configurable option? I'm sure I am one of the first people to run into this issue..... Cheers Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Tue Mar 13 16:18:01 2012 From: comptekki@REDACTED (Wes James) Date: Tue, 13 Mar 2012 09:18:01 -0600 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: On Tue, Mar 13, 2012 at 12:09 AM, Zabrane Mickael wrote: > Hi guys, > > I'd like to know if there's a good and portable method to check if a directory is empty (or not)? > In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. Another thing to look at. When I create an empty folder on my mac (I tested this with 2 folders and some subfolders), they seem to always have a size of 68 bytes when first created (empty). I wrote this to work on that idea: -module(test). -include_lib("kernel/include/file.hrl"). -define(DARWIN_EMPTY_FOLDER_SIZE,68). -export([start/0]). start()-> is_dir_empty("blah"). is_dir_empty(Dir) -> {ok, FileInfo} = file:read_file_info(Dir), case FileInfo#file_info.size of ?DARWIN_EMPTY_FOLDER_SIZE -> empty; _ -> notempty end. -wes From bengt.kleberg@REDACTED Tue Mar 13 16:27:07 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 13 Mar 2012 16:27:07 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: <1331652427.4963.20.camel@seasc1137> Greetings, An empty directory is 2048 bytes on this machine: Linux seasc1137 2.6.27.42-0.1-default #1 SMP 2010-01-06 16:07:25 +0100 i686 athlon i386 GNU/Linux It stays that size when 1 file is added. bengt On Tue, 2012-03-13 at 16:18 +0100, Wes James wrote: > On Tue, Mar 13, 2012 at 12:09 AM, Zabrane Mickael wrote: > > Hi guys, > > > > I'd like to know if there's a good and portable method to check if a directory is empty (or not)? > > In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. > > Another thing to look at. When I create an empty folder on my mac (I > tested this with 2 folders and some subfolders), they seem to always > have a size of 68 bytes when first created (empty). > > I wrote this to work on that idea: > > -module(test). > > -include_lib("kernel/include/file.hrl"). > > -define(DARWIN_EMPTY_FOLDER_SIZE,68). > > -export([start/0]). > > start()-> > is_dir_empty("blah"). > > is_dir_empty(Dir) -> > {ok, FileInfo} = file:read_file_info(Dir), > case FileInfo#file_info.size of > ?DARWIN_EMPTY_FOLDER_SIZE -> empty; > _ -> notempty > end. > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Tue Mar 13 16:29:29 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 13 Mar 2012 16:29:29 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: <58A57DF4-4A3E-40AB-95E9-30B140071785@gmail.com> Awesome Wes. On my OSX 10.7.3: 68 bytes On Linux 32/64 bits Fedora: 4096 bytes On Mar 13, 2012, at 4:18 PM, Wes James wrote: > On Tue, Mar 13, 2012 at 12:09 AM, Zabrane Mickael wrote: >> Hi guys, >> >> I'd like to know if there's a good and portable method to check if a directory is empty (or not)? >> In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. > > Another thing to look at. When I create an empty folder on my mac (I > tested this with 2 folders and some subfolders), they seem to always > have a size of 68 bytes when first created (empty). > > I wrote this to work on that idea: > > -module(test). > > -include_lib("kernel/include/file.hrl"). > > -define(DARWIN_EMPTY_FOLDER_SIZE,68). > > -export([start/0]). > > start()-> > is_dir_empty("blah"). > > is_dir_empty(Dir) -> > {ok, FileInfo} = file:read_file_info(Dir), > case FileInfo#file_info.size of > ?DARWIN_EMPTY_FOLDER_SIZE -> empty; > _ -> notempty > end. > > -wes Regards, Zabrane From comptekki@REDACTED Tue Mar 13 16:33:33 2012 From: comptekki@REDACTED (Wes James) Date: Tue, 13 Mar 2012 09:33:33 -0600 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: <58A57DF4-4A3E-40AB-95E9-30B140071785@gmail.com> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <58A57DF4-4A3E-40AB-95E9-30B140071785@gmail.com> Message-ID: Per Bengt's experience. What happens if you add a file? Does the size change? -wes On Tue, Mar 13, 2012 at 9:29 AM, Zabrane Mickael wrote: > Awesome Wes. > > On my OSX 10.7.3: ?68 bytes > On Linux 32/64 bits Fedora: 4096 bytes > > On Mar 13, 2012, at 4:18 PM, Wes James wrote: > >> On Tue, Mar 13, 2012 at 12:09 AM, Zabrane Mickael wrote: >>> Hi guys, >>> >>> I'd like to know if there's a good and portable method to check if a directory is empty (or not)? >>> In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. >> >> Another thing to look at. ?When I create an empty folder on my mac (I >> tested this with 2 folders and some subfolders), they seem to always >> have a size of 68 bytes when first created (empty). >> >> I wrote this to work on that idea: >> >> -module(test). >> >> -include_lib("kernel/include/file.hrl"). >> >> -define(DARWIN_EMPTY_FOLDER_SIZE,68). >> >> -export([start/0]). >> >> start()-> >> ? ?is_dir_empty("blah"). >> >> is_dir_empty(Dir) -> >> ? ?{ok, FileInfo} = file:read_file_info(Dir), >> ? ?case FileInfo#file_info.size of >> ? ? ? ??DARWIN_EMPTY_FOLDER_SIZE -> empty; >> ? ? ? _ -> notempty >> ? ?end. >> >> -wes > > Regards, > Zabrane > From zabrane3@REDACTED Tue Mar 13 16:34:39 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 13 Mar 2012 16:34:39 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <58A57DF4-4A3E-40AB-95E9-30B140071785@gmail.com> Message-ID: On Mar 13, 2012, at 4:33 PM, Wes James wrote: > Per Bengt's experience. What happens if you add a file? Does the size change? Nothing change when I added a file. Regards, Zabrane From alfredo.dinapoli@REDACTED Tue Mar 13 16:49:17 2012 From: alfredo.dinapoli@REDACTED (CharlesStain) Date: Tue, 13 Mar 2012 08:49:17 -0700 (PDT) Subject: [erlang-questions] OpenCL.framework cause my NIF library to crash Message-ID: <1331653757763-4469412.post@n4.nabble.com> Hi everyone, I'm developing an Erlang binding to a C++ library of mine, which relies on OpenCL. I set up a basic environment, with proper extern "C", as well as linker flags. In my case the magic words are: -undefined dynamic_lookup -dynamiclib Without these additional flags my NIF shared library won't compile. Everything work well until I try to link OpenCL within my NIF library. Bear in mind that I'm on Mac OS and OpenCL comes as a dynamic library (a Cocoa Framework to be more specific) and going to another platform like Linux or Windows is not feasible. When I try to load my "new" NIF library (with OpenCL linked with it), this is the result: laetus alfredodinapoli$ rebar eunit ==> elaetus (eunit) Compiled src/elaetus_app.erl Compiled src/elaetus_sup.erl Compiled src/elaetus.erl Trace/BPT trap: 5 Which is not very informative about what's going wrong. But I suspect it may be related to the dynamic nature of the OpenCL library. In fact, linking my framework, which is a static library, doesn't cause any issue. I would like to know: a) If anyone has had any contact with the NIF + MacOs + C++ world b) If anyone know a workaround for this annoying situation. Thanks in advance, Alfredo Di Napoli -- View this message in context: http://erlang.2086793.n4.nabble.com/OpenCL-framework-cause-my-NIF-library-to-crash-tp4469412p4469412.html Sent from the Erlang Questions mailing list archive at Nabble.com. From massung@REDACTED Tue Mar 13 16:50:29 2012 From: massung@REDACTED (Jeffrey Massung) Date: Tue, 13 Mar 2012 09:50:29 -0600 Subject: [erlang-questions] SSH module and Agent Forwarding Message-ID: Has anyone been able to accomplish this? I've been scouring Google, documentation, and not finding much help. Does it require using the key_cb option in ssh:connect? I don't care if this takes a chunk of code to get working, but if someone has done this in the past and has some pointers or same code to work from, I'd be very appreciative. Thanks! Jeff M. From comptekki@REDACTED Tue Mar 13 16:57:12 2012 From: comptekki@REDACTED (Wes James) Date: Tue, 13 Mar 2012 09:57:12 -0600 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <58A57DF4-4A3E-40AB-95E9-30B140071785@gmail.com> Message-ID: There is also filelib:file_size Another thing to look at. Can you keep track of when the folder is first created: #file_info.atime (keep somwhere like ets table) then later read #file_info.atime and compare? The problem here is that when a file file is written or the folder is just plane read (I did ls of folder and the atime changed) the atime changes. Will the folder ever be accessed other than to have a file be written to it, if so, this may work. -wes On Tue, Mar 13, 2012 at 9:34 AM, Zabrane Mickael wrote: > > On Mar 13, 2012, at 4:33 PM, Wes James wrote: > >> Per Bengt's experience. ?What happens if you add a file? ?Does the size change? > > Nothing change when I added a file. > > Regards, > Zabrane > From mfidelman@REDACTED Tue Mar 13 16:57:22 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 13 Mar 2012 11:57:22 -0400 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: <1331652427.4963.20.camel@seasc1137> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <1331652427.4963.20.camel@seasc1137> Message-ID: <4F5F6E62.20700@meetinghouse.net> um... given an empty directory: 7> file:list_dir("/Users/mfidelman/empty") == {ok,[]}. true given a non-empty directory: 8> file:list_dir("/Users/mfidelman") == {ok,[]}. false note: file:list_dir seems to do the right thing with invisible files - with an invisible file in ../empty 9> file:list_dir("/Users/mfidelman/empty") == {ok,[]}. false 10> file:list_dir("/Users/mfidelman/empty"). {ok,[".foo"]} Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From taavi@REDACTED Tue Mar 13 17:01:03 2012 From: taavi@REDACTED (Taavi Talvik) Date: Tue, 13 Mar 2012 18:01:03 +0200 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: On Mar 13, 2012, at 5:18 PM, Wes James wrote: > On Tue, Mar 13, 2012 at 12:09 AM, Zabrane Mickael wrote: >> Hi guys, >> >> I'd like to know if there's a good and portable method to check if a directory is empty (or not)? >> In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. On unix systems link_count=2 means, that directory is empty: 3> rr("/usr/local/lib/erlang/lib/kernel-2.14.1/include/file.hrl"). [file_descriptor,file_info] 4> file:read_file_info("."). {ok,#file_info{size = 170,type = directory, access = read_write, atime = {{2012,3,13},{17,57,1}}, mtime = {{2007,9,30},{14,7,32}}, ctime = {{2012,3,11},{3,6,39}}, mode = 16877,links = 5,major_device = 234881026, minor_device = 0,inode = 43374422,uid = 501,gid = 501}} 2> file:read_file_info("empty"). {ok,#file_info{size = 68,type = directory, access = read_write, atime = {{2012,3,13},{17,59,44}}, mtime = {{2012,3,13},{17,59,44}}, ctime = {{2012,3,13},{17,59,44}}, mode = 16877,links = 2,major_device = 234881026, minor_device = 0,inode = 43401684,uid = 501,gid = 501}} best regards, taavi -- "We should forget about small efficiencies, about 97 percent of the time. Premature optimization is the root of all evil." Donald Knuth Taavi Talvik taavi@REDACTED +372 5656 9996 From masklinn@REDACTED Tue Mar 13 17:10:39 2012 From: masklinn@REDACTED (Masklinn) Date: Tue, 13 Mar 2012 17:10:39 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: <4F5F6E62.20700@meetinghouse.net> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <1331652427.4963.20.camel@seasc1137> <4F5F6E62.20700@meetinghouse.net> Message-ID: <4EAB4803-5365-4446-864B-CCE28D8AFB99@masklinn.net> On 2012-03-13, at 16:57 , Miles Fidelman wrote: > um... > > given an empty directory: > > 7> file:list_dir("/Users/mfidelman/empty") == {ok,[]}. > true > > given a non-empty directory: > 8> file:list_dir("/Users/mfidelman") == {ok,[]}. > false Quoting the original message: > The problem with them is they both return a list of filenames. file:list_dir is fine when the directory has a few hundred files tops, but OP quotes file counts beyond 150k, which which yield a few megabytes of data moving around before discarding everything in this final match. Not to mention filesystem (in)efficiencies which may make fetching 150k names *very* slow (on HFS+ ? admittedly one hell of a POS ? merely calling `ls` on a directory with ~60k files and redirecting the result to /dev/null takes a good 200ms) From peerst@REDACTED Tue Mar 13 17:14:43 2012 From: peerst@REDACTED (Peer Stritzinger) Date: Tue, 13 Mar 2012 17:14:43 +0100 Subject: [erlang-questions] Erlang Community Conference In-Reply-To: <4F5E30F5.6080802@berlin.ccc.de> References: <4F54FE0F.6090700@berlin.ccc.de> <4F5E30F5.6080802@berlin.ccc.de> Message-ID: The name @erlconf seems to not exist on Twitter ... (yet? anymore?) On Mon, Mar 12, 2012 at 6:23 PM, John-Paul Bader wrote: > Hey again, > > > so after a lot of discussions and even more bad puns about the conference > name we have settled for ?Erlconf? > > I know - its not the most inspirational name ever but we will make it up to > you with an awesome logo and design. > > If you like, you can follow @erlconf on twitter for all the updates we have. > The corresponding domains are also registered but there is nothing to see > yet. > > Right now we are still looking for a venue and once we have that we also > have a date. > > ~ John > > > Tim Fletcher wrote: >>> >>> Any feedback is appreciated! We will have a dedicated interface for >>> tracking who would be interested to come later on. >> >> >> Sounds good. Website and/or twitter account that we can follow would >> be helpful :) >> >> Cheers, >> Tim >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From peerst@REDACTED Tue Mar 13 17:35:06 2012 From: peerst@REDACTED (Peer Stritzinger) Date: Tue, 13 Mar 2012 17:35:06 +0100 Subject: [erlang-questions] Erlang Community Conference In-Reply-To: References: <4F54FE0F.6090700@berlin.ccc.de> <4F5E30F5.6080802@berlin.ccc.de> Message-ID: Nevermind, it looks like it just can't be found by Twitters people search yet. If you use the url directly to the account it works: https://twitter.com/#!/erlconf Cheers -- Peer On Tue, Mar 13, 2012 at 5:14 PM, Peer Stritzinger wrote: > The name @erlconf seems to not exist on Twitter ... (yet? anymore?) > > On Mon, Mar 12, 2012 at 6:23 PM, John-Paul Bader wrote: >> Hey again, >> >> >> so after a lot of discussions and even more bad puns about the conference >> name we have settled for ?Erlconf? >> >> I know - its not the most inspirational name ever but we will make it up to >> you with an awesome logo and design. >> >> If you like, you can follow @erlconf on twitter for all the updates we have. >> The corresponding domains are also registered but there is nothing to see >> yet. >> >> Right now we are still looking for a venue and once we have that we also >> have a date. >> >> ~ John >> >> >> Tim Fletcher wrote: >>>> >>>> Any feedback is appreciated! We will have a dedicated interface for >>>> tracking who would be interested to come later on. >>> >>> >>> Sounds good. Website and/or twitter account that we can follow would >>> be helpful :) >>> >>> Cheers, >>> Tim >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From bob@REDACTED Tue Mar 13 17:54:19 2012 From: bob@REDACTED (Bob Ippolito) Date: Tue, 13 Mar 2012 09:54:19 -0700 Subject: [erlang-questions] OpenCL.framework cause my NIF library to crash In-Reply-To: <1331653757763-4469412.post@n4.nabble.com> References: <1331653757763-4469412.post@n4.nabble.com> Message-ID: Do you also have the -framework OpenCL flag? Static libraries don't automatically link in their dynamic dependencies. On Tuesday, March 13, 2012, CharlesStain wrote: > Hi everyone, > I'm developing an Erlang binding to a C++ library of mine, which relies on > OpenCL. > I set up a basic environment, with proper extern "C", as well as linker > flags. In my case the magic words are: > > -undefined dynamic_lookup -dynamiclib > > Without these additional flags my NIF shared library won't compile. > Everything work well until I try to link OpenCL within my NIF library. Bear > in mind that I'm on Mac OS and OpenCL comes as a dynamic library (a Cocoa > Framework to be more specific) and going to another platform like Linux or > Windows is not feasible. > When I try to load my "new" NIF library (with OpenCL linked with it), this > is the result: > > laetus alfredodinapoli$ rebar eunit > ==> elaetus (eunit) > Compiled src/elaetus_app.erl > Compiled src/elaetus_sup.erl > Compiled src/elaetus.erl > Trace/BPT trap: 5 > > Which is not very informative about what's going wrong. But I suspect it may > be related to the dynamic nature of the OpenCL library. In fact, linking my > framework, which is a static library, doesn't cause any issue. > I would like to know: > > a) If anyone has had any contact with the NIF + MacOs + C++ world > b) If anyone know a workaround for this annoying situation. > > Thanks in advance, > Alfredo Di Napoli > > -- > View this message in context: http://erlang.2086793.n4.nabble.com/OpenCL-framework-cause-my-NIF-library-to-crash-tp4469412p4469412.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roe.adrian@REDACTED Tue Mar 13 18:33:38 2012 From: roe.adrian@REDACTED (Adrian Roe) Date: Tue, 13 Mar 2012 17:33:38 +0000 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <58A57DF4-4A3E-40AB-95E9-30B140071785@gmail.com> Message-ID: <4F5F84F2.1080102@gmail.com> You need to be careful with things like /atime/ - systems using solid state disks tend to routinely set the /noatime /flag to minimise the number of write-cycles the disk undergoes, which on SSDs limit the lifetime of the disk. I strongly suspect that the link count from file:read_file_info() (as in Taavi's post) is the way forward. Adrian On 13/03/12 15:57, Wes James wrote: > There is also filelib:file_size > > Another thing to look at. Can you keep track of when the folder is > first created: > > #file_info.atime (keep somwhere like ets table) > > then later read #file_info.atime and compare? > > The problem here is that when a file file is written or the folder is > just plane read (I did ls of folder and the atime changed) the atime > changes. > > Will the folder ever be accessed other than to have a file be written > to it, if so, this may work. > > -wes > > On Tue, Mar 13, 2012 at 9:34 AM, Zabrane Mickael wrote: >> On Mar 13, 2012, at 4:33 PM, Wes James wrote: >> >>> Per Bengt's experience. What happens if you add a file? Does the size change? >> Nothing change when I added a file. >> >> Regards, >> Zabrane >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Tue Mar 13 19:05:57 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 13 Mar 2012 19:05:57 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: <9DDDF35D-001E-4E05-A470-78F5C0C1A797@gmail.com> Great Taavi. That's the solution I'll use from now (until a better one will be suggested). Thanks to all of you guys !!! Regards, Zabrane On Mar 13, 2012, at 5:01 PM, Taavi Talvik wrote: > On Mar 13, 2012, at 5:18 PM, Wes James wrote: > >> On Tue, Mar 13, 2012 at 12:09 AM, Zabrane Mickael wrote: >>> Hi guys, >>> >>> I'd like to know if there's a good and portable method to check if a directory is empty (or not)? >>> In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. > > > On unix systems link_count=2 means, that directory is empty: > > 3> rr("/usr/local/lib/erlang/lib/kernel-2.14.1/include/file.hrl"). > [file_descriptor,file_info] > 4> file:read_file_info("."). > {ok,#file_info{size = 170,type = directory, > access = read_write, > atime = {{2012,3,13},{17,57,1}}, > mtime = {{2007,9,30},{14,7,32}}, > ctime = {{2012,3,11},{3,6,39}}, > mode = 16877,links = 5,major_device = 234881026, > minor_device = 0,inode = 43374422,uid = 501,gid = 501}} > > 2> file:read_file_info("empty"). > {ok,#file_info{size = 68,type = directory, > access = read_write, > atime = {{2012,3,13},{17,59,44}}, > mtime = {{2012,3,13},{17,59,44}}, > ctime = {{2012,3,13},{17,59,44}}, > mode = 16877,links = 2,major_device = 234881026, > minor_device = 0,inode = 43401684,uid = 501,gid = 501}} > > best regards, > taavi > -- > "We should forget about small efficiencies, about 97 percent of the time. > Premature optimization is the root of all evil." Donald Knuth > > Taavi Talvik > taavi@REDACTED > +372 5656 9996 > From alex.grigorovich@REDACTED Tue Mar 13 19:28:49 2012 From: alex.grigorovich@REDACTED (Alex Grigorovich) Date: Wed, 14 Mar 2012 01:28:49 +0700 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: On Tue, Mar 13, 2012 at 11:01 PM, Taavi Talvik wrote: > On unix systems link_count=2 means, that directory is empty: Unfortunately, this is not going to work: link count equals 2 for any directory that does not contain sub-directories. Regular files are not linked to their containing directory, therefore they do not affect link count. -- Alex From carlsson.richard@REDACTED Tue Mar 13 19:50:23 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 13 Mar 2012 19:50:23 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: <4F5F96EF.4010100@gmail.com> On 2012-03-13 17:01, Taavi Talvik wrote: > On unix systems link_count=2 means, that directory is empty: Nope, it just means that it has no subdirectories. /Richard From alexey.v.romanov@REDACTED Tue Mar 13 19:57:51 2012 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Tue, 13 Mar 2012 22:57:51 +0400 Subject: [erlang-questions] Logging in port drivers Message-ID: What are best practices on logging in C code for port drivers/NIFs/etc.? Obviously, what I currently do in https://github.com/alexeyr/erlang-sqlite3/blob/master/c_src/sqlite3_drv.c is bad (especially in multiuser environment). But how _should_ it be done instead? Yours, Alexey Romanov From pablo.platt@REDACTED Tue Mar 13 20:20:26 2012 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 13 Mar 2012 12:20:26 -0700 (PDT) Subject: [erlang-questions] Erlang meets physics In-Reply-To: References: Message-ID: <1331666426.33915.YahooMailNeo@web112614.mail.gq1.yahoo.com> How do you interact with the hardware? Do you use GPIB C libr and wrap it with a NIF? ________________________________ From: Joe Armstrong To: Jared Kofron Cc: Erlang Questions Sent: Tuesday, March 13, 2012 12:34 PM Subject: Re: [erlang-questions] Erlang meets physics Great news - spread the word ! Just for the record Erlang programmers numbers 1 and 2 (ie myself and Robert Virding) are both ex physicists. When I lecture I often point out the similarity between causality and message reception. You don't know that something has happened until you get a message telling that it has happened. (In physics it's a ray of light, or a photon, or something - forgetting entanglement for the moment) In computing it's the reception of a message. As a ex physicist I know that we can't say anything about simultaneous events occurring at different places in space-time - turn this into computer science and the same arguments apply to things like making sure replicated data is consistent on remote sites - well you can't - at least if you want to change it - Brewer's CAP theorem applies - which for a physicist makes perfect sense. Also as an ex physicist I realize that things do actually happen in parallel in the real world, so modelling them in a sequential programming language (if I wanted to do that) is big time crazy - just describe the parallel stuff in a concurrent language and the program writes itself. Wait a few years till we have million core computers and the parallel problems can be solved 1:1 on parallel computers - and programming simulations and so on will be really easy - but don't even think about doing it in a sequential language... Cheers /Joe On Mon, Mar 12, 2012 at 2:34 AM, Jared Kofron wrote: > Hi All, > I've been using Erlang at work for a few years now, and I thought I'd throw my experience out there, as > my application is a little different than what you usually see on the list - I am a graduate student at the > Center for Nuclear Physics and Astrophysics at the University of Washington, and use Erlang extensively > in my work. > > In my experience, something that Erlang is really great at but doesn't receive much attention for these days > is managing and interacting with hardware. ?In any physics experiment of even modest sizes, you wind up > having to keep track of the state of various pieces of equipment, often modify that state, and constantly > interrogate particular values. ?For example, we might want to change the current in a magnetic trap, turn > that trap off altogether, or simply read back the voltage drop across our superconducting magnet. > > So far, I have deployed Erlang in this zone for two separate experiments (SNO+, a large particle physics > experiment in Canada) and Project 8 (a small nuclear physics experiment here in Seattle). ?Both times have > been great successes, and I have found the reception of Erlang in this market to be great. ?In general, what > I have done is wrap a hardware management layer with some kind of outside world interface. For SNO+, we > used Webmachine and RESTful control, and for Project 8 we actually conduct all communication > by using CouchDB as a message passing interface. > > Physicists are suspicious creatures, but once you demonstrate the feature set that you get for practically > free with OTP, they see the advantage pretty quickly. ?On top of that, the development cycle for sophisticated > applications can be greatly reduced - more than once it made my group float to the top in terms of meeting > goals. > > In short, as far as I am concerned, Erlang has found a new niche in the world of Physics, and I intend to > spread the word as much as I can! > > Jared Kofron > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From james@REDACTED Tue Mar 13 21:42:25 2012 From: james@REDACTED (james) Date: Tue, 13 Mar 2012 20:42:25 +0000 Subject: [erlang-questions] [ANN] Erlang UUID In-Reply-To: <4F5D6E65.90800@gmail.com> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> Message-ID: <4F5FB131.5090604@mansionfamily.plus.com> >> What you said was -- erroneous. >Only if you don't believe in a free market. I believe that prices >adjust and value is immediately lost in what you describe. That is >why I don't consider that approach profitable. You didn't say profitable. What you DID say, was wrong, and no amount of distraction will change that. What you said is: > This is a little misleading. You can never sell anything that >contains GPL software If I sell GreedyCo a GPL solution that helps them make lots of money, it might not be in their interest to let anyone else have it, in case they lose their edge. They just have to hope their competitors don't want to pay. If I sell NeedyCo a GPL solution where they need (or at least desire) my further input and support, and I make that contingent on their NOT exercising their GPL-provided right to distribute it, then they may consider it in their interest not to upset me. If I sell MuggedSenseless some GPL software that is so arcane and confusing internally and with so little design documentation and so few comments that it is impossible for anyone else to maintain, then that's OK too. (Oh wait - that's normal! Silly me!) I'm not going to argue that what I would call 'premium business market' is easy with GPL unless you want to limit yourself to T&M enhancement contracts or selling insurance. Very cheap mass market may or may not work depending on whether your marketing is more shiny. Whether or not it makes good business sense, or you can look yourself in the mirror afterwards and see a professional, is not relevant. What you said is quite simply wrong. James (What would be interesting would be: here is some GPL stuff for $10m. Pay $1m up front and the other $9m when and if you exercise your right to copy it to someone else. Hmm ...) Look: stop trying to disguise your error under a GPL v BSD religious thing. I don't like it either, OK? But I do like to think I tried to understand what it actually says. From max.lapshin@REDACTED Tue Mar 13 21:47:20 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 13 Mar 2012 23:47:20 +0300 Subject: [erlang-questions] Logging in port drivers In-Reply-To: References: Message-ID: The best practice is to use fprintf(stderr, "... \r\n"). You have forgot \r and without it output may be misaligned. From mjtruog@REDACTED Tue Mar 13 22:11:10 2012 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 13 Mar 2012 14:11:10 -0700 Subject: [erlang-questions] Logging in port drivers In-Reply-To: References: Message-ID: <4F5FB7EE.3050106@gmail.com> On 03/13/2012 11:57 AM, Alexey Romanov wrote: > What are best practices on logging in C code for port > drivers/NIFs/etc.? Obviously, what I currently do in > https://github.com/alexeyr/erlang-sqlite3/blob/master/c_src/sqlite3_drv.c > is bad (especially in multiuser environment). But how _should_ it be > done instead If you have a port driver that uses the async thread pool, it will be similar to the Erlang file API. That way you don't delay the scheduler thread. If you want a simpler way of making a port driver do this, you can use GEPD: https://github.com/okeuday/generic-erlang-port--driver- You would just need to take care of the thread-safety concerns, which should be obvious. You also could always create your own thread pool, but I don't see any added benefit here, just more complication. The only problem with the async thread pool approach is you could log enough that you hog the async thread queues, which prevents usage elsewhere... the only solution to that is making the number of async threads higher. With file writes that shouldn't get blocked (just will be randomly slow, unless there is an immediate failure) that doesn't seem to be a problem you should encounter. - Michael From james@REDACTED Tue Mar 13 22:14:38 2012 From: james@REDACTED (james) Date: Tue, 13 Mar 2012 21:14:38 +0000 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> Message-ID: <4F5FB8BE.8040902@mansionfamily.plus.com> >There are many businesses that invested heavily in the previous >iteration of MS development infrastructure (COM-driven Visual Studio 6 >and related tools), and then suddenly had the rug pulled from >underneath them in 2002 when .Net appeared and they were expected to >rewrite/migrate much of their code (I worked for such a victim, and >gathered many now-worthless skills). When did COM stop working? When did you have to throw away working COM code because you want to use CLR? When did an ability to write modular C++ applications not apply on, say, Linux? You're a fashion victim. It can happen to anyone, and probably will, eventually. Suggesting that there was any rug pulling is bizarre; some greener grass turned up over the fence. Some people will call it progress. If you are working with or for people who 'expect to migrate' just because something shiny showed up, that's a problem and it can happen to you with nearly anything. How long do you think 'cloud' will last? James From ok@REDACTED Tue Mar 13 22:25:30 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 14 Mar 2012 10:25:30 +1300 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: <2E65DCDD-0611-4950-8F09-708C20592289@srstrong.com> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <8AE138BB-DACA-4B88-A7EF-03489CB44A32@gmail.com> <2E65DCDD-0611-4950-8F09-708C20592289@srstrong.com> Message-ID: On 13/03/2012, at 9:03 PM, Steve Strong wrote: > Yeah, if there are other processes (erlang or os) then that would be a problem :) - obviously easy to recreate the directory if it did get deleted but you would have that time when it wasn't there. Recently I was doing some multithread programming in an imperative language. There must be a proper technical term, but the one I've been using is 'evanescent property'. An evanescent property is one that quickly becomes out of date. Let's take an example. while (!work_pool.is_empty()) { /* A */ process(work_pool.remove_most_urgent_task()); } terminate(); The code was originally written for a single-cpu system using co-operative multithreading. So it could rely on nothing happening at point A, so that when it came to call work_pool.remove_most_urgent_task() it was *certain* that work_pool.is_empty() was still false. If you have pre-emptive scheduling, some other threads might be scheduled at point A. By the time you get to work_pool.remove_most_urgent_task() the work pool might have been emptied and refilled any number of times. That isn't _likely_, but it can happen. If you have multiple cores, some other thread might be executing at the very same time as this one, and again, by the time work_pool.remove_most_urgent_task() is called, you have NO IDEA whether work_pool.is_empty() is still false or not. This is the kind of thing that keeps on working when your environment is upgraded, right up to the point where it breaks. And then it's a nightmare to reproduce. You can use locking, of course. while (work_pool.lock(), !work_pool.is_empty()) { task = work_pool.remove_most_urgent_task(); work_pool.unlock(); process(task); } work_pool.unlock(); terminate(); Or better still, you can redesign: while (work_pool.remove_most_urgent_task(&task)) { process(task); } terminate(); although that one is tricky in languages that take a rigidly, blindly, and stupidly doctrinaire approach to what the designers (mis)took to be "object orientation" and ban multi-result methods. (I could name a language like that, but I shan't. Guess, while drinking coffee and listening to a gamelan...) I hope the relevance is clear. A directory is just such a shared mutable data structure, and if there are other processes using the file system you have worse problems than just a potentially missing directory. I don't know any way to lock a directory in UNIX (and I'm pretty confident that I _would_ know; there could be a dozen ways to do it in Windows and I wouldn't have a clue). One approach would be to create a '.lock' file in the directory and lock _that_; you just have to make sure that other OS- level processes also lock the lockfile around modifications to the directory. Frankly, I have been worried about this ever since I first learned UNIX. I've worried about "What happens if someone adds or removes a file while I am scanning a directory?" The manual page If a file is removed from or added to the directory after the most recent call to opendir(3C) or rewinddir(3C), whether a subsequent call to readdir() returns an entry for that file is unspecified. is not reassuring. In fact I suspect it's worse than that: in systems with hashed directories I have a nasty feeling that any change to a directory might result in an open directory scan seeing a name a second time and missing some names that are still there. Solaris has the notion of "name-locking" a file system, preventing any changes to directories while the lock is held, but that's a bit _too_ broad, and you have to own the file system to do that. So when scanning a directory, I religiously do NOTHING but read names and store them, in the hope that getting it over quickly will reduce the odds of craziness. From ok@REDACTED Tue Mar 13 22:32:16 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 14 Mar 2012 10:32:16 +1300 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: <4F5F0A92.8010006@gmail.com> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <4F5F0A92.8010006@gmail.com> Message-ID: <13CF7AF8-400D-4FED-80EA-782E9913CA1C@cs.otago.ac.nz> On 13/03/2012, at 9:51 PM, Richard Carlsson wrote: > On 03/13/2012 07:09 AM, Zabrane Mickael wrote: >> Hi guys, >> >> I'd like to know if there's a good and portable method to check if a directory is empty (or not)? >> In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. >> >> I've tested the following working calls: >> a. file:list_dir/1 >> b. filelib:wildcard/2 >> >> The problem with them is they both return a list of filenames. >> I'd like to avoid getting this list back to be able to check if it's empty or not. > > I don't think it can be done. E.g., looking at the POSIX interface towards directories, a directory is treated as a sequence of names and corresponding inode numbers. There is no function for getting the number of entries of a directory, If you just want to know whether the directory is *empty* or not, it's simple and tolerably fast at the C level: is_empty = 1; dirp = opendir("the directory"); while ((entry = readdir(dirp)) != 0) { if (0 != strcmp(entry->d_name, ".") && 0 != strcmp(entry->d_name, "..") ) { is_empty = 0; break; } } closedir(dirp); The OP's problem is that the Erlang libraries have nothing that mirrors this interface exactly, only things to give you the result of a complete scan. From mfidelman@REDACTED Tue Mar 13 22:39:49 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 13 Mar 2012 17:39:49 -0400 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: <13CF7AF8-400D-4FED-80EA-782E9913CA1C@cs.otago.ac.nz> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <4F5F0A92.8010006@gmail.com> <13CF7AF8-400D-4FED-80EA-782E9913CA1C@cs.otago.ac.nz> Message-ID: <4F5FBEA5.9030502@meetinghouse.net> Richard O'Keefe wrote: > > > If you just want to know whether the directory is *empty* or not, it's simple and tolerably > fast at the C level: > > is_empty = 1; > dirp = opendir("the directory"); > while ((entry = readdir(dirp)) != 0) { > if (0 != strcmp(entry->d_name, ".") > && 0 != strcmp(entry->d_name, "..") > ) { > is_empty = 0; > break; > } > } > closedir(dirp); > > The OP's problem is that the Erlang libraries have nothing that mirrors > this interface exactly, only things to give you the result of a complete > scan. > > Of course you can compile a tiny piece of C code and invoke it via an erlang port. :-) -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From ok@REDACTED Tue Mar 13 22:55:24 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 14 Mar 2012 10:55:24 +1300 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: <46ED2993-C07D-477A-90E6-9584CCF7B10D@cs.otago.ac.nz> On 14/03/2012, at 3:33 AM, Wes James wrote: > On Tue, Mar 13, 2012 at 12:09 AM, Zabrane Mickael wrote: >> Hi guys, >> >> I'd like to know if there's a good and portable method to check if a directory is empty (or not)? >> In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. > > One possibility: > > os:cmd("find folder -d -empty"). The OP asked for a >portable< method. -empty is not portable. From ok@REDACTED Tue Mar 13 22:57:17 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 14 Mar 2012 10:57:17 +1300 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: <9D286D45-87E2-4C97-912C-BE4B63BC5706@cs.otago.ac.nz> On 14/03/2012, at 4:18 AM, Wes James wrote: > On Tue, Mar 13, 2012 at 12:09 AM, Zabrane Mickael wrote: >> Hi guys, >> >> I'd like to know if there's a good and portable method to check if a directory is empty (or not)? >> In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. > > Another thing to look at. When I create an empty folder on my mac (I > tested this with 2 folders and some subfolders), they seem to always > have a size of 68 bytes when first created (empty). The OP asked for a >portable< method. f% mkdir fred f% ls -ld fred drwxr-xr-x 2 me cstaff 512 Mar 14 10:56 fred ^^^ From ok@REDACTED Tue Mar 13 22:59:37 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 14 Mar 2012 10:59:37 +1300 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: <299974FF-76A4-45CF-84DB-F83CF59F0B07@cs.otago.ac.nz> On 14/03/2012, at 5:01 AM, Taavi Talvik wrote: > On Mar 13, 2012, at 5:18 PM, Wes James wrote: > >> On Tue, Mar 13, 2012 at 12:09 AM, Zabrane Mickael wrote: >>> Hi guys, >>> >>> I'd like to know if there's a good and portable method to check if a directory is empty (or not)? >>> In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. > > > On unix systems link_count=2 means, that directory is empty: No it doesn't. It means that the directory has no *subdirectories*. f% mkdir fred f% ls -l fred total 0 f% ls -ld fred drwxr-xr-x 2 me cstaff 512 Mar 14 10:56 fred f% touch fred/jim f% ls -ld fred drwxr-xr-x 2 me cstaff 512 Mar 14 10:58 fred f% mkdir fred/harry f% ls -ld fred drwxr-xr-x 3 me cstaff 512 Mar 14 10:59 fred From taavi@REDACTED Tue Mar 13 23:26:37 2012 From: taavi@REDACTED (Taavi Talvik) Date: Wed, 14 Mar 2012 00:26:37 +0200 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: <299974FF-76A4-45CF-84DB-F83CF59F0B07@cs.otago.ac.nz> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <299974FF-76A4-45CF-84DB-F83CF59F0B07@cs.otago.ac.nz> Message-ID: On Mar 13, 2012, at 11:59 PM, Richard O'Keefe wrote: >> On unix systems link_count=2 means, that directory is empty: > > No it doesn't. It means that the directory has no *subdirectories*. Mental note to myself - verify before posting;) Sorry for confusion! Yes - thats correct. Link count > 2 means, that there are subdirectories. (On unix, mac os x behaves differently). Richard's opendir/readir C code is only reasonably portable way. Similar implementation for windows with _findfirst/_findnext is at http://www.two-sdg.demon.co.uk/curbralan/code/dirent/dirent.html C-code and port/NIF are way to go. best regards, taavi -- "We should forget about small efficiencies, about 97 percent of the time. Premature optimization is the root of all evil." Donald Knuth Taavi Talvik taavi@REDACTED +372 5656 9996 From jared.nance@REDACTED Wed Mar 14 00:43:47 2012 From: jared.nance@REDACTED (Jared Kofron) Date: Tue, 13 Mar 2012 16:43:47 -0700 Subject: [erlang-questions] Erlang meets physics In-Reply-To: <1331666426.33915.YahooMailNeo@web112614.mail.gq1.yahoo.com> References: <1331666426.33915.YahooMailNeo@web112614.mail.gq1.yahoo.com> Message-ID: Hi Pablo- My first project was on embedded hardware, and basically consisted of NIFs and Webmachine dispatching. Pretty fun stuff. My current project is a little more involved, but is also pretty interesting: Right now the bus over which communication takes place is abstracted away by having hardware modules which translate API functions into their appropriate wire representation and then transmit those representations over a handle that they have to the correct bus. In essence what I've done is taken instruments and buses and given them something like behaviors. Hardware can perform read,write,or configure, for example. So if I want to read the center frequency of a sweeper, I might say hp8340b:read(high_frequency_sweeper,<<"cw_freq">>). which the device module (in this case hp8340b) translates into the GPIB command "OPCW", and based on the device address (which is governed by the atom in the first argument), dispatches it over a bus handle that it owns in a state variable via bus:send_query/2. At the moment, we only communicate with things over GPIB via ethernet using prologix devices to do the translation for us - basically glorified telnet - but it gets the job done. Everything is in a very alpha stage right now for this project, but it is working really nicely so far. JK On Mar 13, 2012, at 12:20 PM, Pablo Platt wrote: > How do you interact with the hardware? > Do you use GPIB C libr and wrap it with a NIF? > > From: Joe Armstrong > To: Jared Kofron > Cc: Erlang Questions > Sent: Tuesday, March 13, 2012 12:34 PM > Subject: Re: [erlang-questions] Erlang meets physics > > Great news - spread the word ! > > Just for the record Erlang programmers numbers 1 and 2 (ie myself and > Robert Virding) > are both ex physicists. > > When I lecture I often point out the similarity between causality and > message reception. > You don't know that something has happened until you get a message > telling that it has happened. > > (In physics it's a ray of light, or a photon, or something - > forgetting entanglement for the moment) > > In computing it's the reception of a message. > > As a ex physicist I know that we can't say anything about simultaneous > events occurring > at different places in space-time - turn this into computer science > and the same arguments > apply to things like making sure replicated data is consistent on > remote sites - well you can't > - at least if you want to change it - Brewer's CAP theorem applies - > which for a physicist makes > perfect sense. > > Also as an ex physicist I realize that things do actually happen in > parallel in the real world, > so modelling them in a sequential programming language (if I wanted to do that) > is big time crazy - just describe the parallel stuff in a concurrent > language and the program > writes itself. Wait a few years till we have million core computers > and the parallel problems > can be solved 1:1 on parallel computers - and programming simulations > and so on will be > really easy - but don't even think about doing it in a sequential language... > > Cheers > > /Joe > > > On Mon, Mar 12, 2012 at 2:34 AM, Jared Kofron wrote: > > Hi All, > > I've been using Erlang at work for a few years now, and I thought I'd throw my experience out there, as > > my application is a little different than what you usually see on the list - I am a graduate student at the > > Center for Nuclear Physics and Astrophysics at the University of Washington, and use Erlang extensively > > in my work. > > > > In my experience, something that Erlang is really great at but doesn't receive much attention for these days > > is managing and interacting with hardware. In any physics experiment of even modest sizes, you wind up > > having to keep track of the state of various pieces of equipment, often modify that state, and constantly > > interrogate particular values. For example, we might want to change the current in a magnetic trap, turn > > that trap off altogether, or simply read back the voltage drop across our superconducting magnet. > > > > So far, I have deployed Erlang in this zone for two separate experiments (SNO+, a large particle physics > > experiment in Canada) and Project 8 (a small nuclear physics experiment here in Seattle). Both times have > > been great successes, and I have found the reception of Erlang in this market to be great. In general, what > > I have done is wrap a hardware management layer with some kind of outside world interface. For SNO+, we > > used Webmachine and RESTful control, and for Project 8 we actually conduct all communication > > by using CouchDB as a message passing interface. > > > > Physicists are suspicious creatures, but once you demonstrate the feature set that you get for practically > > free with OTP, they see the advantage pretty quickly. On top of that, the development cycle for sophisticated > > applications can be greatly reduced - more than once it made my group float to the top in terms of meeting > > goals. > > > > In short, as far as I am concerned, Erlang has found a new niche in the world of Physics, and I intend to > > spread the word as much as I can! > > > > Jared Kofron > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jared.nance@REDACTED Wed Mar 14 00:47:43 2012 From: jared.nance@REDACTED (Jared Kofron) Date: Tue, 13 Mar 2012 16:47:43 -0700 Subject: [erlang-questions] Erlang meets physics In-Reply-To: References: Message-ID: <3ACF868A-3822-4D37-BC42-4E154482FCE4@gmail.com> Hi Joe- What a surprise! I had no idea that you and Robert V were both recovered physicists. I absolutely agree with you re: concurrency in erlang. I find handling the concurrent aspect of my programs to be essentially trivial in erlang, especially once you leverage the power of supervision trees. JK On Mar 13, 2012, at 3:34 AM, Joe Armstrong wrote: > Great news - spread the word ! > > Just for the record Erlang programmers numbers 1 and 2 (ie myself and > Robert Virding) > are both ex physicists. > > When I lecture I often point out the similarity between causality and > message reception. > You don't know that something has happened until you get a message > telling that it has happened. > > (In physics it's a ray of light, or a photon, or something - > forgetting entanglement for the moment) > > In computing it's the reception of a message. > > As a ex physicist I know that we can't say anything about simultaneous > events occurring > at different places in space-time - turn this into computer science > and the same arguments > apply to things like making sure replicated data is consistent on > remote sites - well you can't > - at least if you want to change it - Brewer's CAP theorem applies - > which for a physicist makes > perfect sense. > > Also as an ex physicist I realize that things do actually happen in > parallel in the real world, > so modelling them in a sequential programming language (if I wanted to do that) > is big time crazy - just describe the parallel stuff in a concurrent > language and the program > writes itself. Wait a few years till we have million core computers > and the parallel problems > can be solved 1:1 on parallel computers - and programming simulations > and so on will be > really easy - but don't even think about doing it in a sequential language... > > Cheers > > /Joe > > > On Mon, Mar 12, 2012 at 2:34 AM, Jared Kofron wrote: >> Hi All, >> I've been using Erlang at work for a few years now, and I thought I'd throw my experience out there, as >> my application is a little different than what you usually see on the list - I am a graduate student at the >> Center for Nuclear Physics and Astrophysics at the University of Washington, and use Erlang extensively >> in my work. >> >> In my experience, something that Erlang is really great at but doesn't receive much attention for these days >> is managing and interacting with hardware. In any physics experiment of even modest sizes, you wind up >> having to keep track of the state of various pieces of equipment, often modify that state, and constantly >> interrogate particular values. For example, we might want to change the current in a magnetic trap, turn >> that trap off altogether, or simply read back the voltage drop across our superconducting magnet. >> >> So far, I have deployed Erlang in this zone for two separate experiments (SNO+, a large particle physics >> experiment in Canada) and Project 8 (a small nuclear physics experiment here in Seattle). Both times have >> been great successes, and I have found the reception of Erlang in this market to be great. In general, what >> I have done is wrap a hardware management layer with some kind of outside world interface. For SNO+, we >> used Webmachine and RESTful control, and for Project 8 we actually conduct all communication >> by using CouchDB as a message passing interface. >> >> Physicists are suspicious creatures, but once you demonstrate the feature set that you get for practically >> free with OTP, they see the advantage pretty quickly. On top of that, the development cycle for sophisticated >> applications can be greatly reduced - more than once it made my group float to the top in terms of meeting >> goals. >> >> In short, as far as I am concerned, Erlang has found a new niche in the world of Physics, and I intend to >> spread the word as much as I can! >> >> Jared Kofron >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From otaibinm@REDACTED Wed Mar 14 01:20:38 2012 From: otaibinm@REDACTED (Naif M. Otaibi) Date: Tue, 13 Mar 2012 20:20:38 -0400 Subject: [erlang-questions] Using eTorrent UPNP implementation Message-ID: Hey all, I have been trying to wrap my head around how upnp is implemented in etorrent as I wish to use the implementation to do NAT-traveral for my application. The thing is, no documentation exists to help me. I've been advised that Edward Wang wrote the code for upnp. If anybody can please shed some light on which upnp modules to import in my application and which functions to call. I looked at the etorrent_upnp_*.erl files and made little sense of them. Best regards, -------------- next part -------------- An HTML attachment was scrubbed... URL: From shahrdad1@REDACTED Wed Mar 14 01:23:42 2012 From: shahrdad1@REDACTED (Shahrdad Shadab) Date: Tue, 13 Mar 2012 20:23:42 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F5FB8BE.8040902@mansionfamily.plus.com> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <4F5FB8BE.8040902@mansionfamily.plus.com> Message-ID: On Tue, Mar 13, 2012 at 5:14 PM, james wrote: > >There are many businesses that invested heavily in the previous > >iteration of MS development infrastructure (COM-driven Visual Studio 6 > >and related tools), and then suddenly had the rug pulled from >underneath > them in 2002 when .Net appeared and they were expected to >rewrite/migrate > much of their code (I worked for such a victim, and >gathered many > now-worthless skills). > > When did COM stop working? When did you have to throw away working COM > code because you want to use CLR? When did an ability to write modular C++ > applications not apply on, say, Linux? > > You're a fashion victim. It can happen to anyone, and probably will, > eventually. > > Suggesting that there was any rug pulling is bizarre; some greener grass > turned up over the fence. Some people will call it progress. > If you are working with or for people who 'expect to migrate' just because > something shiny showed up, that's a problem and it can happen to you with > nearly anything. How long do you think 'cloud' will last? > > James > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > Joe pointed to a very important fact, Java/J2ee is industry (so is c# and .net) but Erlnag is a language. Each time I have to look at a WSDL or XML schema to fix a production bug in a J2ee application is I ask why Erlang shouldn't be industry? Just compare the simplicity and in particular the beauty of distributed Erlang with awkwardness of webservice / JMS communications. Quite frankly folks, they are really ugly! So are their .net siblings. This is not because I love Erlang, I just follow the same sense of beauty that guided mathematicians and theoretical physicists for years when they come up with innovative ideas. As Hardy used to say "There is no place for ugly mathematics". Why IT is missing (or ignoring) such a sense? I don't think what we do is more abstract than pure math (Manifold theory for instance). Maybe because IT is too young but still we need to start sometime from somewhere. Thanks Shahrdad -- Software Architect & Computer Scientist -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Wed Mar 14 01:58:49 2012 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 14 Mar 2012 00:58:49 -0000 (GMT) Subject: [erlang-questions] Erlang meets physics In-Reply-To: <1331666426.33915.YahooMailNeo@web112614.mail.gq1.yahoo.com> Message-ID: We used ports. External hardware was a "process" with which you communicated through messages. The asynchronous model fitted hardware very well, at least the hardware in which we were interested. Robert ----- Original Message ----- > How do you interact with the hardware? > Do you use GPIB C libr and wrap it with a NIF? > From: Joe Armstrong > To: Jared Kofron > Cc: Erlang Questions > Sent: Tuesday, March 13, 2012 12:34 PM > Subject: Re: [erlang-questions] Erlang meets physics > Great news - spread the word ! > Just for the record Erlang programmers numbers 1 and 2 (ie myself and > Robert Virding) > are both ex physicists. > When I lecture I often point out the similarity between causality and > message reception. > You don't know that something has happened until you get a message > telling that it has happened. > (In physics it's a ray of light, or a photon, or something - > forgetting entanglement for the moment) > In computing it's the reception of a message. > As a ex physicist I know that we can't say anything about > simultaneous > events occurring > at different places in space-time - turn this into computer science > and the same arguments > apply to things like making sure replicated data is consistent on > remote sites - well you can't > - at least if you want to change it - Brewer's CAP theorem applies - > which for a physicist makes > perfect sense. > Also as an ex physicist I realize that things do actually happen in > parallel in the real world, > so modelling them in a sequential programming language (if I wanted > to do that) > is big time crazy - just describe the parallel stuff in a concurrent > language and the program > writes itself. Wait a few years till we have million core computers > and the parallel problems > can be solved 1:1 on parallel computers - and programming simulations > and so on will be > really easy - but don't even think about doing it in a sequential > language... > Cheers > /Joe > On Mon, Mar 12, 2012 at 2:34 AM, Jared Kofron < jared.nance@REDACTED > > wrote: > > Hi All, > > I've been using Erlang at work for a few years now, and I thought > > I'd throw my experience out there, as > > my application is a little different than what you usually see on > > the list - I am a graduate student at the > > Center for Nuclear Physics and Astrophysics at the University of > > Washington, and use Erlang extensively > > in my work. > > > > In my experience, something that Erlang is really great at but > > doesn't receive much attention for these days > > is managing and interacting with hardware. In any physics > > experiment of even modest sizes, you wind up > > having to keep track of the state of various pieces of equipment, > > often modify that state, and constantly > > interrogate particular values. For example, we might want to change > > the current in a magnetic trap, turn > > that trap off altogether, or simply read back the voltage drop > > across our superconducting magnet. > > > > So far, I have deployed Erlang in this zone for two separate > > experiments (SNO+, a large particle physics > > experiment in Canada) and Project 8 (a small nuclear physics > > experiment here in Seattle). Both times have > > been great successes, and I have found the reception of Erlang in > > this market to be great. In general, what > > I have done is wrap a hardware management layer with some kind of > > outside world interface. For SNO+, we > > used Webmachine and RESTful control, and for Project 8 we actually > > conduct all communication > > by using CouchDB as a message passing interface. > > > > Physicists are suspicious creatures, but once you demonstrate the > > feature set that you get for practically > > free with OTP, they see the advantage pretty quickly. On top of > > that, the development cycle for sophisticated > > applications can be greatly reduced - more than once it made my > > group float to the top in terms of meeting > > goals. > > > > In short, as far as I am concerned, Erlang has found a new niche in > > the world of Physics, and I intend to > > spread the word as much as I can! > > > > Jared Kofron > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngocdaothanh@REDACTED Wed Mar 14 02:36:37 2012 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Wed, 14 Mar 2012 10:36:37 +0900 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <4F5FB8BE.8040902@mansionfamily.plus.com> Message-ID: The result of Erlang vs Java is obvious. But how about Erlang vs Scala (with Akka, http://akka.io/)? > Joe pointed to a very important fact, Java/J2ee is industry (so is c# and > .net) but Erlnag is a language. > ? Each time I have to look at a WSDL or XML schema to fix a production bug > in a J2ee application is I ask why Erlang shouldn't be industry? Just > compare the simplicity and in particular the beauty of distributed Erlang > with awkwardness of webservice / JMS communications. Quite frankly folks, > they are really ugly! So are their .net siblings. This is not because I love > Erlang, I just follow the same sense of beauty that guided mathematicians > and theoretical physicists for years when they come up with innovative > ideas. As Hardy used to say "There is no place for ugly mathematics". Why IT > is missing (or ignoring) such a sense? I don't think what we do is more > abstract than pure math (Manifold theory for instance). Maybe because IT is > too young but still we need to start sometime from somewhere. > > Thanks > Shahrdad From mfidelman@REDACTED Wed Mar 14 02:38:42 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 13 Mar 2012 21:38:42 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <4F5FB8BE.8040902@mansionfamily.plus.com> Message-ID: <4F5FF6A2.7070401@meetinghouse.net> Maybe this is obvious, but seems to me that running a concurrency engine on top of a JVM just adds a layer of unnecessary overhead. Ngoc Dao wrote: > The result of Erlang vs Java is obvious. > But how about Erlang vs Scala (with Akka, http://akka.io/)? > > >> Joe pointed to a very important fact, Java/J2ee is industry (so is c# and >> .net) but Erlnag is a language. >> Each time I have to look at a WSDL or XML schema to fix a production bug >> in a J2ee application is I ask why Erlang shouldn't be industry? Just >> compare the simplicity and in particular the beauty of distributed Erlang >> with awkwardness of webservice / JMS communications. Quite frankly folks, >> they are really ugly! So are their .net siblings. This is not because I love >> Erlang, I just follow the same sense of beauty that guided mathematicians >> and theoretical physicists for years when they come up with innovative >> ideas. As Hardy used to say "There is no place for ugly mathematics". Why IT >> is missing (or ignoring) such a sense? I don't think what we do is more >> abstract than pure math (Manifold theory for instance). Maybe because IT is >> too young but still we need to start sometime from somewhere. >> >> Thanks >> Shahrdad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From ok@REDACTED Wed Mar 14 02:44:20 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 14 Mar 2012 14:44:20 +1300 Subject: [erlang-questions] Erlang meets physics In-Reply-To: <3ACF868A-3822-4D37-BC42-4E154482FCE4@gmail.com> References: <3ACF868A-3822-4D37-BC42-4E154482FCE4@gmail.com> Message-ID: <6C8DEA82-B1CE-4EBD-90BA-72E79E4A149A@cs.otago.ac.nz> On 14/03/2012, at 12:47 PM, Jared Kofron wrote: > Hi Joe- > What a surprise! I had no idea that you and Robert V were both recovered physicists. My master's degree was in Underwater Acoustics... There must be lots of us about. (Can that be why I think Joe is right so often? Nah, surely not...) From ok@REDACTED Wed Mar 14 03:26:20 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 14 Mar 2012 15:26:20 +1300 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <4F5FB8BE.8040902@mansionfamily.plus.com> Message-ID: On 14/03/2012, at 1:23 PM, Shahrdad Shadab wrote: > Joe pointed to a very important fact, Java/J2ee is industry (so is c# and .net) but Erlnag is a language. > Each time I have to look at a WSDL or XML schema to fix a production bug in a J2ee application is I ask why Erlang shouldn't be industry? Just compare the simplicity and in particular the beauty of distributed Erlang with awkwardness of webservice / JMS communications. Quite frankly folks, they are really ugly! So are their .net siblings. This is not because I love Erlang, I just follow the same sense of beauty that guided mathematicians and theoretical physicists for years when they come up with innovative ideas. As Hardy used to say "There is no place for ugly mathematics". Why IT is missing (or ignoring) such a sense? I don't think what we do is more abstract than pure math (Manifold theory for instance). Maybe because IT is too young but still we need to start sometime from somewhere. http://c2.com/gci/wiki?BeautyIsOurBusiness BeautyIsOurBusiness is for me an empirical observation, not a philosophical position. When a junior programmer tells me that it's ugly but it works, it is very tempting to reply simply: "No it doesn't." --AndersMunch See http://www.amazon.com/Beauty-Our-Business-Birthday-Monographs/dp/0387972994 There is a big difference here between CS and IT. IT is about business, first, last, and all the time. In CS, the aim is to contribute to knowledge. If you have failed to communicate clearly, you haven't (yet) done your job as a computer scientist. IT, however, has short horizons. Pushing complexity from development to maintenance always seems like a good idea when you are doing development. (Of course, when you are doing maintenance, you have another opinion.) I was about to say what my favourite bug is at the moment, then realised I was wrong. My favourite bug *symptom* concerns the T-stick we got yesterday from Telecom New Zealand, an HSUPA USB Stick Model MF 190, made by ZTE Corporation. It's supposed to work with both Mac OS and Windows. Plug it into the side of a MacBook Pro [bug 1: it is too wide, so I _can't_ have my 1.5TB Time Machine drive _and_ the 3G modem plugged in at the same time], the Telecom Connection Manager application starts automatically, click on the "Connect" button in the "Internet" panel, and it crashes. Every single time. Completely reliable failure. How does an Internet connection kit that crashes every time you try to connect to the internet *ever* get through any serious testing? So my *second* favourite bug is in Mono, specifically in System.IO.StringReader.ReadLine(). The default method in System.IO.TextReader.ReadLine() is fine, but in order to go faster, this one basically does - use low level code to look for the end of the line - grab out a substring The problem is that thanks to Windows copying DOS which copied CP/M which copied (DEC?), lines might be terminated by CR+LF; thanks to Unix (which followed the original ASCII-63 suggestion that LF could be a line terminator all by itself), lines might be terminated by LF; and thanks to classic MacOS, lines might be terminated by CR. So the code goes - find the next CR, if any. - find the next LF, if any. - which is earlier? ... The result is that if you have a string of length b containing LF terminators only, you search the whole string for a CR. So if the next line is n characters, it costs O(b) time to find it, not O(n). And that means that reading lines is O(b^2) instead of O(b). All because someone decided to be clever. "It looks ugly, but it works." "Yes, but the performance is ugly too." From ok@REDACTED Wed Mar 14 03:44:06 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 14 Mar 2012 15:44:06 +1300 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <4F5FB8BE.8040902@mansionfamily.plus.com> Message-ID: <621961EC-5F28-41A6-A5E3-18E3ED025798@cs.otago.ac.nz> On 14/03/2012, at 2:36 PM, Ngoc Dao wrote: > The result of Erlang vs Java is obvious. > But how about Erlang vs Scala (with Akka, http://akka.io/)? Akka can extend Scala, which extends Java. As that page shows, it can extend Java directly. But You cannot make a reliable system by *extending* an unreliable system. Scala adds many good, even wonderful, things to Java. But there is still a "Window into Hell" (Griswold's term, used in the SL5 phase of what became the Icon project). Akka adds some great things to Scala and Java. But there is still a "Window into Hell". A good part of Erlang's strength is the things it *CAN'T* do. It's rather like the way people used to praise C++ as a safer C, when in practice C++ just added a vast new range of bugs without making any of the old bugs harder to write. If I _had_ to write code to interoperate with Java in the same address space, I would *certainly* do as much as I could in Scala, and if that involved things Erlang would have been a good choice for, well, hello Akka! Let me put it another way: You cannot get a simple system by adding simplicity to a complex system. From mfidelman@REDACTED Wed Mar 14 04:05:22 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 13 Mar 2012 23:05:22 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <621961EC-5F28-41A6-A5E3-18E3ED025798@cs.otago.ac.nz> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <4F5FB8BE.8040902@mansionfamily.plus.com> <621961EC-5F28-41A6-A5E3-18E3ED025798@cs.otago.ac.nz> Message-ID: <4F600AF2.8040508@meetinghouse.net> Richard O'Keefe wrote: > > If I _had_ to write code to interoperate with Java in the same > address space, I would *certainly* do as much as I could in Scala, > and if that involved things Erlang would have been a good choice > for, well, hello Akka! > > Let me put it another way: > > You cannot get a simple system by adding > simplicity to a complex system. > > So why am I suddenly reminded of Greenspun's 10th law? -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From sunwood360@REDACTED Wed Mar 14 05:36:00 2012 From: sunwood360@REDACTED (envelopes envelopes) Date: Tue, 13 Mar 2012 21:36:00 -0700 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F5FF6A2.7070401@meetinghouse.net> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <4F5FB8BE.8040902@mansionfamily.plus.com> <4F5FF6A2.7070401@meetinghouse.net> Message-ID: According to this blog, Akka seems better /close performance when compared to Erlang. http://letitcrash.com/post/14783691760/akka-vs-erlang ".... Erlang R14B04 did *1 million* messages per second while Akka 2.0-SNAPSHOT did *2.1 million* per second..." this one also has some interesting results: https://plus.google.com/u/0/112820434312193778084/posts/HdKFx4VQtJj On Tue, Mar 13, 2012 at 6:38 PM, Miles Fidelman wrote: > Maybe this is obvious, but seems to me that running a concurrency engine > on top of a JVM just adds a layer of unnecessary overhead. > > > Ngoc Dao wrote: > >> The result of Erlang vs Java is obvious. >> But how about Erlang vs Scala (with Akka, http://akka.io/)? >> >> >> Joe pointed to a very important fact, Java/J2ee is industry (so is c# and >>> .net) but Erlnag is a language. >>> Each time I have to look at a WSDL or XML schema to fix a production >>> bug >>> in a J2ee application is I ask why Erlang shouldn't be industry? Just >>> compare the simplicity and in particular the beauty of distributed Erlang >>> with awkwardness of webservice / JMS communications. Quite frankly folks, >>> they are really ugly! So are their .net siblings. This is not because I >>> love >>> Erlang, I just follow the same sense of beauty that guided mathematicians >>> and theoretical physicists for years when they come up with innovative >>> ideas. As Hardy used to say "There is no place for ugly mathematics". >>> Why IT >>> is missing (or ignoring) such a sense? I don't think what we do is more >>> abstract than pure math (Manifold theory for instance). Maybe because IT >>> is >>> too young but still we need to start sometime from somewhere. >>> >>> Thanks >>> Shahrdad >>> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngocdaothanh@REDACTED Wed Mar 14 05:49:39 2012 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Wed, 14 Mar 2012 13:49:39 +0900 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <4F5FB8BE.8040902@mansionfamily.plus.com> <4F5FF6A2.7070401@meetinghouse.net> Message-ID: This one says it can send 1 billion messages per second: https://github.com/laforge49/JActor Explanation: http://jactor.sourceforge.net/slides/Actors-in-the-Small-22-feb-2012.pdf On Wed, Mar 14, 2012 at 1:36 PM, envelopes envelopes wrote: > According to this blog, Akka seems better /close performance when compared > to Erlang. > > http://letitcrash.com/post/14783691760/akka-vs-erlang > > ".... Erlang R14B04 did 1 million messages per second while Akka > 2.0-SNAPSHOT did 2.1 million per second..." > > this one also has some interesting results: > ?? https://plus.google.com/u/0/112820434312193778084/posts/HdKFx4VQtJj > > > > On Tue, Mar 13, 2012 at 6:38 PM, Miles Fidelman > wrote: >> >> Maybe this is obvious, but seems to me that running a concurrency engine >> on top of a JVM just adds a layer of unnecessary overhead. From bourinov@REDACTED Wed Mar 14 06:21:36 2012 From: bourinov@REDACTED (Max Bourinov) Date: Wed, 14 Mar 2012 09:21:36 +0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <4F5FB8BE.8040902@mansionfamily.plus.com> <4F5FF6A2.7070401@meetinghouse.net> Message-ID: Akka has very flashy statements on its website [1]. I want to mention that some very important aspects (like gc, memory allocation etc.) are not covered on [1]. Yeah, it is JVM and we all know how it will work. Makes no sense to expect something new there :-) So, I believe that 2.1 million messages in Akka [2] and 1 Million in Erlang are absolutely different messages. --- 1. http://akka.io/ 2. http://letitcrash.com/post/14783691760/akka-vs-erlang Best regards, Max On Wed, Mar 14, 2012 at 8:36 AM, envelopes envelopes wrote: > According to this blog, Akka seems better /close performance when compared > to Erlang. > > http://letitcrash.com/post/14783691760/akka-vs-erlang > > ".... Erlang R14B04 did *1 million* messages per second while Akka > 2.0-SNAPSHOT did *2.1 million* per second..." > > this one also has some interesting results: > https://plus.google.com/u/0/112820434312193778084/posts/HdKFx4VQtJj > > > > > On Tue, Mar 13, 2012 at 6:38 PM, Miles Fidelman < > mfidelman@REDACTED> wrote: > >> Maybe this is obvious, but seems to me that running a concurrency engine >> on top of a JVM just adds a layer of unnecessary overhead. >> >> >> Ngoc Dao wrote: >> >>> The result of Erlang vs Java is obvious. >>> But how about Erlang vs Scala (with Akka, http://akka.io/)? >>> >>> >>> Joe pointed to a very important fact, Java/J2ee is industry (so is c# >>>> and >>>> .net) but Erlnag is a language. >>>> Each time I have to look at a WSDL or XML schema to fix a production >>>> bug >>>> in a J2ee application is I ask why Erlang shouldn't be industry? Just >>>> compare the simplicity and in particular the beauty of distributed >>>> Erlang >>>> with awkwardness of webservice / JMS communications. Quite frankly >>>> folks, >>>> they are really ugly! So are their .net siblings. This is not because I >>>> love >>>> Erlang, I just follow the same sense of beauty that guided >>>> mathematicians >>>> and theoretical physicists for years when they come up with innovative >>>> ideas. As Hardy used to say "There is no place for ugly mathematics". >>>> Why IT >>>> is missing (or ignoring) such a sense? I don't think what we do is more >>>> abstract than pure math (Manifold theory for instance). Maybe because >>>> IT is >>>> too young but still we need to start sometime from somewhere. >>>> >>>> Thanks >>>> Shahrdad >>>> >>> ______________________________**_________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/**listinfo/erlang-questions >>> >> >> >> -- >> In theory, there is no difference between theory and practice. >> In practice, there is. .... Yogi Berra >> >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Wed Mar 14 07:22:28 2012 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 14 Mar 2012 07:22:28 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <621961EC-5F28-41A6-A5E3-18E3ED025798@cs.otago.ac.nz> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <4F5FB8BE.8040902@mansionfamily.plus.com> <621961EC-5F28-41A6-A5E3-18E3ED025798@cs.otago.ac.nz> Message-ID: On 14 Mar 2012, at 03:44, Richard O'Keefe wrote: > You cannot make a reliable system by *extending* an > unreliable system. Hmm, I guess that depends on what you read into the word 'extending'. Erlang processes are unreliable, but supervision structures, which build on erlang processes, are more reliable. A system of erlang nodes can be more reliable than a singe node. A pair of computers in a redundant configuration can be more reliable than a single computer. In essense, one *always* builds reliable (complex) systems out of less reliable parts. That said, Erlang has some very nice *fundamental* facilities to enable this, and adding fault-tolerance as an afterthought in other languages is likely as hard as adding strong static type checking to Erlang - that is, it can be done, and has been done to quite some extent, but those who have worked in a language where it was envisioned from the start will argue that it's a pale copy of the Real Thing. BR, Ulf W -------------- next part -------------- An HTML attachment was scrubbed... URL: From zerthurd@REDACTED Wed Mar 14 07:25:04 2012 From: zerthurd@REDACTED (Maxim Treskin) Date: Wed, 14 Mar 2012 13:25:04 +0700 Subject: [erlang-questions] Rebar, common_test and cover issue In-Reply-To: References: Message-ID: Thank you for reply, it is really root of the problem. I forgot mention about library meck which loads module and redefines some of its functions. When I remove meck, coverage works as expected. I should think how to replace meck. On 13 March 2012 19:58, Siri Hansen wrote: > This is not related to the name of the SUITEs. What happens is that the > modules (my_module1 and my_module2 - and possible other modules as derived > from your cover spec file) are cover compiled and loaded by common_test > before the test starts. The result of the cover compilation exists only in > memory so if the test code explicitly loads these modules again the > original beam files will be used and the cover compiled version disappears. > The only way to overcome this is to avoid loading of code in your tests. > /siri > > Den 10:55 13. mars 2012 skrev Maxim Treskin f?lgende: > > Yes, this modules loaded in tests. How I can fix this? Rename SUIT modules? >> >> >> On 13 March 2012 16:38, Siri Hansen wrote: >> >>> This would typically happen if your test loads my_module1 and >>> my_module2. Could that be the case? >>> /siri >>> >>> Den 10:30 13. mars 2012 skrev Maxim Treskin f?lgende: >>> >>>> Hello >>>> >>>> I have strange behaviour of rebar+common_test+cover. >>>> There is two files in ./test/: >>>> >>>> my_module1_SUITE.erl >>>> my_module2_SUITE.erl >>>> >>>> where ct suites defined. When I run `rebar ct verbose=1` I see: >>>> >>>> ``` >>>> ... >>>> Cover compiling 'myapp' (16 files) - this may take some time... done >>>> ... >>>> Cover analysing... >>>> WARNING: Analysis failed for my_module1. Reason: >>>> {error,{not_cover_compiled,my_module1}} >>>> WARNING: Analysis failed for my_module2. Reason: >>>> {error,{not_cover_compiled,my_module2}} >>>> ``` >>>> >>>> Why I see no any error before at cover compilation step? >>>> >>>> Moreover, if I just replace my_module1_SUITE:all/0 to return just empty >>>> list [], warning "Analysis failed for my_module1" disappears, and cover for >>>> this file appears in cover.html log. >>>> >>>> Can you explain me this strange issues? Is this any bug or expected >>>> behaviour? >>>> >>>> Thank you. >>>> >>>> -- >>>> Maxim Treskin >>>> http://metachord.com >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> >> >> -- >> Maxim Treskin >> > > -- Maxim Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Wed Mar 14 07:33:32 2012 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 14 Mar 2012 07:33:32 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <4F5FB8BE.8040902@mansionfamily.plus.com> <4F5FF6A2.7070401@meetinghouse.net> Message-ID: <7CD83170-AF2D-4A7A-9372-F38B77CDA77F@feuerlabs.com> On 14 Mar 2012, at 06:21, Max Bourinov wrote: > So, I believe that 2.1 million messages in Akka [2] and 1 Million in Erlang are absolutely different messages. Perhaps this blog article can help illustrate some of the differences: http://blog.typesafe.com/non-blocking-message-flow-with-akka-actors BR, Ulf W -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Wed Mar 14 07:46:17 2012 From: bourinov@REDACTED (Max Bourinov) Date: Wed, 14 Mar 2012 09:46:17 +0300 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <7CD83170-AF2D-4A7A-9372-F38B77CDA77F@feuerlabs.com> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <4F5FB8BE.8040902@mansionfamily.plus.com> <4F5FF6A2.7070401@meetinghouse.net> <7CD83170-AF2D-4A7A-9372-F38B77CDA77F@feuerlabs.com> Message-ID: Ulf, thank you for the link! Erlang is way better for me :-) Especially when I think about meeting deadlines... Best regards, Max On Wed, Mar 14, 2012 at 9:33 AM, Ulf Wiger wrote: > > On 14 Mar 2012, at 06:21, Max Bourinov wrote: > > So, I believe that 2.1 million messages in Akka [2] and 1 Million in > Erlang are absolutely different messages. > > > Perhaps this blog article can help illustrate some of the differences: > > http://blog.typesafe.com/non-blocking-message-flow-with-akka-actors > > BR, > Ulf W > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Wed Mar 14 08:31:14 2012 From: lukas@REDACTED (Lukas Larsson) Date: Wed, 14 Mar 2012 08:31:14 +0100 Subject: [erlang-questions] Rebar, common_test and cover issue In-Reply-To: References: Message-ID: Hi, meck should retain the cover info if used properly. Also make sure you have the latest version. I think some work has been done with cover recently. Lukas On Mar 14, 2012 7:25 AM, "Maxim Treskin" wrote: > Thank you for reply, it is really root of the problem. > I forgot mention about library meck which loads module and redefines some > of its functions. When I remove meck, coverage works as expected. I should > think how to replace meck. > > On 13 March 2012 19:58, Siri Hansen wrote: > >> This is not related to the name of the SUITEs. What happens is that the >> modules (my_module1 and my_module2 - and possible other modules as derived >> from your cover spec file) are cover compiled and loaded by common_test >> before the test starts. The result of the cover compilation exists only in >> memory so if the test code explicitly loads these modules again the >> original beam files will be used and the cover compiled version disappears. >> The only way to overcome this is to avoid loading of code in your tests. >> /siri >> >> Den 10:55 13. mars 2012 skrev Maxim Treskin f?lgende: >> >> Yes, this modules loaded in tests. How I can fix this? Rename SUIT >>> modules? >>> >>> >>> On 13 March 2012 16:38, Siri Hansen wrote: >>> >>>> This would typically happen if your test loads my_module1 and >>>> my_module2. Could that be the case? >>>> /siri >>>> >>>> Den 10:30 13. mars 2012 skrev Maxim Treskin f?lgende: >>>> >>>>> Hello >>>>> >>>>> I have strange behaviour of rebar+common_test+cover. >>>>> There is two files in ./test/: >>>>> >>>>> my_module1_SUITE.erl >>>>> my_module2_SUITE.erl >>>>> >>>>> where ct suites defined. When I run `rebar ct verbose=1` I see: >>>>> >>>>> ``` >>>>> ... >>>>> Cover compiling 'myapp' (16 files) - this may take some time... done >>>>> ... >>>>> Cover analysing... >>>>> WARNING: Analysis failed for my_module1. Reason: >>>>> {error,{not_cover_compiled,my_module1}} >>>>> WARNING: Analysis failed for my_module2. Reason: >>>>> {error,{not_cover_compiled,my_module2}} >>>>> ``` >>>>> >>>>> Why I see no any error before at cover compilation step? >>>>> >>>>> Moreover, if I just replace my_module1_SUITE:all/0 to return just >>>>> empty list [], warning "Analysis failed for my_module1" disappears, and >>>>> cover for this file appears in cover.html log. >>>>> >>>>> Can you explain me this strange issues? Is this any bug or expected >>>>> behaviour? >>>>> >>>>> Thank you. >>>>> >>>>> -- >>>>> Maxim Treskin >>>>> http://metachord.com >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>> >>> >>> -- >>> Maxim Treskin >>> >> >> > > > -- > Maxim Treskin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Wed Mar 14 09:07:59 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 14 Mar 2012 09:07:59 +0100 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <299974FF-76A4-45CF-84DB-F83CF59F0B07@cs.otago.ac.nz> Message-ID: <1E18980F-B04E-4710-8494-77BB1096FAE9@gmail.com> Yes and no. When testing Taavi's solution on OSX, the "link_count" takes into account both files and subdirs. But on Linux, it only counts subdirs. Regards, Zabrane On Mar 13, 2012, at 11:26 PM, Taavi Talvik wrote: > > On Mar 13, 2012, at 11:59 PM, Richard O'Keefe wrote: >>> On unix systems link_count=2 means, that directory is empty: >> >> No it doesn't. It means that the directory has no *subdirectories*. > > Mental note to myself - verify before posting;) > Sorry for confusion! > > Yes - thats correct. Link count > 2 means, that there are subdirectories. > (On unix, mac os x behaves differently). > > Richard's opendir/readir C code is only reasonably portable way. > Similar implementation for windows with _findfirst/_findnext is at > http://www.two-sdg.demon.co.uk/curbralan/code/dirent/dirent.html > > C-code and port/NIF are way to go. > > best regards, > taavi > -- > "We should forget about small efficiencies, about 97 percent of the time. > Premature optimization is the root of all evil." Donald Knuth > > Taavi Talvik > taavi@REDACTED > +372 5656 9996 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From alfredo.dinapoli@REDACTED Wed Mar 14 09:09:32 2012 From: alfredo.dinapoli@REDACTED (Alfredo Di Napoli) Date: Wed, 14 Mar 2012 09:09:32 +0100 Subject: [erlang-questions] OpenCL.framework cause my NIF library to crash In-Reply-To: References: <1331653757763-4469412.post@n4.nabble.com> Message-ID: On 13/mar/2012, at 17:54, Bob Ippolito wrote: > Do you also have the -framework OpenCL flag? Static libraries don't automatically link in their dynamic dependencies. Yes I've compiled with this flag both my static framework library as well as my shared NIF, but the problem still persist. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngocdaothanh@REDACTED Wed Mar 14 09:17:11 2012 From: ngocdaothanh@REDACTED (ngocdaothanh) Date: Wed, 14 Mar 2012 01:17:11 -0700 (PDT) Subject: [erlang-questions] Erlang: Bring cloud to your application Message-ID: <93bb94b8-30b9-4e4c-a732-aeda513646d3@qb4g2000pbb.googlegroups.com> Hi, I've just created this presentation: http://www.slideshare.net/ngocdaothanh/cloud-erlang Erlang is impressive but my words are not good enough to express that. Please comment to help me improve the presentation. Thanks, Ngoc From michael.eugene.turner@REDACTED Wed Mar 14 10:02:49 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 14 Mar 2012 18:02:49 +0900 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <4F5FB8BE.8040902@mansionfamily.plus.com> Message-ID: "In CS, the aim is to contribute to knowledge. If you have failed to communicate clearly, you haven't (yet) done your job as a computer scientist. IT, however, has short horizons." To complete Richard's thought: "... If you have failed to communicate clearly, your IT job is that much safer." Up to a point, anyway. ;-) -michael turner On Wed, Mar 14, 2012 at 11:26 AM, Richard O'Keefe wrote: > > On 14/03/2012, at 1:23 PM, Shahrdad Shadab wrote: >> Joe pointed to a very important fact, Java/J2ee is industry (so is c# and .net) but Erlnag is a language. >> ? Each time I have to look at a WSDL or XML schema to fix a production bug in a J2ee application is I ask why Erlang shouldn't be industry? Just compare the simplicity and in particular the beauty of distributed Erlang with awkwardness of webservice / JMS communications. Quite frankly folks, they are really ugly! So are their .net siblings. This is not because I love Erlang, I just follow the same sense of beauty that guided mathematicians and theoretical physicists for years when they come up with innovative ideas. As Hardy used to say "There is no place for ugly mathematics". Why IT is missing (or ignoring) such a sense? I don't think what we do is more abstract than pure math (Manifold theory for instance). Maybe because IT is too young but still we need to start sometime from somewhere. > > > http://c2.com/gci/wiki?BeautyIsOurBusiness > > ? ? ? ?BeautyIsOurBusiness is for me an empirical observation, > ? ? ? ?not a philosophical position. > ? ? ? ?When a junior programmer tells me that it's ugly but it works, > ? ? ? ?it is very tempting to reply simply: "No it doesn't." > ? ? ? ?--AndersMunch > > See http://www.amazon.com/Beauty-Our-Business-Birthday-Monographs/dp/0387972994 > > There is a big difference here between CS and IT. ?IT is about business, first, last, > and all the time. ?In CS, the aim is to contribute to knowledge. ?If you have failed > to communicate clearly, you haven't (yet) done your job as a computer scientist. ?IT, > however, has short horizons. ?Pushing complexity from development to maintenance > always seems like a good idea when you are doing development. ?(Of course, when you > are doing maintenance, you have another opinion.) > > I was about to say what my favourite bug is at the moment, then realised I was > wrong. ?My favourite bug *symptom* concerns the T-stick we got yesterday from > Telecom New Zealand, an HSUPA USB Stick Model MF 190, made by ZTE Corporation. > It's supposed to work with both Mac OS and Windows. ?Plug it into the side of a > MacBook Pro [bug 1: it is too wide, so I _can't_ have my 1.5TB Time Machine > drive _and_ the 3G modem plugged in at the same time], the Telecom Connection > Manager application starts automatically, click on the "Connect" button in the > "Internet" panel, and it crashes. ?Every single time. ?Completely reliable failure. > How does an Internet connection kit that crashes every time you try to connect > to the internet *ever* get through any serious testing? > > So my *second* favourite bug is in Mono, specifically in > System.IO.StringReader.ReadLine(). > The default method in System.IO.TextReader.ReadLine() is fine, > but in order to go faster, this one basically does > ? ? ? ?- use low level code to look for the end of the line > ? ? ? ?- grab out a substring > The problem is that thanks to Windows copying DOS which copied CP/M > which copied (DEC?), lines might be terminated by CR+LF; thanks to > Unix (which followed the original ASCII-63 suggestion that LF could > be a line terminator all by itself), lines might be terminated by LF; > and thanks to classic MacOS, lines might be terminated by CR. > So the code goes > ? ? ? ?- find the next CR, if any. > ? ? ? ?- find the next LF, if any. > ? ? ? ?- which is earlier? > ? ? ? ?... > The result is that if you have a string of length b containing LF terminators > only, you search the whole string for a CR. ?So if the next line is n characters, > it costs O(b) time to find it, not O(n). ?And that means that reading lines is > O(b^2) instead of O(b). ?All because someone decided to be clever. > "It looks ugly, but it works." ?"Yes, but the performance is ugly too." > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.eugene.turner@REDACTED Wed Mar 14 10:06:55 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 14 Mar 2012 18:06:55 +0900 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> <58A57DF4-4A3E-40AB-95E9-30B140071785@gmail.com> Message-ID: > There is also filelib:file_size Which also always returns zero on Winblows, regardless of directory content. -michael turner On Wed, Mar 14, 2012 at 12:57 AM, Wes James wrote: > There is also filelib:file_size > > Another thing to look at. ?Can you keep track of when the folder is > first created: > > ?#file_info.atime (keep somwhere like ets table) > > then later read #file_info.atime and compare? > > The problem here is that when a file file is written or the folder is > just plane read (I did ls of folder and the atime changed) the atime > changes. > > Will the folder ever be accessed other than to have a file be written > to it, if so, this may work. > > -wes > > On Tue, Mar 13, 2012 at 9:34 AM, Zabrane Mickael wrote: >> >> On Mar 13, 2012, at 4:33 PM, Wes James wrote: >> >>> Per Bengt's experience. ?What happens if you add a file? ?Does the size change? >> >> Nothing change when I added a file. >> >> Regards, >> Zabrane >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.eugene.turner@REDACTED Wed Mar 14 10:12:49 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 14 Mar 2012 18:12:49 +0900 Subject: [erlang-questions] Best way to check if a given directory is empty (or not)? In-Reply-To: References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <92BBDDD3-C2DD-4588-A4DB-EE747F14D4C9@cs.otago.ac.nz> Message-ID: > On unix systems link_count=2 means, that directory is empty On Windows, I'm getting link_count = 1 no matter what's in the directory. Could be a bug, though, I guess. -michael turner On Wed, Mar 14, 2012 at 1:01 AM, Taavi Talvik wrote: > On Mar 13, 2012, at 5:18 PM, Wes James wrote: > >> On Tue, Mar 13, 2012 at 12:09 AM, Zabrane Mickael wrote: >>> Hi guys, >>> >>> I'd like to know if there's a good and portable method to check if a directory is empty (or not)? >>> In my case, the directory can contains huge number (more than 150K in average) of files if it is not empty. > > > On unix systems link_count=2 means, that directory is empty: > > 3> rr("/usr/local/lib/erlang/lib/kernel-2.14.1/include/file.hrl"). > [file_descriptor,file_info] > 4> file:read_file_info("."). > {ok,#file_info{size = 170,type = directory, > ? ? ? ? ? ? ? access = read_write, > ? ? ? ? ? ? ? atime = {{2012,3,13},{17,57,1}}, > ? ? ? ? ? ? ? mtime = {{2007,9,30},{14,7,32}}, > ? ? ? ? ? ? ? ctime = {{2012,3,11},{3,6,39}}, > ? ? ? ? ? ? ? mode = 16877,links = 5,major_device = 234881026, > ? ? ? ? ? ? ? minor_device = 0,inode = 43374422,uid = 501,gid = 501}} > > 2> file:read_file_info("empty"). > {ok,#file_info{size = 68,type = directory, > ? ? ? ? ? ? ? access = read_write, > ? ? ? ? ? ? ? atime = {{2012,3,13},{17,59,44}}, > ? ? ? ? ? ? ? mtime = {{2012,3,13},{17,59,44}}, > ? ? ? ? ? ? ? ctime = {{2012,3,13},{17,59,44}}, > ? ? ? ? ? ? ? mode = 16877,links = 2,major_device = 234881026, > ? ? ? ? ? ? ? minor_device = 0,inode = 43401684,uid = 501,gid = 501}} > > best regards, > taavi > -- > "We should forget about small efficiencies, about 97 percent of the time. > Premature optimization is the root of all evil." Donald Knuth > > Taavi Talvik > taavi@REDACTED > +372 5656 9996 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zerthurd@REDACTED Wed Mar 14 10:33:07 2012 From: zerthurd@REDACTED (Maxim Treskin) Date: Wed, 14 Mar 2012 16:33:07 +0700 Subject: [erlang-questions] Rebar, common_test and cover issue In-Reply-To: References: Message-ID: Yes, I use latest version of meck from github On 14 March 2012 14:31, Lukas Larsson wrote: > Hi, meck should retain the cover info if used properly. Also make sure you > have the latest version. I think some work has been done with cover > recently. > > Lukas > On Mar 14, 2012 7:25 AM, "Maxim Treskin" wrote: > >> Thank you for reply, it is really root of the problem. >> I forgot mention about library meck which loads module and redefines some >> of its functions. When I remove meck, coverage works as expected. I should >> think how to replace meck. >> >> On 13 March 2012 19:58, Siri Hansen wrote: >> >>> This is not related to the name of the SUITEs. What happens is that the >>> modules (my_module1 and my_module2 - and possible other modules as derived >>> from your cover spec file) are cover compiled and loaded by common_test >>> before the test starts. The result of the cover compilation exists only in >>> memory so if the test code explicitly loads these modules again the >>> original beam files will be used and the cover compiled version disappears. >>> The only way to overcome this is to avoid loading of code in your tests. >>> /siri >>> >>> Den 10:55 13. mars 2012 skrev Maxim Treskin f?lgende: >>> >>> Yes, this modules loaded in tests. How I can fix this? Rename SUIT >>>> modules? >>>> >>>> >>>> On 13 March 2012 16:38, Siri Hansen wrote: >>>> >>>>> This would typically happen if your test loads my_module1 and >>>>> my_module2. Could that be the case? >>>>> /siri >>>>> >>>>> Den 10:30 13. mars 2012 skrev Maxim Treskin f?lgende: >>>>> >>>>>> Hello >>>>>> >>>>>> I have strange behaviour of rebar+common_test+cover. >>>>>> There is two files in ./test/: >>>>>> >>>>>> my_module1_SUITE.erl >>>>>> my_module2_SUITE.erl >>>>>> >>>>>> where ct suites defined. When I run `rebar ct verbose=1` I see: >>>>>> >>>>>> ``` >>>>>> ... >>>>>> Cover compiling 'myapp' (16 files) - this may take some time... done >>>>>> ... >>>>>> Cover analysing... >>>>>> WARNING: Analysis failed for my_module1. Reason: >>>>>> {error,{not_cover_compiled,my_module1}} >>>>>> WARNING: Analysis failed for my_module2. Reason: >>>>>> {error,{not_cover_compiled,my_module2}} >>>>>> ``` >>>>>> >>>>>> Why I see no any error before at cover compilation step? >>>>>> >>>>>> Moreover, if I just replace my_module1_SUITE:all/0 to return just >>>>>> empty list [], warning "Analysis failed for my_module1" disappears, and >>>>>> cover for this file appears in cover.html log. >>>>>> >>>>>> Can you explain me this strange issues? Is this any bug or expected >>>>>> behaviour? >>>>>> >>>>>> Thank you. >>>>>> >>>>>> -- >>>>>> Maxim Treskin >>>>>> http://metachord.com >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Maxim Treskin >>>> >>> >>> >> >> >> -- >> Maxim Treskin >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> -- Maxim Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From janburse@REDACTED Wed Mar 14 11:06:08 2012 From: janburse@REDACTED (Jan Burse) Date: Wed, 14 Mar 2012 11:06:08 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: Message-ID: <4F606D90.5070206@fastmail.fm> Shahrdad Shadab schrieb: > _Java performance is nowhere near Erlang. Where is the evidence? I don't know whether Joe would approve with this statement? Did a little micro benchmark: Erlang is slower than Java. For general purpose programming. Did not test some message passing or so. Here are some results for Naive Reverse: Erlang 64-bit R15B: ~60 ms Java 64-bit JDK 1.7: ~30 ms (Turbo boost ~15ms) Tested on Windows 7, Sony VAIO VPC-SA3J1E/XI, CPU i5 2430M, 4GB RAM ------------------ benchmark.erl -------------------------- -module(benchmark). -export([test/0,nrev/0]). append([],X) -> X; append([X|Y],Z) -> [X|append(Y,Z)]. reverse([]) -> []; reverse([X|Y]) -> append(reverse(Y),[X]). data() -> [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20, 21,22,23,24,25,26,27,28,29,30]. nrev() -> reverse(data()). for(0,F) -> true; for(N,F) when N>0 -> apply(benchmark,F,[]), for(N-1,F). bench(N,F) -> {_,_,X}=os:timestamp(), for(N,F), {_,_,Y}=os:timestamp(), io:write(F), io:put_chars("\tin "), io:write(Y-X), io:put_chars(" (mics)"), io:nl(). test() -> bench(6000,nrev). ---------------------- ConsCell.java --------------------------- package demo.perpetual; /** */ final class ConsCell { private final Object head; private final Object tail; ConsCell(Object h, Object t) { head = h; tail = t; } Object getHead() { return head; } Object getTail() { return tail; } } ---------------------- NaiveReverse.java ----------------------------- package demo.perpetual; /** */ public final class NaiveReverse { private static Object data; static { Object help = null; for (int i = 30; i >= 1; i--) { help = new ConsCell(Integer.valueOf(i), help); } data = help; } private static Object append(Object alfa, Object beta) { if (alfa == null) { return beta; } else { ConsCell help = (ConsCell) alfa; return new ConsCell(help.getHead(), append(help.getTail(), beta)); } } private static Object reverse(Object alfa) { if (alfa == null) { return null; } else { ConsCell help = (ConsCell) alfa; return append(reverse(help.getTail()), new ConsCell(help.getHead(), null)); } } private static void _for(int n) { for (int i=0; i References: Message-ID: Hi Naif, Edward here. To use etorrent upnp implementation, you need to include following source files (https://github.com/edwardw/etorrent/tree/master/apps/etorrent/src): etorrent_upnp_sup.erl etorrent_upnp_proto.erl etorrent_upnp_net.erl etorrent_upnp_handler.erl etorrent_upnp_entity.erl And also: - add its dependencies, cowboy and lhttpc (https://github.com/jlouis/etorrent/blob/master/rebar.config#L17) - add it to your app.src (https://github.com/jlouis/etorrent/blob/master/apps/etorrent/src/etorrent.app.src#L29) - start upnp supervisor when your application starts (https://github.com/jlouis/etorrent/blob/master/apps/etorrent/src/etorrent_sup.erl#L74). Once started, the upnp supervisor will take care of the first step of upnp protocol, i.e. discovery. - then you are ready to call whatever upnp actions out there. Currently etorrent upnp implements one of them, AddPortMapping, as defined in following specification: http://upnp.org/specs/gw/UPnP-gw-WANIPConnection-v1-Service.pdf It is also what you are looking for, I believe. The erlang implementation for it is here: https://github.com/jlouis/etorrent/blob/master/apps/etorrent/src/etorrent_upnp_net.erl#L153 It is very straightforward to implement other upnp actions modeling after this. Hope this helps. Fell free to ask questions if you have any. Regards, Edward On Wed, Mar 14, 2012 at 8:20 AM, Naif M. Otaibi wrote: > Hey all, > > I have been trying to wrap my head around how upnp is implemented in > etorrent as I wish to use the implementation to do NAT-traveral for my > application. The thing is, no documentation exists to help me. > > I've been advised that Edward Wang wrote the code for upnp. If anybody can > please shed some light on which upnp modules to import in my application and > which functions to call. I looked at the etorrent_upnp_*.erl files and made > little sense of them. > > Best regards, > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From edward.yu.wang@REDACTED Wed Mar 14 11:29:00 2012 From: edward.yu.wang@REDACTED (Edward Wang) Date: Wed, 14 Mar 2012 18:29:00 +0800 Subject: [erlang-questions] Using eTorrent UPNP implementation In-Reply-To: References: Message-ID: One more thing. The etorrent upnp implementation adds port mappings automatically: https://github.com/jlouis/etorrent/blob/master/apps/etorrent/src/etorrent_upnp_entity.erl#L102 You may want to do that differently. On Wed, Mar 14, 2012 at 8:20 AM, Naif M. Otaibi wrote: > Hey all, > > I have been trying to wrap my head around how upnp is implemented in > etorrent as I wish to use the implementation to do NAT-traveral for my > application. The thing is, no documentation exists to help me. > > I've been advised that Edward Wang wrote the code for upnp. If anybody can > please shed some light on which upnp modules to import in my application and > which functions to call. I looked at the etorrent_upnp_*.erl files and made > little sense of them. > > Best regards, > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From michael.eugene.turner@REDACTED Wed Mar 14 14:17:57 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 14 Mar 2012 22:17:57 +0900 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F606D90.5070206@fastmail.fm> References: <4F606D90.5070206@fastmail.fm> Message-ID: There must be some corollary of Godwin's Law saying that all discussions of programming languages, if they go on long enough, must eventually degenerate into disagreements over either benchmark results or open source licenses. ;-) -michael turner On Wed, Mar 14, 2012 at 7:06 PM, Jan Burse wrote: > Shahrdad Shadab schrieb: >> >> _Java performance is nowhere near Erlang. > > > Where is the evidence? I don't know > whether Joe would approve with this > statement? > > Did a little micro benchmark: Erlang is > slower than Java. For general purpose > programming. Did not test some message > passing or so. > > Here are some results for Naive Reverse: > > Erlang 64-bit R15B: ~60 ms > Java 64-bit JDK 1.7: ~30 ms (Turbo boost ~15ms) > > Tested on Windows 7, > Sony VAIO VPC-SA3J1E/XI, > CPU i5 2430M, 4GB RAM > > ------------------ benchmark.erl -------------------------- > > -module(benchmark). > -export([test/0,nrev/0]). > > append([],X) -> X; > append([X|Y],Z) -> [X|append(Y,Z)]. > > reverse([]) -> []; > reverse([X|Y]) -> append(reverse(Y),[X]). > > data() -> [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20, > 21,22,23,24,25,26,27,28,29,30]. > > nrev() -> reverse(data()). > > for(0,F) -> true; > for(N,F) when N>0 -> apply(benchmark,F,[]), for(N-1,F). > > bench(N,F) -> > {_,_,X}=os:timestamp(), > for(N,F), > {_,_,Y}=os:timestamp(), > io:write(F), > io:put_chars("\tin "), > io:write(Y-X), > io:put_chars(" (mics)"), > io:nl(). > > test() -> bench(6000,nrev). > > ---------------------- ConsCell.java --------------------------- > > package demo.perpetual; > > /** > ?*/ > final class ConsCell { > ? ?private final Object head; > ? ?private final Object tail; > > ? ?ConsCell(Object h, Object t) { > ? ? ? ?head = h; > ? ? ? ?tail = t; > ? ?} > > ? ?Object getHead() { > ? ? ? ?return head; > ? ?} > > ? ?Object getTail() { > ? ? ? ?return tail; > ? ?} > > } > > ---------------------- NaiveReverse.java ----------------------------- > > package demo.perpetual; > > /** > ?*/ > public final class NaiveReverse { > ? ?private static Object data; > > ? ?static { > ? ? ? ?Object help = null; > ? ? ? ?for (int i = 30; i >= 1; i--) { > ? ? ? ? ? ?help = new ConsCell(Integer.valueOf(i), help); > ? ? ? ?} > ? ? ? ?data = help; > ? ?} > > ? ?private static Object append(Object alfa, Object beta) { > ? ? ? ?if (alfa == null) { > ? ? ? ? ? ?return beta; > ? ? ? ?} else { > ? ? ? ? ? ?ConsCell help = (ConsCell) alfa; > ? ? ? ? ? ?return new ConsCell(help.getHead(), append(help.getTail(), > beta)); > ? ? ? ?} > ? ?} > > ? ?private static Object reverse(Object alfa) { > ? ? ? ?if (alfa == null) { > ? ? ? ? ? ?return null; > ? ? ? ?} else { > ? ? ? ? ? ?ConsCell help = (ConsCell) alfa; > ? ? ? ? ? ?return append(reverse(help.getTail()), > ? ? ? ? ? ? ? ? ? ?new ConsCell(help.getHead(), null)); > ? ? ? ?} > ? ?} > > ? ?private static void _for(int n) { > ? ? ? ?for (int i=0; i ? ? ? ? ? ?reverse(data); > ? ? ? ?} > ? ?} > > ? ?private static void bench(int n) { > ? ? ? ?long t1=System.nanoTime(); > ? ? ? ?_for(n); > ? ? ? ?long t2=System.nanoTime(); > ? ? ? ?System.out.println("nrev\tin "+(t2-t1)+" (nanos)"); > ? ?} > > ? ?public static void main(String[] args) { > ? ? ? ?bench(6000); > ? ? ? ?bench(6000); > ? ?} > } > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mfidelman@REDACTED Wed Mar 14 14:24:41 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Wed, 14 Mar 2012 09:24:41 -0400 Subject: [erlang-questions] Erlang: Bring cloud to your application In-Reply-To: References: <4F606D90.5070206@fastmail.fm> Message-ID: <4F609C19.5030808@meetinghouse.net> *ngocdaothanh* <> wrote: / > I've just created this presentation: > http://www.slideshare.net/ngocdaothanh/cloud-erlang > > Erlang is impressive but my words are not good enough to express that. > Please comment to help me improve the presentation. > / Nice! -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From sedrik@REDACTED Wed Mar 14 15:07:46 2012 From: sedrik@REDACTED (Fredrik Andersson) Date: Wed, 14 Mar 2012 15:07:46 +0100 Subject: [erlang-questions] Erlang: Bring cloud to your application In-Reply-To: <93bb94b8-30b9-4e4c-a732-aeda513646d3@qb4g2000pbb.googlegroups.com> References: <93bb94b8-30b9-4e4c-a732-aeda513646d3@qb4g2000pbb.googlegroups.com> Message-ID: Cool work. A small note, maybe you should not call the cookie for mypassword1 since that kinda implies that it has something to do with security. Cheers On Wed, Mar 14, 2012 at 9:17 AM, ngocdaothanh wrote: > Hi, > > I've just created this presentation: > http://www.slideshare.net/ngocdaothanh/cloud-erlang > > Erlang is impressive but my words are not good enough to express that. > Please comment to help me improve the presentation. > > Thanks, > Ngoc > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.hughes@REDACTED Wed Mar 14 15:20:15 2012 From: john.hughes@REDACTED (John Hughes) Date: Wed, 14 Mar 2012 15:20:15 +0100 Subject: [erlang-questions] Nitrogen on several ports Message-ID: I?m running nitrogen on inets, configured to serve *two* ports, an ordinary ip port and an ssl port. That works fine?nitrogen serves the same information on both ports, but if I want to interact securely with the site then I can do it on the ssl port. What I would LIKE to do is to disable the parts of the site that really require secure access, when I visit it through the non-encrypted port. But to do that, I need to find out from nitrogen which port the request being served came through. Does anyone know of a way to do that? John -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Wed Mar 14 15:28:32 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 14 Mar 2012 15:28:32 +0100 Subject: [erlang-questions] opcode 153; this emulator supports only up to 152 ? Message-ID: <6C88BAB2-FC2B-45AB-B1D6-FCFC6D5A2CF1@gmail.com> Hi guys, I'm using a release compiled on R14B04. This release try to load some BEAM files compiled on R15B. Then I got this: 15:15:20.762 [error] beam/beam_load.c(1365): Error loading module store_manager: use of opcode 153; this emulator supports only up to 152 How can I fix that without recompiling my whole release under R15B? Regards, Zabrane From bengt.kleberg@REDACTED Wed Mar 14 15:45:32 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 14 Mar 2012 15:45:32 +0100 Subject: [erlang-questions] opcode 153; this emulator supports only up to 152 ? In-Reply-To: <6C88BAB2-FC2B-45AB-B1D6-FCFC6D5A2CF1@gmail.com> References: <6C88BAB2-FC2B-45AB-B1D6-FCFC6D5A2CF1@gmail.com> Message-ID: <1331736332.5273.28.camel@seasc1137> Greetings, You can recompile the R15B modules with R14B(04). bengt On Wed, 2012-03-14 at 15:28 +0100, Zabrane Mickael wrote: > Hi guys, > > I'm using a release compiled on R14B04. This release try to load some BEAM files compiled on R15B. > > Then I got this: > 15:15:20.762 [error] beam/beam_load.c(1365): Error loading module store_manager: > use of opcode 153; this emulator supports only up to 152 > > How can I fix that without recompiling my whole release under R15B? > > Regards, > Zabrane > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Wed Mar 14 15:58:28 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 14 Mar 2012 15:58:28 +0100 Subject: [erlang-questions] opcode 153; this emulator supports only up to 152 ? In-Reply-To: <1331736332.5273.28.camel@seasc1137> References: <6C88BAB2-FC2B-45AB-B1D6-FCFC6D5A2CF1@gmail.com> <1331736332.5273.28.camel@seasc1137> Message-ID: On Mar 14, 2012, at 3:45 PM, Bengt Kleberg wrote: > Greetings, > > You can recompile the R15B modules with R14B(04). That's what I want to avoid too. I don't have an R14B04 compiler anymore on my machines. Regards, Zabrane > bengt > > On Wed, 2012-03-14 at 15:28 +0100, Zabrane Mickael wrote: >> Hi guys, >> >> I'm using a release compiled on R14B04. This release try to load some BEAM files compiled on R15B. >> >> Then I got this: >> 15:15:20.762 [error] beam/beam_load.c(1365): Error loading module store_manager: >> use of opcode 153; this emulator supports only up to 152 >> >> How can I fix that without recompiling my whole release under R15B? >> >> Regards, >> Zabrane >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From pablo.platt@REDACTED Wed Mar 14 16:33:25 2012 From: pablo.platt@REDACTED (Pablo Platt) Date: Wed, 14 Mar 2012 08:33:25 -0700 (PDT) Subject: [erlang-questions] Erlang meets physics In-Reply-To: References: <1331666426.33915.YahooMailNeo@web112614.mail.gq1.yahoo.com> Message-ID: <1331739205.24889.YahooMailNeo@web112620.mail.gq1.yahoo.com> >which the device module (in this case hp8340b) translates into the GPIB command "OPCW", and based on the >device address (which is governed by the atom in the first argument), dispatches it over a bus handle that it owns >in a state variable via?bus:send_query/2.?? Can you explain how the device module translate to the GPIB command "OPCW"? Is it pure erlang? Maybe you can share some code? Thanks ________________________________ From: Jared Kofron To: Erlang Questions Sent: Wednesday, March 14, 2012 1:43 AM Subject: Re: [erlang-questions] Erlang meets physics Hi Pablo- My first project was on embedded hardware, and basically consisted of NIFs and Webmachine dispatching. Pretty fun stuff. ? My current project is a little more involved, but is also pretty interesting: Right now the bus over which communication takes place is abstracted away by having hardware modules which translate API functions into their appropriate wire representation and then transmit those representations over a handle that they have to the correct bus. ? In essence what I've done is taken instruments and buses and given them something like behaviors. ? Hardware can perform read,write,or configure, for example. ?So if I want to read the center frequency of a sweeper, I might say hp8340b:read(high_frequency_sweeper,<<"cw_freq">>). which the device module (in this case hp8340b) translates into the GPIB command "OPCW", and based on the device address (which is governed by the atom in the first argument), dispatches it over a bus handle that it owns in a state variable via?bus:send_query/2. ? At the moment, we only communicate with things over GPIB via ethernet using prologix devices to do the translation for us - basically glorified telnet - but it gets the job done. ?Everything is in a very alpha stage right now for this project, but it is working really nicely so far. JK On Mar 13, 2012, at 12:20 PM, Pablo Platt wrote: How do you interact with the hardware? >Do you use GPIB C libr and wrap it with a NIF? > > > >________________________________ > From: Joe Armstrong >To: Jared Kofron >Cc: Erlang Questions >Sent: Tuesday, March 13, 2012 12:34 PM >Subject: Re: [erlang-questions] Erlang meets physics > >Great news - spread the word ! > >Just for the record Erlang programmers numbers 1 and 2 (ie myself and >Robert Virding) >are both ex physicists. > >When I lecture I often point out the similarity between causality and >message reception. >You don't know that something has happened until you get a message >telling that it has happened. > >(In physics it's a ray of light, or a photon, or something - >forgetting entanglement for the moment) > >In computing it's the reception of a message. > >As a ex physicist I know that we can't say anything about simultaneous >events occurring >at different places in space-time - turn this into computer science >and the same arguments >apply to things like making sure replicated data is consistent on >remote sites - well you can't >- at least if you want to change it - Brewer's CAP theorem applies - >which for a physicist makes >perfect sense. > >Also as an ex physicist I realize that things do actually happen in >parallel in the real world, >so modelling them in a sequential programming language (if I wanted to do that) >is big time crazy - just describe the parallel stuff in a concurrent >language and the program >writes itself. Wait a few years till we have million core computers >and the parallel problems >can be solved 1:1 on parallel computers - and programming simulations >and so on will be >really easy - but don't even think about doing it in a sequential language... > >Cheers > >/Joe > > >On Mon, Mar 12, 2012 at 2:34 AM, Jared Kofron wrote: >> Hi All, >> I've been using Erlang at work for a few years now, and I thought I'd throw my experience out there, as >> my application is a little different than what you usually see on the list - I am a graduate student at the >> Center for Nuclear Physics and Astrophysics at the University of Washington, and use Erlang extensively >> in my work. >> >> In my experience, something that Erlang is really great at but doesn't receive much attention for these days >> is managing and interacting with hardware. ?In any physics experiment of even modest sizes, you wind up >> having to keep track of the state of various pieces of equipment, often modify that state, and constantly >> interrogate particular values. ?For example, we might want to change the current in a magnetic trap, turn >> that trap off altogether, or simply read back the voltage drop across our superconducting magnet. >> >> So far, I have deployed Erlang in this zone for two separate experiments (SNO+, a large particle physics >> experiment in Canada) and Project 8 (a small nuclear physics experiment here in Seattle). ?Both times have >> been great successes, and I have found the reception of Erlang in this market to be great. ?In general, what >> I have done is wrap a hardware management layer with some kind of outside world interface. For SNO+, we >> used Webmachine and RESTful control, and for Project 8 we actually conduct all communication >> by using CouchDB as a message passing interface. >> >> Physicists are suspicious creatures, but once you demonstrate the feature set that you get for practically >> free with OTP, they see the advantage pretty quickly. ?On top of that, the development cycle for sophisticated >> applications can be greatly reduced - more than once it made my group float to the top in terms of meeting >> goals. >> >> In short, as far as I am concerned, Erlang has found a new niche in the world of Physics, and I intend to >> spread the word as much as I can! >> >> Jared Kofron >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From kenneth.lundin@REDACTED Wed Mar 14 17:02:50 2012 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 14 Mar 2012 17:02:50 +0100 Subject: [erlang-questions] opcode 153; this emulator supports only up to 152 ? In-Reply-To: References: <6C88BAB2-FC2B-45AB-B1D6-FCFC6D5A2CF1@gmail.com> <1331736332.5273.28.camel@seasc1137> Message-ID: The compatibility goes the other way i.e. modules built with R14B can be run on R15B not the other way around, at least that is nothing that we guarantee. /Kenneth , Erlang/OTP Ericsson On Wed, Mar 14, 2012 at 3:58 PM, Zabrane Mickael wrote: > > On Mar 14, 2012, at 3:45 PM, Bengt Kleberg wrote: > > > Greetings, > > > > You can recompile the R15B modules with R14B(04). > > That's what I want to avoid too. I don't have an R14B04 compiler anymore > on my machines. > > Regards, > Zabrane > > > > bengt > > > > On Wed, 2012-03-14 at 15:28 +0100, Zabrane Mickael wrote: > >> Hi guys, > >> > >> I'm using a release compiled on R14B04. This release try to load some > BEAM files compiled on R15B. > >> > >> Then I got this: > >> 15:15:20.762 [error] beam/beam_load.c(1365): Error loading module > store_manager: > >> use of opcode 153; this emulator supports only up to 152 > >> > >> How can I fix that without recompiling my whole release under R15B? > >> > >> Regards, > >> Zabrane > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Wed Mar 14 17:04:23 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 14 Mar 2012 17:04:23 +0100 Subject: [erlang-questions] opcode 153; this emulator supports only up to 152 ? In-Reply-To: References: <6C88BAB2-FC2B-45AB-B1D6-FCFC6D5A2CF1@gmail.com> <1331736332.5273.28.camel@seasc1137> Message-ID: <4CF2F368-F144-4384-A255-8AA57CC2F2B3@gmail.com> Thanks Kenneth. I'll re-compile my release with R15B. Regards, Zabrane On Mar 14, 2012, at 5:02 PM, Kenneth Lundin wrote: > The compatibility goes the other way i.e. modules built with R14B can be run on R15B not the other way around, at least that is > nothing that we guarantee. > > /Kenneth , Erlang/OTP Ericsson > > On Wed, Mar 14, 2012 at 3:58 PM, Zabrane Mickael wrote: > > On Mar 14, 2012, at 3:45 PM, Bengt Kleberg wrote: > > > Greetings, > > > > You can recompile the R15B modules with R14B(04). > > That's what I want to avoid too. I don't have an R14B04 compiler anymore on my machines. > > Regards, > Zabrane > > > > bengt > > > > On Wed, 2012-03-14 at 15:28 +0100, Zabrane Mickael wrote: > >> Hi guys, > >> > >> I'm using a release compiled on R14B04. This release try to load some BEAM files compiled on R15B. > >> > >> Then I got this: > >> 15:15:20.762 [error] beam/beam_load.c(1365): Error loading module store_manager: > >> use of opcode 153; this emulator supports only up to 152 > >> > >> How can I fix that without recompiling my whole release under R15B? > >> > >> Regards, > >> Zabrane > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Wed Mar 14 17:13:50 2012 From: gumm@REDACTED (Jesse Gumm) Date: Wed, 14 Mar 2012 11:13:50 -0500 Subject: [erlang-questions] Nitrogen on several ports In-Reply-To: References: Message-ID: Hi John, You can get the socket from Nitrogen and from the socket determine its type: In code that could be something like is_ssl() -> Socket = wf:socket(), is_tuple(Socket) andalso element(1,Socket)=:=sslsocket. Though I personally prefer to throw nginx in front of Nitrogen deal with that kind of routing (URL-based, anyway). -Jesse On Wed, Mar 14, 2012 at 9:20 AM, John Hughes wrote: > I?m running nitrogen on inets, configured to serve *two* ports, an ordinary > ip port and an ssl port. That works fine?nitrogen serves the same > information on both ports, but if I want to interact securely with the site > then I can do it on the ssl port. > > What I would LIKE to do is to disable the parts of the site that really > require secure access, when I visit it through the non-encrypted port. But > to do that, I need to find out from nitrogen which port the request being > served came through. Does anyone know of a way to do that? > > John > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm From phil.toland@REDACTED Wed Mar 14 19:44:10 2012 From: phil.toland@REDACTED (Phillip Toland) Date: Wed, 14 Mar 2012 13:44:10 -0500 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F606D90.5070206@fastmail.fm> Message-ID: This reminds me of the early days of Ruby on Rails. The Java people would create micro benchmarks showing Java outperforming Ruby by a significant margin. The Rails folks would then produce actual sites that outperformed comparable Java sites by a significant margin. When most of the time is spent in network IO raw computational performance is far less of an issue. The JVM performs very well and Java looks good in simple benchmarks. Actual Java applications, not so much. ~p On Wed, Mar 14, 2012 at 8:17 AM, Michael Turner wrote: > There must be some corollary of Godwin's Law saying that all > discussions of programming languages, if they go on long enough, must > eventually degenerate into disagreements over either benchmark results > or open source licenses. ;-) > > -michael turner > > On Wed, Mar 14, 2012 at 7:06 PM, Jan Burse wrote: >> Shahrdad Shadab schrieb: >>> >>> _Java performance is nowhere near Erlang. >> >> >> Where is the evidence? I don't know >> whether Joe would approve with this >> statement? >> >> Did a little micro benchmark: Erlang is >> slower than Java. For general purpose >> programming. Did not test some message >> passing or so. >> >> Here are some results for Naive Reverse: >> >> Erlang 64-bit R15B: ~60 ms >> Java 64-bit JDK 1.7: ~30 ms (Turbo boost ~15ms) >> >> Tested on Windows 7, >> Sony VAIO VPC-SA3J1E/XI, >> CPU i5 2430M, 4GB RAM >> >> ------------------ benchmark.erl -------------------------- >> >> -module(benchmark). >> -export([test/0,nrev/0]). >> >> append([],X) -> X; >> append([X|Y],Z) -> [X|append(Y,Z)]. >> >> reverse([]) -> []; >> reverse([X|Y]) -> append(reverse(Y),[X]). >> >> data() -> [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20, >> 21,22,23,24,25,26,27,28,29,30]. >> >> nrev() -> reverse(data()). >> >> for(0,F) -> true; >> for(N,F) when N>0 -> apply(benchmark,F,[]), for(N-1,F). >> >> bench(N,F) -> >> {_,_,X}=os:timestamp(), >> for(N,F), >> {_,_,Y}=os:timestamp(), >> io:write(F), >> io:put_chars("\tin "), >> io:write(Y-X), >> io:put_chars(" (mics)"), >> io:nl(). >> >> test() -> bench(6000,nrev). >> >> ---------------------- ConsCell.java --------------------------- >> >> package demo.perpetual; >> >> /** >> ?*/ >> final class ConsCell { >> ? ?private final Object head; >> ? ?private final Object tail; >> >> ? ?ConsCell(Object h, Object t) { >> ? ? ? ?head = h; >> ? ? ? ?tail = t; >> ? ?} >> >> ? ?Object getHead() { >> ? ? ? ?return head; >> ? ?} >> >> ? ?Object getTail() { >> ? ? ? ?return tail; >> ? ?} >> >> } >> >> ---------------------- NaiveReverse.java ----------------------------- >> >> package demo.perpetual; >> >> /** >> ?*/ >> public final class NaiveReverse { >> ? ?private static Object data; >> >> ? ?static { >> ? ? ? ?Object help = null; >> ? ? ? ?for (int i = 30; i >= 1; i--) { >> ? ? ? ? ? ?help = new ConsCell(Integer.valueOf(i), help); >> ? ? ? ?} >> ? ? ? ?data = help; >> ? ?} >> >> ? ?private static Object append(Object alfa, Object beta) { >> ? ? ? ?if (alfa == null) { >> ? ? ? ? ? ?return beta; >> ? ? ? ?} else { >> ? ? ? ? ? ?ConsCell help = (ConsCell) alfa; >> ? ? ? ? ? ?return new ConsCell(help.getHead(), append(help.getTail(), >> beta)); >> ? ? ? ?} >> ? ?} >> >> ? ?private static Object reverse(Object alfa) { >> ? ? ? ?if (alfa == null) { >> ? ? ? ? ? ?return null; >> ? ? ? ?} else { >> ? ? ? ? ? ?ConsCell help = (ConsCell) alfa; >> ? ? ? ? ? ?return append(reverse(help.getTail()), >> ? ? ? ? ? ? ? ? ? ?new ConsCell(help.getHead(), null)); >> ? ? ? ?} >> ? ?} >> >> ? ?private static void _for(int n) { >> ? ? ? ?for (int i=0; i> ? ? ? ? ? ?reverse(data); >> ? ? ? ?} >> ? ?} >> >> ? ?private static void bench(int n) { >> ? ? ? ?long t1=System.nanoTime(); >> ? ? ? ?_for(n); >> ? ? ? ?long t2=System.nanoTime(); >> ? ? ? ?System.out.println("nrev\tin "+(t2-t1)+" (nanos)"); >> ? ?} >> >> ? ?public static void main(String[] args) { >> ? ? ? ?bench(6000); >> ? ? ? ?bench(6000); >> ? ?} >> } >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ~p From janburse@REDACTED Wed Mar 14 19:55:46 2012 From: janburse@REDACTED (Jan Burse) Date: Wed, 14 Mar 2012 19:55:46 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F606D90.5070206@fastmail.fm> Message-ID: <4F60E9B2.2010206@fastmail.fm> Hi, The benchmark says nothing about the language design. The language design is very neat. It says something about the compiler backend and the runtime system. Probably the meme that Erlang outperforms Java has reached life cycle end. Since Java is quite old (born 1995) one would expect that they have already pushed the limits in speed. But the boundaries have changed. When transitioning from 32-bit to 64-bit (more registers), and from JDK 1.6 to JDK 1.7 (better backend and runtime) I observed a speed up factor of 2.6. Same application and same hardware, just using a different JVM. So the competition is on. Maybe there will be a new beam interpreter in a few years, that can keep up with the development. Hint: The newer CPUs don't like branches. So threaded byte code interpreters, even with big instructions don't make much sense. Must go native code generation. Bye Phillip Toland schrieb: > This reminds me of the early days of Ruby on Rails. The Java people From janburse@REDACTED Wed Mar 14 19:58:56 2012 From: janburse@REDACTED (Jan Burse) Date: Wed, 14 Mar 2012 19:58:56 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F60E9B2.2010206@fastmail.fm> References: <4F606D90.5070206@fastmail.fm> <4F60E9B2.2010206@fastmail.fm> Message-ID: <4F60EA70.7020409@fastmail.fm> Jan Burse schrieb: > So the competition is on. Maybe there will be a new > beam interpreter in a few years, that can keep up > with the development. Hint: The newer CPUs don't > like branches. P.S.: I didn't play with the compiler options, inline, etc.. maybe something changes. Just used the default setting on both sides. From john.hughes@REDACTED Wed Mar 14 20:46:54 2012 From: john.hughes@REDACTED (John Hughes) Date: Wed, 14 Mar 2012 20:46:54 +0100 Subject: [erlang-questions] Nitrogen on several ports In-Reply-To: References: Message-ID: Hi Jesse, Thank you! That sounds like just what I need. I didn't realise wf:socket() would tell me the socket type--very useful! I am actually using Apache as a reverse proxy to forward port 80 to 8000 already... my problem is I want to access the *same page* (i.e. same .erl file) through http and https, and see different options in each case. I suppose I could make two different pages, but that would be more complex and less maintainable however I did it (code duplication or a more complex module structure). I'm not sure nginx would really solve that problem for me. John ----- Original Message ----- From: "Jesse Gumm" To: "John Hughes" Cc: Sent: Wednesday, March 14, 2012 5:13 PM Subject: Re: [erlang-questions] Nitrogen on several ports Hi John, You can get the socket from Nitrogen and from the socket determine its type: In code that could be something like is_ssl() -> Socket = wf:socket(), is_tuple(Socket) andalso element(1,Socket)=:=sslsocket. Though I personally prefer to throw nginx in front of Nitrogen deal with that kind of routing (URL-based, anyway). -Jesse On Wed, Mar 14, 2012 at 9:20 AM, John Hughes wrote: > I?m running nitrogen on inets, configured to serve *two* ports, an > ordinary > ip port and an ssl port. That works fine?nitrogen serves the same > information on both ports, but if I want to interact securely with the > site > then I can do it on the ssl port. > > What I would LIKE to do is to disable the parts of the site that really > require secure access, when I visit it through the non-encrypted port. But > to do that, I need to find out from nitrogen which port the request being > served came through. Does anyone know of a way to do that? > > John > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm From watson.timothy@REDACTED Wed Mar 14 21:20:33 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 14 Mar 2012 20:20:33 +0000 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F60E9B2.2010206@fastmail.fm> References: <4F606D90.5070206@fastmail.fm> <4F60E9B2.2010206@fastmail.fm> Message-ID: On 14 March 2012 18:55, Jan Burse wrote: > Hi, > > The benchmark says nothing about the language design. > The language design is very neat. It says something > about the compiler backend and the runtime system. > > Probably the meme that Erlang outperforms Java has > reached life cycle end. Since Java is quite old > (born 1995) one would expect that they have already > pushed the limits in speed. > > But the boundaries have changed. When transitioning > from 32-bit to 64-bit (more registers), and from JDK 1.6 > to JDK 1.7 (better backend and runtime) I observed a > speed up factor of 2.6. Same application and same > hardware, just using a different JVM. > > So the competition is on. Maybe there will be a new > beam interpreter in a few years, that can keep up > with the development. Hint: The newer CPUs don't > like branches. > > So threaded byte code interpreters, even with > big instructions don't make much sense. Must go > native code generation. > Of course we have HIPE for native code, but personally I don't actually know what kind of code it generates - must go and read up on it. I've seen a lot of these kind of arguments in numerous languages, some with a VM (like Java and .NET) and others (such as OCaml, Haskell, etc) without. JIT compilation is pretty powerful stuff, but I'm not actually convinced that's why you see such great throughput on the JVM - I suspect it has a lot more to do with mutable data being at the heart of the VM design and being able to utilise efficient swapping operations such as CAS. Also there are pros and cons to having a VM at all. Well written Haskell can have much better performance and throughput characteristics than the JVM for many applications, compiled straight down to native code without some additional runtime support generated into the executable or library. The JVM isn't the fastest game in town and just like any other programming environment, it can crash unexpectedly and badly written code can leak memory and perform terribly. Doesn't mean the JVM is badly designed - it's not - just that it's built by humans and therefore imperfect. The main advantages that a VM gives appears to be around O&M, tracing and diagnostics. And the portability thing, but as GHC demonstrates this can be done in other ways. From ok@REDACTED Thu Mar 15 00:27:14 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 15 Mar 2012 12:27:14 +1300 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F606D90.5070206@fastmail.fm> References: <4F606D90.5070206@fastmail.fm> Message-ID: <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> On 14/03/2012, at 11:06 PM, Jan Burse wrote: > Shahrdad Shadab schrieb: >> _Java performance is nowhere near Erlang. > > Where is the evidence? I don't know > whether Joe would approve with this > statement? Java is a language with several implementations; even within a single implementation the -client and -server options will give you different performance. I have no shortage of examples where Java is slower than the local copy of 'mawk'. (Yep, rewriting AWK into Java and adding all the type declarations and so on can make it *slower*.) I also have no shortage of examples where Java is faster than mawk. It all depends on what you are doing and how you are doing it. Early publications about Mercury showed it doing list manipulation usefully faster than hand-written C, which was queer (but true!) because it compiled via C. I suspect that matching binaries is likely to be faster in Erlang than in Java, simply because it is not something Java implementors have ever been focussed on. I can pretty much guarantee that computing Singular Value Decompositions in Erlang is going to be *horrible* compared with Java (although the Java SVD code in a package I bought turned out to be 8 times slower than C). There is often surprisingly little connection between the speed of a _language_ (implementation) and the speed of _systems_ built using it. In this context, it's worth pointing out that there are some really good profiling and monitoring tools for *both* Java and Erlang. From janburse@REDACTED Thu Mar 15 00:32:20 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 15 Mar 2012 00:32:20 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F606D90.5070206@fastmail.fm> <4F60E9B2.2010206@fastmail.fm> Message-ID: <4F612A84.6060407@fastmail.fm> Tim Watson schrieb: > I suspect it has a lot more to do with mutable data being at > the heart of the VM design and being able to utilise efficient > swapping operations such as CAS. I choose an example that was implemented in Java with immutable data. Also the example is sequential, except maybe for the Java GC which runs concurrently. Also no penguins were harmed in the example. From janburse@REDACTED Thu Mar 15 00:55:31 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 15 Mar 2012 00:55:31 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> References: <4F606D90.5070206@fastmail.fm> <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> Message-ID: <4F612FF3.7030804@fastmail.fm> Richard O'Keefe schrieb: > Java is a language with several implementations; even within a single > implementation the -client and -server options will give you different > performance. I guess the -client version is the default, and that is what I used. > Early publications about Mercury showed it doing list > manipulation usefully faster than hand-written C, which > was queer (but true!) because it compiled via C. I was positively surprised by the Erlang performance of ~60ms for the nrev example and for() implementation I used. GNU Prolog does it in around ~250ms with for backtracking. Didn't check YAP Prolog, but I guess it will do it in around ~25ms. But the Java performance also surprised me. Especially since the code contains many new expressions. But rumors have it that new is just a pointer decrement in Java. Bye From ok@REDACTED Thu Mar 15 01:09:29 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 15 Mar 2012 13:09:29 +1300 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F606D90.5070206@fastmail.fm> References: <4F606D90.5070206@fastmail.fm> Message-ID: Jan Burse's naive reverse for Erlang is not quite what we usually mean by nrev; he measures the time to create the initial data list as well, which the Java version doesn't. Revising the Erlang code slightly, I got 2.8 usec per call. I also revised the Java code slightly, for example, NOT to use accessor methods, and NOT to require any casting during the benchmark. Also, both benchmarks were revised by doing t0 = now run a dummy benchmark calling an identity function t1 = now run the real benchmark t2 = now report ((t2 - t1) - (t1 - t0))/(double)number-of-iterations as is traditional in the naive reverse benchmark. The final score: Erlang: 2.8 usec per call Java : 3.2 usec per call (reported as 3226.977 nsec) Machine: a MacBook pro with a 2GHz Intel core 2 Duo cpu running Mac OS X 10.6.8. Multiplying these numbers by 6000 to match Jan Burse's: Me Him Him, scaled Erlang 16.8 m ~60 ms ~ 72.9 ms Java 19.2 ms ~30 ms ~ 36.5 ms (The scaled column assumes that the 2430 number is CPU speed; if that's not what it is, ignore that column.) There's one tiny change that accounts for this. And it's the reason this benchmark really tells us nothing worth knowing. From cgsmcmlxxv@REDACTED Thu Mar 15 01:21:58 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Thu, 15 Mar 2012 01:21:58 +0100 Subject: [erlang-questions] Erlang meets physics In-Reply-To: <1331739205.24889.YahooMailNeo@web112620.mail.gq1.yahoo.com> References: <1331666426.33915.YahooMailNeo@web112614.mail.gq1.yahoo.com> <1331739205.24889.YahooMailNeo@web112620.mail.gq1.yahoo.com> Message-ID: Hi, Firstly, I am glad to see physicists getting interest in Erlang. I see Erlang best usage in physics in few fields from Astrophysics/Astroparticle Physics/Elementary Particle Physics: 1. GRID; 2. integrated in the readout of an array of detectors for atmospheric showers, but there are few fields in which Erlang still cannot replace C/C++ (or Verilog in some cases): 1. accelerator (high and medium energy) physics (high frequency readout may not be compatible with Erlang, or vice-versa); 2. Monte Carlo simulations (even if it provides nice concurrency and code hot swap, it lacks some quite important characteristics). There may be some other fields from physics where Erlang may be suitable (except for communications for which is intended), but I am not the person to speak about. Nevertheless, it's good to know Erlang enters the above mentioned fields. Good luck! CGS On Wed, Mar 14, 2012 at 4:33 PM, Pablo Platt wrote: > >which the device module (in this case hp8340b) translates into the GPIB > command "OPCW", and based on the > >device address (which is governed by the atom in the first argument), > dispatches it over a bus handle that it owns > >in a state variable via bus:send_query/2. > > Can you explain how the device module translate to the GPIB command "OPCW"? > Is it pure erlang? > Maybe you can share some code? > > Thanks > > > > > ________________________________ > From: Jared Kofron > To: Erlang Questions > Sent: Wednesday, March 14, 2012 1:43 AM > Subject: Re: [erlang-questions] Erlang meets physics > > > Hi Pablo- > My first project was on embedded hardware, and basically consisted of NIFs > and Webmachine dispatching. > Pretty fun stuff. > > My current project is a little more involved, but is also pretty > interesting: > > Right now the bus over which communication takes place is abstracted away > by having hardware modules > which translate API functions into their appropriate wire representation > and then transmit those representations > over a handle that they have to the correct bus. > > In essence what I've done is taken instruments and buses and given them > something like behaviors. > Hardware can perform read,write,or configure, for example. So if I want > to read the center frequency of a sweeper, > I might say > > hp8340b:read(high_frequency_sweeper,<<"cw_freq">>). > > which the device module (in this case hp8340b) translates into the GPIB > command "OPCW", and based on the > device address (which is governed by the atom in the first argument), > dispatches it over a bus handle that it owns > in a state variable via bus:send_query/2. > > At the moment, we only communicate with things over GPIB via ethernet > using prologix devices to do the translation > for us - basically glorified telnet - but it gets the job done. > Everything is in a very alpha stage right now for this project, > but it is working really nicely so far. > > JK > > On Mar 13, 2012, at 12:20 PM, Pablo Platt wrote: > > How do you interact with the hardware? > >Do you use GPIB C libr and wrap it with a NIF? > > > > > > > >________________________________ > > From: Joe Armstrong > >To: Jared Kofron > >Cc: Erlang Questions > >Sent: Tuesday, March 13, 2012 12:34 PM > >Subject: Re: [erlang-questions] Erlang meets physics > > > >Great news - spread the word ! > > > >Just for the record Erlang programmers numbers 1 and 2 (ie myself and > >Robert Virding) > >are both ex physicists. > > > >When I lecture I often point out the similarity between causality and > >message reception. > >You don't know that something has happened until you get a message > >telling that it has happened. > > > >(In physics it's a ray of light, or a photon, or something - > >forgetting entanglement for the moment) > > > >In computing it's the reception of a message. > > > >As a ex physicist I know that we can't say anything about simultaneous > >events occurring > >at different places in space-time - turn this into computer science > >and the same arguments > >apply to things like making sure replicated data is consistent on > >remote sites - well you can't > >- at least if you want to change it - Brewer's CAP theorem applies - > >which for a physicist makes > >perfect sense. > > > >Also as an ex physicist > I realize that things do actually happen in > >parallel in the real world, > >so modelling them in a sequential programming language (if I wanted to do > that) > >is big time crazy - just describe the parallel stuff in a concurrent > >language and the program > >writes itself. Wait a few years till we have million core computers > >and the parallel problems > >can be solved 1:1 on parallel computers - and programming simulations > >and so on will be > >really easy - but don't even think about doing it in a sequential > language... > > > >Cheers > > > >/Joe > > > > > >On Mon, Mar 12, 2012 at 2:34 AM, Jared Kofron > wrote: > >> Hi All, > >> I've been using Erlang at work for a few years now, and I thought I'd > throw my experience out there, as > >> my application is a little different than what you usually see on the > list - I am a graduate > student at the > >> Center for Nuclear Physics and Astrophysics at the University of > Washington, and use Erlang extensively > >> in my work. > >> > >> In my experience, something that Erlang is really great at but doesn't > receive much attention for these days > >> is managing and interacting with hardware. In any physics experiment > of even modest sizes, you wind up > >> having to keep track of the state of various pieces of equipment, often > modify that state, and constantly > >> interrogate particular values. For example, we might want to change > the current in a magnetic trap, turn > >> that trap off altogether, or simply read back the voltage drop across > our superconducting magnet. > >> > >> So far, I have deployed Erlang in this zone for two separate > experiments (SNO+, a large particle physics > >> experiment in Canada) and Project 8 (a small nuclear physics experiment > here in Seattle). Both times > have > >> been great successes, and I have found the reception of Erlang in this > market to be great. In general, what > >> I have done is wrap a hardware management layer with some kind of > outside world interface. For SNO+, we > >> used Webmachine and RESTful control, and for Project 8 we actually > conduct all communication > >> by using CouchDB as a message passing interface. > >> > >> Physicists are suspicious creatures, but once you demonstrate the > feature set that you get for practically > >> free with OTP, they see the advantage pretty quickly. On top of that, > the development cycle for sophisticated > >> applications can be greatly reduced - more than once it made my group > float to the top in terms of meeting > >> goals. > >> > >> In short, as far as I am concerned, Erlang has found a new niche in the > world of Physics, and I intend to > >> spread the word as much as I can! > >> > >> Jared > Kofron > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >_______________________________________________ > >erlang-questions mailing list > >erlang-questions@REDACTED > >http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From janburse@REDACTED Thu Mar 15 01:25:13 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 15 Mar 2012 01:25:13 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F606D90.5070206@fastmail.fm> Message-ID: <4F6136E9.2040309@fastmail.fm> Richard O'Keefe schrieb: > Machine: a MacBook pro with a 2GHz Intel core 2 > Duo cpu running Mac OS X 10.6.8. > > Multiplying these numbers by 6000 to match Jan Burse's: > Me Him Him, scaled > Erlang 16.8 m ~60 ms ~ 72.9 ms > Java 19.2 ms ~30 ms ~ 36.5 ms > (The scaled column assumes that the 2430 number is CPU > speed; if that's not what it is, ignore that column.) What Java did you use? There are pretty differences between JDK 1.6 and JDK 1.7, and between 32-bit and 64-bit. You don't say this in your post. (This is kind of the crucial point of my post, that JDK 1.7 is special) Getting JDK 1.7 for a Mac is a little tricky but possible. I have 1.7.0u.jdk, and lacks a little bit behind the Windows JDK 1.7 in updates. > he measures the time to create the initial data > list as well, which the Java version doesn't. I am assuming that Erlang optimizes away constant expressions similarly. But anyway the algorithm is O(n^2), so creating a list O(n) shouldn't change that much. But I can redo the test on my Windows machine. Hold on. Bye From janburse@REDACTED Thu Mar 15 01:28:44 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 15 Mar 2012 01:28:44 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F6136E9.2040309@fastmail.fm> References: <4F606D90.5070206@fastmail.fm> <4F6136E9.2040309@fastmail.fm> Message-ID: <4F6137BC.2040705@fastmail.fm> Jan Burse schrieb: > Getting JDK 1.7 for a Mac is a little tricky but possible. I > have 1.7.0u.jdk, and lacks a little bit behind the Windows > JDK 1.7 in updates. Not only getting, also running on a Mac is a little bit tricky. On my machine it did not register as a default JVM, so I have always to use a very long path in the console: /Library/Java/JavaVirtualMachines/1.7.0u.jdk/Contents/Home/bin/java From jared.nance@REDACTED Thu Mar 15 01:30:07 2012 From: jared.nance@REDACTED (Jared Kofron) Date: Wed, 14 Mar 2012 17:30:07 -0700 Subject: [erlang-questions] Erlang meets physics In-Reply-To: References: <1331666426.33915.YahooMailNeo@web112614.mail.gq1.yahoo.com> <1331739205.24889.YahooMailNeo@web112620.mail.gq1.yahoo.com> Message-ID: <40FAC330-B3A3-4123-BE08-FFF28A1290A6@gmail.com> On Mar 14, 2012, at 5:21 PM, CGS wrote: > Hi, > > Firstly, I am glad to see physicists getting interest in Erlang. > > I see Erlang best usage in physics in few fields from Astrophysics/Astroparticle Physics/Elementary Particle Physics: > 1. GRID; > 2. integrated in the readout of an array of detectors for atmospheric showers, Very true, these are great test cases for Erlang. EPICS is a project that accelerator folks wrote a long time ago (and still use and maintain to an extent) that I think is a no-brainer application for Erlang: http://www.aps.anl.gov/epics/. It is written in C and forms a distributed data acquisition system. > but there are few fields in which Erlang still cannot replace C/C++ (or Verilog in some cases): > 1. accelerator (high and medium energy) physics (high frequency readout may not be compatible with Erlang, or vice-versa); > 2. Monte Carlo simulations (even if it provides nice concurrency and code hot swap, it lacks some quite important characteristics). > > There may be some other fields from physics where Erlang may be suitable (except for communications for which is intended), but I am not the person to speak about. Nevertheless, it's good to know Erlang enters the above mentioned fields. > > Good luck! > CGS > > > > > On Wed, Mar 14, 2012 at 4:33 PM, Pablo Platt wrote: > >which the device module (in this case hp8340b) translates into the GPIB command "OPCW", and based on the > >device address (which is governed by the atom in the first argument), dispatches it over a bus handle that it owns > >in a state variable via bus:send_query/2. > > Can you explain how the device module translate to the GPIB command "OPCW"? > Is it pure erlang? > Maybe you can share some code? > > Thanks > > > > > ________________________________ > From: Jared Kofron > To: Erlang Questions > Sent: Wednesday, March 14, 2012 1:43 AM > Subject: Re: [erlang-questions] Erlang meets physics > > > Hi Pablo- > My first project was on embedded hardware, and basically consisted of NIFs and Webmachine dispatching. > Pretty fun stuff. > > My current project is a little more involved, but is also pretty interesting: > > Right now the bus over which communication takes place is abstracted away by having hardware modules > which translate API functions into their appropriate wire representation and then transmit those representations > over a handle that they have to the correct bus. > > In essence what I've done is taken instruments and buses and given them something like behaviors. > Hardware can perform read,write,or configure, for example. So if I want to read the center frequency of a sweeper, > I might say > > hp8340b:read(high_frequency_sweeper,<<"cw_freq">>). > > which the device module (in this case hp8340b) translates into the GPIB command "OPCW", and based on the > device address (which is governed by the atom in the first argument), dispatches it over a bus handle that it owns > in a state variable via bus:send_query/2. > > At the moment, we only communicate with things over GPIB via ethernet using prologix devices to do the translation > for us - basically glorified telnet - but it gets the job done. Everything is in a very alpha stage right now for this project, > but it is working really nicely so far. > > JK > > On Mar 13, 2012, at 12:20 PM, Pablo Platt wrote: > > How do you interact with the hardware? > >Do you use GPIB C libr and wrap it with a NIF? > > > > > > > >________________________________ > > From: Joe Armstrong > >To: Jared Kofron > >Cc: Erlang Questions > >Sent: Tuesday, March 13, 2012 12:34 PM > >Subject: Re: [erlang-questions] Erlang meets physics > > > >Great news - spread the word ! > > > >Just for the record Erlang programmers numbers 1 and 2 (ie myself and > >Robert Virding) > >are both ex physicists. > > > >When I lecture I often point out the similarity between causality and > >message reception. > >You don't know that something has happened until you get a message > >telling that it has happened. > > > >(In physics it's a ray of light, or a photon, or something - > >forgetting entanglement for the moment) > > > >In computing it's the reception of a message. > > > >As a ex physicist I know that we can't say anything about simultaneous > >events occurring > >at different places in space-time - turn this into computer science > >and the same arguments > >apply to things like making sure replicated data is consistent on > >remote sites - well you can't > >- at least if you want to change it - Brewer's CAP theorem applies - > >which for a physicist makes > >perfect sense. > > > >Also as an ex physicist > I realize that things do actually happen in > >parallel in the real world, > >so modelling them in a sequential programming language (if I wanted to do that) > >is big time crazy - just describe the parallel stuff in a concurrent > >language and the program > >writes itself. Wait a few years till we have million core computers > >and the parallel problems > >can be solved 1:1 on parallel computers - and programming simulations > >and so on will be > >really easy - but don't even think about doing it in a sequential language... > > > >Cheers > > > >/Joe > > > > > >On Mon, Mar 12, 2012 at 2:34 AM, Jared Kofron wrote: > >> Hi All, > >> I've been using Erlang at work for a few years now, and I thought I'd throw my experience out there, as > >> my application is a little different than what you usually see on the list - I am a graduate > student at the > >> Center for Nuclear Physics and Astrophysics at the University of Washington, and use Erlang extensively > >> in my work. > >> > >> In my experience, something that Erlang is really great at but doesn't receive much attention for these days > >> is managing and interacting with hardware. In any physics experiment of even modest sizes, you wind up > >> having to keep track of the state of various pieces of equipment, often modify that state, and constantly > >> interrogate particular values. For example, we might want to change the current in a magnetic trap, turn > >> that trap off altogether, or simply read back the voltage drop across our superconducting magnet. > >> > >> So far, I have deployed Erlang in this zone for two separate experiments (SNO+, a large particle physics > >> experiment in Canada) and Project 8 (a small nuclear physics experiment here in Seattle). Both times > have > >> been great successes, and I have found the reception of Erlang in this market to be great. In general, what > >> I have done is wrap a hardware management layer with some kind of outside world interface. For SNO+, we > >> used Webmachine and RESTful control, and for Project 8 we actually conduct all communication > >> by using CouchDB as a message passing interface. > >> > >> Physicists are suspicious creatures, but once you demonstrate the feature set that you get for practically > >> free with OTP, they see the advantage pretty quickly. On top of that, the development cycle for sophisticated > >> applications can be greatly reduced - more than once it made my group float to the top in terms of meeting > >> goals. > >> > >> In short, as far as I am concerned, Erlang has found a new niche in the world of Physics, and I intend to > >> spread the word as much as I can! > >> > >> Jared > Kofron > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >_______________________________________________ > >erlang-questions mailing list > >erlang-questions@REDACTED > >http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jared.nance@REDACTED Thu Mar 15 01:32:38 2012 From: jared.nance@REDACTED (Jared Kofron) Date: Wed, 14 Mar 2012 17:32:38 -0700 Subject: [erlang-questions] Erlang meets physics In-Reply-To: <1331739205.24889.YahooMailNeo@web112620.mail.gq1.yahoo.com> References: <1331666426.33915.YahooMailNeo@web112614.mail.gq1.yahoo.com> <1331739205.24889.YahooMailNeo@web112620.mail.gq1.yahoo.com> Message-ID: <961AF67D-1E9C-41B1-A5D8-DFACB6AD63C8@gmail.com> Hi Pablo- The translation is a simple pattern match - locator_to_ch_data(<<"cw_freq">>) -> "CW", which is then transformed into the final string which is passed over the wire. If you'd like to take a look at the code it's at http://github.com/kofron/dripline. It's very very alpha, although deployed in limited production it works quite well. Unfortunately my first project, called hmhj, is closed source due to concerns from the SNO+ collaboration. JK On Mar 14, 2012, at 8:33 AM, Pablo Platt wrote: >> which the device module (in this case hp8340b) translates into the GPIB command "OPCW", and based on the >> device address (which is governed by the atom in the first argument), dispatches it over a bus handle that it owns >> in a state variable via bus:send_query/2. > > Can you explain how the device module translate to the GPIB command "OPCW"? > Is it pure erlang? > Maybe you can share some code? > > Thanks > > > > > ________________________________ > From: Jared Kofron > To: Erlang Questions > Sent: Wednesday, March 14, 2012 1:43 AM > Subject: Re: [erlang-questions] Erlang meets physics > > > Hi Pablo- > My first project was on embedded hardware, and basically consisted of NIFs and Webmachine dispatching. > Pretty fun stuff. > > My current project is a little more involved, but is also pretty interesting: > > Right now the bus over which communication takes place is abstracted away by having hardware modules > which translate API functions into their appropriate wire representation and then transmit those representations > over a handle that they have to the correct bus. > > In essence what I've done is taken instruments and buses and given them something like behaviors. > Hardware can perform read,write,or configure, for example. So if I want to read the center frequency of a sweeper, > I might say > > hp8340b:read(high_frequency_sweeper,<<"cw_freq">>). > > which the device module (in this case hp8340b) translates into the GPIB command "OPCW", and based on the > device address (which is governed by the atom in the first argument), dispatches it over a bus handle that it owns > in a state variable via bus:send_query/2. > > At the moment, we only communicate with things over GPIB via ethernet using prologix devices to do the translation > for us - basically glorified telnet - but it gets the job done. Everything is in a very alpha stage right now for this project, > but it is working really nicely so far. > > JK > > On Mar 13, 2012, at 12:20 PM, Pablo Platt wrote: > > How do you interact with the hardware? >> Do you use GPIB C libr and wrap it with a NIF? >> >> >> >> ________________________________ >> From: Joe Armstrong >> To: Jared Kofron >> Cc: Erlang Questions >> Sent: Tuesday, March 13, 2012 12:34 PM >> Subject: Re: [erlang-questions] Erlang meets physics >> >> Great news - spread the word ! >> >> Just for the record Erlang programmers numbers 1 and 2 (ie myself and >> Robert Virding) >> are both ex physicists. >> >> When I lecture I often point out the similarity between causality and >> message reception. >> You don't know that something has happened until you get a message >> telling that it has happened. >> >> (In physics it's a ray of light, or a photon, or something - >> forgetting entanglement for the moment) >> >> In computing it's the reception of a message. >> >> As a ex physicist I know that we can't say anything about simultaneous >> events occurring >> at different places in space-time - turn this into computer science >> and the same arguments >> apply to things like making sure replicated data is consistent on >> remote sites - well you can't >> - at least if you want to change it - Brewer's CAP theorem applies - >> which for a physicist makes >> perfect sense. >> >> Also as an ex physicist > I realize that things do actually happen in >> parallel in the real world, >> so modelling them in a sequential programming language (if I wanted to do that) >> is big time crazy - just describe the parallel stuff in a concurrent >> language and the program >> writes itself. Wait a few years till we have million core computers >> and the parallel problems >> can be solved 1:1 on parallel computers - and programming simulations >> and so on will be >> really easy - but don't even think about doing it in a sequential language... >> >> Cheers >> >> /Joe >> >> >> On Mon, Mar 12, 2012 at 2:34 AM, Jared Kofron wrote: >>> Hi All, >>> I've been using Erlang at work for a few years now, and I thought I'd throw my experience out there, as >>> my application is a little different than what you usually see on the list - I am a graduate > student at the >>> Center for Nuclear Physics and Astrophysics at the University of Washington, and use Erlang extensively >>> in my work. >>> >>> In my experience, something that Erlang is really great at but doesn't receive much attention for these days >>> is managing and interacting with hardware. In any physics experiment of even modest sizes, you wind up >>> having to keep track of the state of various pieces of equipment, often modify that state, and constantly >>> interrogate particular values. For example, we might want to change the current in a magnetic trap, turn >>> that trap off altogether, or simply read back the voltage drop across our superconducting magnet. >>> >>> So far, I have deployed Erlang in this zone for two separate experiments (SNO+, a large particle physics >>> experiment in Canada) and Project 8 (a small nuclear physics experiment here in Seattle). Both times > have >>> been great successes, and I have found the reception of Erlang in this market to be great. In general, what >>> I have done is wrap a hardware management layer with some kind of outside world interface. For SNO+, we >>> used Webmachine and RESTful control, and for Project 8 we actually conduct all communication >>> by using CouchDB as a message passing interface. >>> >>> Physicists are suspicious creatures, but once you demonstrate the feature set that you get for practically >>> free with OTP, they see the advantage pretty quickly. On top of that, the development cycle for sophisticated >>> applications can be greatly reduced - more than once it made my group float to the top in terms of meeting >>> goals. >>> >>> In short, as far as I am concerned, Erlang has found a new niche in the world of Physics, and I intend to >>> spread the word as much as I can! >>> >>> Jared > Kofron >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jared.nance@REDACTED Thu Mar 15 01:33:39 2012 From: jared.nance@REDACTED (Jared Kofron) Date: Wed, 14 Mar 2012 17:33:39 -0700 Subject: [erlang-questions] Erlang meets physics In-Reply-To: References: Message-ID: <65BD3BF5-30D2-4323-98FF-A792C5051DEA@gmail.com> Yes, my first project was initially also done over ports - the async model was perfect. On Mar 13, 2012, at 5:58 PM, Robert Virding wrote: > We used ports. External hardware was a "process" with which you communicated through messages. The asynchronous model fitted hardware very well, at least the hardware in which we were interested. > > Robert > > How do you interact with the hardware? > Do you use GPIB C libr and wrap it with a NIF? > > From: Joe Armstrong > To: Jared Kofron > Cc: Erlang Questions > Sent: Tuesday, March 13, 2012 12:34 PM > Subject: Re: [erlang-questions] Erlang meets physics > > Great news - spread the word ! > > Just for the record Erlang programmers numbers 1 and 2 (ie myself and > Robert Virding) > are both ex physicists. > > When I lecture I often point out the similarity between causality and > message reception. > You don't know that something has happened until you get a message > telling that it has happened. > > (In physics it's a ray of light, or a photon, or something - > forgetting entanglement for the moment) > > In computing it's the reception of a message. > > As a ex physicist I know that we can't say anything about simultaneous > events occurring > at different places in space-time - turn this into computer science > and the same arguments > apply to things like making sure replicated data is consistent on > remote sites - well you can't > - at least if you want to change it - Brewer's CAP theorem applies - > which for a physicist makes > perfect sense. > > Also as an ex physicist I realize that things do actually happen in > parallel in the real world, > so modelling them in a sequential programming language (if I wanted to do that) > is big time crazy - just describe the parallel stuff in a concurrent > language and the program > writes itself. Wait a few years till we have million core computers > and the parallel problems > can be solved 1:1 on parallel computers - and programming simulations > and so on will be > really easy - but don't even think about doing it in a sequential language... > > Cheers > > /Joe > > > On Mon, Mar 12, 2012 at 2:34 AM, Jared Kofron wrote: > > Hi All, > > I've been using Erlang at work for a few years now, and I thought I'd throw my experience out there, as > > my application is a little different than what you usually see on the list - I am a graduate student at the > > Center for Nuclear Physics and Astrophysics at the University of Washington, and use Erlang extensively > > in my work. > > > > In my experience, something that Erlang is really great at but doesn't receive much attention for these days > > is managing and interacting with hardware. In any physics experiment of even modest sizes, you wind up > > having to keep track of the state of various pieces of equipment, often modify that state, and constantly > > interrogate particular values. For example, we might want to change the current in a magnetic trap, turn > > that trap off altogether, or simply read back the voltage drop across our superconducting magnet. > > > > So far, I have deployed Erlang in this zone for two separate experiments (SNO+, a large particle physics > > experiment in Canada) and Project 8 (a small nuclear physics experiment here in Seattle). Both times have > > been great successes, and I have found the reception of Erlang in this market to be great. In general, what > > I have done is wrap a hardware management layer with some kind of outside world interface. For SNO+, we > > used Webmachine and RESTful control, and for Project 8 we actually conduct all communication > > by using CouchDB as a message passing interface. > > > > Physicists are suspicious creatures, but once you demonstrate the feature set that you get for practically > > free with OTP, they see the advantage pretty quickly. On top of that, the development cycle for sophisticated > > applications can be greatly reduced - more than once it made my group float to the top in terms of meeting > > goals. > > > > In short, as far as I am concerned, Erlang has found a new niche in the world of Physics, and I intend to > > spread the word as much as I can! > > > > Jared Kofron > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From janburse@REDACTED Thu Mar 15 01:40:17 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 15 Mar 2012 01:40:17 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F6136E9.2040309@fastmail.fm> References: <4F606D90.5070206@fastmail.fm> <4F6136E9.2040309@fastmail.fm> Message-ID: <4F613A71.1010906@fastmail.fm> Jan Burse schrieb: > (The scaled column assumes that the 2430 number is CPU > speed; if that's not what it is, ignore that column.) Little complication: On the windows machine, it will go up in 100 MHz steps relatively fast (~20 ms ??) if a single core is only used. See Turbo Boost http://en.wikipedia.org/wiki/Intel_Turbo_Boost It will go to 3.0 GHz or on the machine I used. On another machine it usually goes up to 3.4 GHz. Thats why I was running the Java example twice: bench(6000). bench(6000). And got ~30ms and then ~15ms. But Erlang should be also subject to the Turbo Boost. When I did some other testing I used CPU-Z to see what the effective clock rate was. And it went up for SWI Prolog, GNU Prolog, etc.. equally well. Bye From janburse@REDACTED Thu Mar 15 01:44:26 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 15 Mar 2012 01:44:26 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F613A71.1010906@fastmail.fm> References: <4F606D90.5070206@fastmail.fm> <4F6136E9.2040309@fastmail.fm> <4F613A71.1010906@fastmail.fm> Message-ID: <4F613B6A.90002@fastmail.fm> Jan Burse schrieb: > (The scaled column assumes that the 2430 number is CPU > speed; if that's not what it is, ignore that column.) Scaling usually doesn't match if different architectures are used, i.e. 32-bit vs 64-bit. But otherwise I have even seen a perfect match between Linux and Windows on the same machine. Especially for the JVM, since it does its own memory management. Not sure if scaling matches if the machine changes. Tried to see in some instances the size of the on chip cache, but didn't see a big difference. What is perfectly seen is the CPU clock rate. Quite frightening when we think that Tablets have only 1 GHz or so... Bye From watson.timothy@REDACTED Thu Mar 15 02:27:12 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 15 Mar 2012 01:27:12 +0000 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F613B6A.90002@fastmail.fm> References: <4F606D90.5070206@fastmail.fm> <4F6136E9.2040309@fastmail.fm> <4F613A71.1010906@fastmail.fm> <4F613B6A.90002@fastmail.fm> Message-ID: On 15 March 2012 00:44, Jan Burse wrote: > Jan Burse schrieb: >> >> ?(The scaled column assumes that the 2430 number is CPU >> ?speed; if that's not what it is, ignore that column.) > > Scaling usually doesn't match if different architectures are > used, i.e. 32-bit vs 64-bit. > Tweaking of benchmarks aside though, they're not really indicative half the time. One of the classic shootout benchmarks, for example, has Haskell, Python (!) and even Lisp - maybe that's my ignorance about Lisp seeping out though - beating all the Java-7 implementations: http://shootout.alioth.debian.org/u64q/benchmark.php?test=pidigits&lang=all We could go on tweaking benchmarks forever and not learn all that much. Benchmarking and tuning running systems is much harder, more fun and profitable. I found your suggestion about native code generation (which I assume to be describing a JIT method similar to those used in the JVM/CLR) interesting though. Care to elaborate - it might bring out some of the finer points and enliven a discussion about what's good, bad and ugly about beam and where it is heading - lots of cool improvements keep happening all the time. From mfidelman@REDACTED Thu Mar 15 02:31:13 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Wed, 14 Mar 2012 21:31:13 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> References: <4F606D90.5070206@fastmail.fm> <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> Message-ID: <4F614661.6070105@meetinghouse.net> Richard O'Keefe wrote: > > There is often surprisingly little connection between the speed of a > _language_ (implementation) and the speed of _systems_ built using it. > > Which is almost the point - there's a rather direct correlation between the speed of a system, and how well the system architecture aligns with the constructs and run-time environment of the language it's implemented in. If you're building a system that is highly concurrent in nature, implementing it in Erlang is a big win, while implementing it in Java is a big loss. Yes, you can take something that's inherently concurrent, and design an efficient Java design, but you end up with something that's a conceptual mess. The example that comes to mind is simulation. In a previous life, I worked for a company that made military simulation software (massively multiplayer games for folks who shoot real bullets). Having a networking background, where the natural inclination is to spawn a process for every incoming task, I assumed that our simulators operated in a similar way - simulate 1000s tanks, spawn 1000 processes and do everything asynchronously. But no... as our coders informed me (I'm a systems guy), you can't do that in Java (or C++, which was the bulk of our code) - the context switching would bring everything to its knees. Instead, each tank was implemented as an object - with 4 main threads winding their way through every object, 40 times per second. The code was fast, but not surprisingly, it was amazingly brittle - change one object (say add a new weapon to a particular tank) and all kinds of things would have to be rewritten. My instinct that yes, indeed, one could implement each entity as an actor is what led me to discover Erlang - as pretty much the ONLY run-time environment optimized for massive concurrency. On the other hand, inherently serial applications probably run faster in languages and run-time environments optimized for small numbers of threads. Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From desired.mta@REDACTED Thu Mar 15 02:38:41 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Thu, 15 Mar 2012 01:38:41 +0000 Subject: [erlang-questions] Nitrogen on several ports In-Reply-To: References: Message-ID: On Wed, Mar 14, 2012 at 19:46, John Hughes wrote: > Hi Jesse, > > Thank you! That sounds like just what I need. I didn't realise wf:socket() > would tell me the socket type--very useful! > > I am actually using Apache as a reverse proxy to forward port 80 to 8000 > already... my problem is I want to access the *same page* (i.e. same .erl > file) through http and https, and see different options in each case. I > suppose I could make two different pages, but that would be more complex and > less maintainable however I did it (code duplication or a more complex > module structure). I'm not sure nginx would really solve that problem for > me. Depends where you want routing decisions to be handled. In front webserver (nginx) or in Erlang application? If in a-la-nginx, simply configure access rules for both http and https virtualhosts individually. If nitrogen case, I would think about doing the Erlang application on a single http port, and using a real web server to make "real" 80 and 443. And append some kind of header in each case (X-port: 80/443 for instance). I don't know the exact syntax for web servers, but should be easy to figure it out. Motiejus Jak?tys From gumm@REDACTED Thu Mar 15 04:09:07 2012 From: gumm@REDACTED (Jesse Gumm) Date: Wed, 14 Mar 2012 22:09:07 -0500 Subject: [erlang-questions] Nitrogen on several ports In-Reply-To: References: Message-ID: Further, if you need to find out if the client is connecting with ssl, you can add an nginx (or apache) rule to add a custom header for clients connecting through ssl, then check for the existence of the header in nitrogen. The main reason I'd do that is to minimize the number of servers with the ssl cert installed. Though in the long run, for performance, you can switch all static files to be served from nginx/apache, and leave all the dynamic stuff to nitrogen/inets. But if this isn't exactly a high performance app, then most of this of overkill anyway. -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 www.sigma-star.com @jessegumm On Mar 14, 2012 8:39 PM, "Motiejus Jak?tys" wrote: > On Wed, Mar 14, 2012 at 19:46, John Hughes wrote: > > Hi Jesse, > > > > Thank you! That sounds like just what I need. I didn't realise > wf:socket() > > would tell me the socket type--very useful! > > > > I am actually using Apache as a reverse proxy to forward port 80 to 8000 > > already... my problem is I want to access the *same page* (i.e. same .erl > > file) through http and https, and see different options in each case. I > > suppose I could make two different pages, but that would be more complex > and > > less maintainable however I did it (code duplication or a more complex > > module structure). I'm not sure nginx would really solve that problem for > > me. > > Depends where you want routing decisions to be handled. In front > webserver (nginx) or in Erlang application? > > If in a-la-nginx, simply configure access rules for both http and > https virtualhosts individually. > If nitrogen case, I would think about doing the Erlang application on > a single http port, and using a real web server to make "real" 80 and > 443. And append some kind of header in each case (X-port: 80/443 for > instance). > > I don't know the exact syntax for web servers, but should be easy to > figure it out. > > Motiejus Jak?tys > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Mar 15 04:31:04 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 15 Mar 2012 16:31:04 +1300 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F613A71.1010906@fastmail.fm> References: <4F606D90.5070206@fastmail.fm> <4F6136E9.2040309@fastmail.fm> <4F613A71.1010906@fastmail.fm> Message-ID: <6CCF61DD-571F-4274-B239-A62C17B79D5D@cs.otago.ac.nz> [about Turbo Boost] Sadly, the graphical Turbo Boost monitor from Intel is only available for Windows. Well, OK, I have Windows on the laptop as well. But no, it's only available for Windows 7, and I have Vista. Oh the misery. However, the speed effect is indeed clearly visible, which is why my test programs do 6000,6000,20000,20000,1000000,1000000 runs, for both languages. This is of course another reason why micro-benchmarks of this kind tell us very little. Even using the performance counters to count cycles (I *do* like the 'ptime' command in Solaris) is problematic, because cores and memories no longer have a fixed speed relationship. From sunwood360@REDACTED Thu Mar 15 08:01:59 2012 From: sunwood360@REDACTED (envelopes envelopes) Date: Thu, 15 Mar 2012 00:01:59 -0700 Subject: [erlang-questions] erljs - Run Erlang in JavaScript Message-ID: https://github.com/baryluk/erljs an interesting project -------------- next part -------------- An HTML attachment was scrubbed... URL: From paperless@REDACTED Thu Mar 15 09:01:39 2012 From: paperless@REDACTED (Tim McNamara) Date: Thu, 15 Mar 2012 21:01:39 +1300 Subject: [erlang-questions] [OT] Re: GPL vs. whatever [was: Erlang UUID] In-Reply-To: <4F5D736A.8070901@meetinghouse.net> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> <4F5D736A.8070901@meetinghouse.net> Message-ID: If you're interested in using this approach for your business model, then the AGPL can be a better option. On 12 March 2012 16:54, Miles Fidelman wrote: > Just to throw in a different aspect of the GPL vs. BSD discussion: > > IMHO, GPL is a far better license than BSD for a developer that intends to > commercialize a product. > > The initial developer (copyright holder) always has the option to release > code under a dual license - GPL, BSD, or whatever for an open source > release, something more restrictive for the commercial product (potentially > with proprietary extensions). > > With GPL, you pretty much eliminate any competition - anybody else who > extends the code is faced with copyleft considerations, they CAN'T take your > code, combine it with their own code, and slap a proprietary license around > the assemblage. ?With BSD, or Apache, (or LGPL for that matter), they can. > > Of course, if you dual-license your code under GPL and a proprietary > license, things can come back to haunt you if you want to incorporate > community-generated extensions into your upstream code base. ?In that case > the GPL and copyleft apply to you. > > -- > In theory, there is no difference between theory and practice. > In practice, there is. ? .... Yogi Berra > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From janburse@REDACTED Thu Mar 15 09:01:50 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 15 Mar 2012 09:01:50 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <6CCF61DD-571F-4274-B239-A62C17B79D5D@cs.otago.ac.nz> References: <4F606D90.5070206@fastmail.fm> <4F6136E9.2040309@fastmail.fm> <4F613A71.1010906@fastmail.fm> <6CCF61DD-571F-4274-B239-A62C17B79D5D@cs.otago.ac.nz> Message-ID: <4F61A1EE.8050307@fastmail.fm> Richard O'Keefe schrieb: > This is of course another reason why micro-benchmarks of this kind tell > us very little. Even using the performance counters to count cycles (I > *do* like the 'ptime' command in Solaris) is problematic, because cores > and memories no longer have a fixed speed relationship. To benchmark, or not to benchmark, what was the question? Happy PI Day: http://www.youtube.com/watch?v=uXoh6vi6J5U From torben.lehoff@REDACTED Thu Mar 15 09:52:43 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Thu, 15 Mar 2012 09:52:43 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F614661.6070105@meetinghouse.net> References: <4F606D90.5070206@fastmail.fm> <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> <4F614661.6070105@meetinghouse.net> Message-ID: <4F61ADDB.6010604@gmail.com> On 15/3/12 2:31 , Miles Fidelman wrote: > Richard O'Keefe wrote: >> >> There is often surprisingly little connection between the speed of a >> _language_ (implementation) and the speed of _systems_ built using it. >> >> > Which is almost the point - there's a rather direct correlation > between the speed of a system, and how well the system architecture > aligns with the constructs and run-time environment of the language > it's implemented in. > > If you're building a system that is highly concurrent in nature, > implementing it in Erlang is a big win, while implementing it in Java > is a big loss. Yes, you can take something that's inherently > concurrent, and design an efficient Java design, but you end up with > something that's a conceptual mess. That is a very good point and rhymes well with the maxim of choosing the right tool for the job at hand... something that can be extremely difficult for all of us since we all have our favourite tool. Hint: learn more than one tool! It triggers me to take a step back and take another stab at the original topic of this thread. When I did my grass root (read guerilla warfare) work to be allowed to use Erlang for a project in Motorola the question of performance never became a serious issue. Why not? Because it was easy to see that if Ericsson could write telephone switches in Erlang it would have sufficient horsepower to do the things needed in another telecom system. The big worry was: "Who is using Erlang?" which comes back to the "Nobody has been fired for choosing Microsoft | IBM | Oracle"-maxim. Eventually I managed to gather enough evidence through the Erlang community (I am eternally grateful for the not-to-be-made-public inputs that I received from other Erlang warriors) to convince management that it would not be a dead-end to try out Erlang. And now I will allow myself to digress from the original topic for a very personal experience... working with Erlang has been the best experience of my professional life. Why? Because it was simply the right tool for the job at hand! It allowed us to get a lot done since the semantic gap between the domain and the programming language was so small. Not only did we get a lot done - we also had very few errors and when we had errors it was very easy to debug, fix and an re-deploy them. (During Interoperability testing with our competitors we had turn-around times on bugs of 15 minutes versus their 1 day... I rest my case!). So I am all for Joe's approach to project funding: get a prototype going and then evolve it into a product in small increments. You will get a prototype very fast - this mailing list is great for advice on how to wire things together and it is not that difficult to get a quick'n'dirty solution done in Erlang even without being an expert. If you want to do something cool with Erlang you have burn for it - at least if you are not deciding what you should work on direcly - and then go and execute it! Enjoy the ride!! Torben > > The example that comes to mind is simulation. In a previous life, I > worked for a company that made military simulation software (massively > multiplayer games for folks who shoot real bullets). > > Having a networking background, where the natural inclination is to > spawn a process for every incoming task, I assumed that our simulators > operated in a similar way - simulate 1000s tanks, spawn 1000 processes > and do everything asynchronously. But no... as our coders informed me > (I'm a systems guy), you can't do that in Java (or C++, which was the > bulk of our code) - the context switching would bring everything to > its knees. > > Instead, each tank was implemented as an object - with 4 main threads > winding their way through every object, 40 times per second. The code > was fast, but not surprisingly, it was amazingly brittle - change one > object (say add a new weapon to a particular tank) and all kinds of > things would have to be rewritten. > > My instinct that yes, indeed, one could implement each entity as an > actor is what led me to discover Erlang - as pretty much the ONLY > run-time environment optimized for massive concurrency. > > On the other hand, inherently serial applications probably run faster > in languages and run-time environments optimized for small numbers of > threads. > > Miles Fidelman > > > -- http://www.linkedin.com/in/torbenhoffmann From janburse@REDACTED Thu Mar 15 09:53:07 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 15 Mar 2012 09:53:07 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F606D90.5070206@fastmail.fm> <4F6136E9.2040309@fastmail.fm> <4F613A71.1010906@fastmail.fm> <4F613B6A.90002@fastmail.fm> Message-ID: <4F61ADF3.5040204@fastmail.fm> Tim Watson schrieb: > http://shootout.alioth.debian.org/u64q/benchmark.php?test=pidigits&lang=all Seems that shootout is a benchmark and a programming contest at the same time. The Erlang PI solution, which ranks after the Java solution, uses only one process for the computation (plus one for IO), whereas the Java solution uses 3 threads for the computation. That many of the Java solutions are multi-threaded is seen by the ratio of CPU secs and Elapsed sec. But interestingly the average ratio for Erlang is also high. I guess the ratio is rather an effect of the programmed solution design, than of the compiler backend or language runtime system. For example the Erlang fannkuch-redux spawns 12 processes. Here are the ratios and the total times: Java 7 -server Erlang HiPE CPU Elapsed Ratio CPU Elapsed Ratio k-nucleotide 28.41 7.80 3.6 384.87 138.94 2.8 mandelbrot 27.49 7.01 3.9 266.21 75.90 3.5 reverse-complement 2.67 1.27 2.1 20.36 10.32 2.0 fannkuch-redux 68.93 17.45 4.0 503.82 126.59 4.0 fasta 5.26 5.18 1.0 31.07 31.07 1.0 n-body 22.51 22.49 1.0 129.44 129.48 1.0 pidigits 8.57 3.86 2.2 16.28 15.13 1.1 binary-trees 18.02 8.66 2.1 72.11 28.15 2.6 regex-dna 36.57 12.35 3.0 150.17 40.10 3.7 spectral-norm 17.57 4.51 3.9 47.65 12.12 3.9 Total 236.00 90.58 2.6 1621.98 607.80 2.7 (Source: http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=java&lang2=hipe) But when the programming is similar, i.e. high ratio, one might guess that the tests then boil down to a comparison of the compiler backend and the language runtime system. The total elapsed time 607.80 secs Erlang vs 90.58 secs Java reminds me of the naive reverse bachmark I did, i.e. 60ms vs 15ms (turbo). But its a little worse, and I didn't use Erlang HIPE. The figures for Erlang are much worse. So micro benchmarks can give you a first impression of what one has to expect from non-micro benchmarks. Kind of love at first sight, and then get disappointed? But the above benchmarks would sure turn my consumer box into a frying pan (ventilator kicking in), with all this parallelism. Bye From erlang@REDACTED Thu Mar 15 10:52:12 2012 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 15 Mar 2012 10:52:12 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F61ADDB.6010604@gmail.com> References: <4F606D90.5070206@fastmail.fm> <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> <4F614661.6070105@meetinghouse.net> <4F61ADDB.6010604@gmail.com> Message-ID: On Thu, Mar 15, 2012 at 9:52 AM, Torben Hoffmann wrote: > > > On 15/3/12 2:31 , Miles Fidelman wrote: >> >> Richard O'Keefe wrote: >>> >>> >>> There is often surprisingly little connection between the speed of a >>> _language_ (implementation) and the speed of _systems_ built using it. >>> >>> >> Which is almost the point - there's a rather direct correlation between >> the speed of a system, and how well the system architecture aligns with the >> constructs and run-time environment of the language it's implemented in. >> >> If you're building a system that is highly concurrent in nature, >> implementing it in Erlang is a big win, while implementing it in Java is a >> big loss. ?Yes, you can take something that's inherently concurrent, and >> design an efficient Java design, but you end up with something that's a >> conceptual mess. > > That is a very good point and rhymes well with the maxim of choosing the > right tool for the job at hand... something that can be extremely difficult > for all of us since we all have our favourite tool. Hint: learn more than > one tool! > > It triggers me to take a step back and take another stab at the original > topic of this thread. > > When I did my grass root (read guerilla warfare) work to be allowed to use > Erlang for a project in Motorola the question of performance never became a > serious issue. > > Why not? Because it was easy to see that if Ericsson could write telephone > switches in Erlang it would have sufficient horsepower to do the things > needed in another telecom system. > > The big worry was: "Who is using Erlang?" which comes back to the "Nobody > has been fired for choosing Microsoft | IBM | Oracle"-maxim. Eventually I > managed to gather enough evidence through the Erlang community (I am > eternally grateful for the not-to-be-made-public inputs that I received from > other Erlang warriors) to convince management that it would not be a > dead-end to try out Erlang. > > And now I will allow myself to digress from the original topic for a very > personal experience... working with Erlang has been the best experience of > my professional life. > Why? Because it was simply the right tool for the job at hand! It allowed us > to get a lot done since the semantic gap between the domain and the > programming language was so small. Not only did we get a lot done - we also > had very few errors and when we had errors it was very easy to debug, fix > and an re-deploy them. (During Interoperability testing with our competitors > we had turn-around times on bugs of 15 minutes versus their 1 day... I rest > my case!). > > So I am all for Joe's approach to project funding: get a prototype going and > then evolve it into a product in small increments. You will get a prototype > very fast - this mailing list is great for advice on how to wire things > together and it is not that difficult to get a quick'n'dirty solution done > in Erlang even without being an expert. Actually this is how to make money :-) "All" you have to do is: a) - choose a new and "likely to be trendy" standard b) - implement it *all* in Erlang c) - do significant work with the standardization committee d) - make a product e) - give the product away or sell it f) - sell support Examples of this are (xmpp - process one) (netconf - tail-f) (AMQP - rabbit MQ) (this is a list of (protocol, company) pairs) The tricky bit is choosing a) (many choices) b) (more tricky than you think) and in the c) .. f) parts much can go wrong. Note - the above three standards all involved communication and protocol implementation which Erlang is pretty good at. /Joe > If you want to do something cool with Erlang you have burn for it - at least > if you are not deciding what you should work on direcly - and then go and > execute it! > > Enjoy the ride!! > Torben > > >> >> The example that comes to mind is simulation. ?In a previous life, I >> worked for a company that made military simulation software (massively >> multiplayer games for folks who shoot real bullets). >> >> Having a networking background, where the natural inclination is to spawn >> a process for every incoming task, I assumed that our simulators operated in >> a similar way - simulate 1000s tanks, spawn 1000 processes and do everything >> asynchronously. ?But no... as our coders informed me (I'm a systems guy), >> you can't do that in Java (or C++, which was the bulk of our code) - the >> context switching would bring everything to its knees. >> >> Instead, each tank was implemented as an object - with 4 main threads >> winding their way through every object, 40 times per second. ?The code was >> fast, but not surprisingly, it was amazingly brittle - change one object >> (say add a new weapon to a particular tank) and all kinds of things would >> have to be rewritten. >> >> My instinct that yes, indeed, one could implement each entity as an actor >> is what led me to discover Erlang - as pretty much the ONLY run-time >> environment optimized for massive concurrency. >> >> On the other hand, inherently serial applications probably run faster in >> languages and run-time environments optimized for small numbers of threads. >> >> Miles Fidelman >> >> >> > > -- > http://www.linkedin.com/in/torbenhoffmann > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Thu Mar 15 11:17:43 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 15 Mar 2012 11:17:43 +0100 Subject: [erlang-questions] More "eper" help or tutos In-Reply-To: References: <1C73706A-9C51-4B30-A3F5-1850F70FF257@gmail.com> Message-ID: <5CF40ED3-75DC-43F2-804F-829AA4D1FB81@gmail.com> OK I see. Do you know any other tool equivalent to "eper"? Regards, Zabrane On Mar 15, 2012, at 11:15 AM, mats cronqvist wrote: > the documentation is pretty sucky. And it will not improve in the short run :< > sherk is somewhat rotted... doesn't work well on multicore machines. > > On Tue, Mar 6, 2012 at 3:21 AM, Zabrane Mickael wrote: > Hi Mats, Hi guys, > > Is there any elaborated tuto about eper: > http://code.google.com/p/eper/ > > The only one I found was a small but useful wiki page about redbug: > http://code.google.com/p/eper/wiki/redbug > > I'm interested to learn how to use gperf and sherk > > Help or pointers appreciated! > > Regards, > Zabrane > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.melin@REDACTED Thu Mar 15 11:40:22 2012 From: per.melin@REDACTED (Per Melin) Date: Thu, 15 Mar 2012 11:40:22 +0100 Subject: [erlang-questions] [ANN] Brim - HTML templating library Message-ID: <6E4F3076-B078-4C47-B33E-561B43A1EEE2@gmail.com> Brim is inspired by Enlive for Clojure, meaning that the HTML templates are completely free of the embedded code, annotations, special attributes, etc that we usually see. It also uses CSS selectors to reference the parts of the HTML we want to transform with Erlang. https://github.com/permelin/brim It is version 0.0.1 and while usable it's too early to be very useful yet. I'm announcing it anyway because I really want input on the API, which I am struggling with. An example (since there is no documentation): A web page that shows a table with the applications running on our Erlang node. Here is the template, filled with some dummy text so that we can test what it will look like in a browser.
node name will go here
application yada yada yada version
Then the code that will transform the above HTML and fill it with data. -module(running_apps). -export([html/0]). html() -> % Read the template Doc0 = brim:document("priv/apps.html"), % Set the caption to the output from node() Doc1 = brim:content(Doc0, "#apps > caption", node()), % Fill the table tbody -- see rows/1 below Doc2 = brim:content(Doc1, "#apps > tbody", rows(Doc1)), % And to really show off the CSS selector support we'll add the % class "pointless" on all whose id does not start with "ker" Doc3 = brim:add_class(Doc2, "tr:not([id^=ker])", "pointless"), % Print to screen -- use brim:render/1 instead to get io list brim:print(Doc3). rows(Doc) -> % Create one clone of the for each item returned from % application:which_application() and transform them as specified % by the fun. brim:map( Doc, "#apps tr", application:which_applications(), fun(TR0, {App, Description, Version}) -> TR1 = brim:content(TR0, "td:first-child", App), TR2 = brim:content(TR1, "td:nth-child(2)", Description), TR3 = brim:content(TR2, "td:last-child", Version), % And finally set the id of each to the name of the % corresponding application, for no good reason brim:id(TR3, "tr", atom_to_list(App)) end). And here is the output (formatted for readability):
nonode@REDACTED
inets INETS CXC 138 49 5.7.1
stdlib ERTS CXC 138 10 1.17.5
kernel ERTS CXC 138 10 2.14.5
As you can see, there is a lot of explicit state threading (Doc0, Doc1, Doc2, Doc3, TR0, TR1, TR2, TR3) which I fear can get annoying when dealing with larger templates. I'm trying out different ways to minimize that (and verbosity in general). The most obvious (to me) is to move the state argument to last position to facilitate chaining. html() -> Doc = brim:document("priv/apps.html"), brim:print( brim:add_class("tr:not([id^=ker])", "pointless", brim:content("#apps > caption", node(), brim:content("#apps > tbody", app_rows(Doc), Doc)))). Another alternative below. But I think it has too many limitations to be feasible. html() -> Doc = brim:document("priv/apps.html"), brim:do(Doc, [brim:content("#apps > caption", node()), brim:content("#apps > tbody", app_rows(Doc)), brim:add_class("tr:not([id^=ker])", "pointless"), brim:print()]). As I said, all input is appreciated. From koops.j@REDACTED Thu Mar 15 11:44:00 2012 From: koops.j@REDACTED (Jeroen Koops) Date: Thu, 15 Mar 2012 11:44:00 +0100 Subject: [erlang-questions] Documentation error in Diameter AVP specification? Message-ID: Hi all, In the documentation for the Diameter dict-file format, at http://www.erlang.org/doc/man/diameter_dict.html, it says, under the @messages tag: > Defines the messages of the application. The section content consists of definitions of the form specified in > section 3.2 of RFC 3588, "Command Code ABNF specification". The examples given show a diameter specified as follows: < Diameter Header: 287, REQ, PXY > However, RFC3588 specifies that a header should be specified as: header = "<" Diameter-Header:" command-id [r-bit] [p-bit] [e-bit] [application-id]">" Note the dash in Diameter-Header. Using a dash in a .dia file causes an error when compiling the file with diameterc, so it seems that diameterc does not completely follow RFC3588. Am I misunderstanding something here, or is this a bug for which I can submit a patch? The best patch I can think of is to modify diameterc in such a way that both 'Diameter-Header' and 'Diameter Header' are accepted, with a note in the document saying that 'Diameter Header' is accepted but deprecated. An easier patch would jus add a note to the documentation pointing out the difference with the RFC. -- Jeroen Koops M: koops.j@REDACTED T: +31-6-55590300 -------------- next part -------------- An HTML attachment was scrubbed... URL: From avinash@REDACTED Thu Mar 15 11:49:01 2012 From: avinash@REDACTED (Avinash Dhumane) Date: Thu, 15 Mar 2012 16:19:01 +0530 Subject: [erlang-questions] ODBC Connectivity with Netezza Message-ID: We have an integration glue application in Erlang (on Windows) that connects to heterogeneous data sources SQL Server, AS/400, and Netezza via ODBC application provided in the standard Erlang distribution. While the drill works perfectly for SQL Server and AS/400, Netezza is encountering following connectivity problem: Erlang R14B02 (erts-5.8.3) [smp:2:2] [rq:2] [async-threads:0] Eshell V5.8.3 (abort with ^G) 1> odbc:start(). ok 2> odbc:connect("DSN=NZSQL;UID=TEST;PWD=TEST", []). {error,"[Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified SQLSTATE IS: IM002 Connection to database failed."} 3> Microsoft help link: http://msdn.microsoft.com/en-us/library/windows/desktop/ms711810(v=vs.85).aspx says: If the data source name cannot be found or ServerName is a null pointer, and the default data source specification does not exist, the Driver Manager returns SQL_ERROR with SQLSTATE IM002 (Data source name not found and no default driver specified). Obviously, this is not the case with our setup. We tried with both "User DSN" and "System DSN" in Windows "ODBC Data Source Administrator" tool, but the same error persists in both the cases. Does Netezza require different connection string? Please advise. Thanks Avinash PS: We have NetezzaSQL driver (NSSQLODBC.DLL) version 4.06.05.10119 installed on the Windows box that runs the Erlang application. From michael.eugene.turner@REDACTED Thu Mar 15 12:30:09 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 15 Mar 2012 20:30:09 +0900 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F606D90.5070206@fastmail.fm> <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> <4F614661.6070105@meetinghouse.net> <4F61ADDB.6010604@gmail.com> Message-ID: Joe writes: > "All" you have to do is: > > ? a) - choose a new and "likely to be trendy" standard > ? b) - implement it *all* in Erlang > ? c) - do significant work with the standardization committee > ? d) - make a product > ? e) - give the product away or sell it > ? f) - sell support > > Examples of this are (xmpp - process one) (netconf - tail-f) (AMQP - rabbit MQ) > (this is a list of (protocol, company) pairs) > > The tricky bit is choosing a) (many choices) b) (more tricky than you > think) and in the c) .. f) parts much can go wrong. (a) A long time ago, in a galaxy far, far away, a company called Autodesk was just a gang of friends who all chose to hack on different product ideas, while also pledging fealty to whichever idea really took off. What took off was Autocad. I think all those Autodesk startup people got rich. So if a bunch of good Erlang programmers got together under a similar agreement, and each chose a different "likely to be trendy" standard, the chances of catching a wave might be greatly improved. Erlang's features make each programmer significantly more productive than others attempting the same with less-suitable languages/platforms, so the intuitively obvious "dilution of focus" counter-argument isn't quite as strong here. (b) Let's say they all worked in a collegial "open door" manner where they could pick each others' brains. This would be more likely if they all keep each other briefed on their progress at regularly scheduled demos and presentations, and always at least went to lunch together several times a week even if they mostly worked at home. Then the risks in "implement it *all* in Erlang" might also be reduced: *somebody* in the group is likely to know how to do what you don't know how to do. (c) Let's say this group appoints one of their number to be nothing but a liaison to standards committees. (There are people who actually thrive on this kind of work, believe it or not.) This reduces the effort per programmer on the legwork and wrangling related to staying abreast of standards and contributing to them. (d,e,f) Many a startup has come to grief on issues of execution. If your pick for the multi-tasking liaison is also someone with significant product management experience in protocol software products, you've got a key member who gives your startup a fighting chance at defying the harrowing infant-mortality statistics for young companies. What happens when the clear winner emerges among the varying efforts? Your standards-committee liaison is almost completely up to speed on a whole spectrum of issues in marketing, product management, technical publications, etc. for the *specific* idea that has become the company's main focus. Maybe not a CEO type. But he/she might do until the real thing comes along. And venture capitalists are often only too happy to give you "the real thing," as a condition of first-round financing. (But watch out for that, because sometimes they regard certain fledgling endeavors as little more than training wheels or reality tests for startup-CEO-wannabe types -- i.e., the real "product" for them might be the CEO pick, with your company's failure supposedly at least providing some instructive and/or character-building education.) ... is how I'd strategize it, anyway. -michael turner On Thu, Mar 15, 2012 at 6:52 PM, Joe Armstrong wrote: > On Thu, Mar 15, 2012 at 9:52 AM, Torben Hoffmann > wrote: >> >> >> On 15/3/12 2:31 , Miles Fidelman wrote: >>> >>> Richard O'Keefe wrote: >>>> >>>> >>>> There is often surprisingly little connection between the speed of a >>>> _language_ (implementation) and the speed of _systems_ built using it. >>>> >>>> >>> Which is almost the point - there's a rather direct correlation between >>> the speed of a system, and how well the system architecture aligns with the >>> constructs and run-time environment of the language it's implemented in. >>> >>> If you're building a system that is highly concurrent in nature, >>> implementing it in Erlang is a big win, while implementing it in Java is a >>> big loss. ?Yes, you can take something that's inherently concurrent, and >>> design an efficient Java design, but you end up with something that's a >>> conceptual mess. >> >> That is a very good point and rhymes well with the maxim of choosing the >> right tool for the job at hand... something that can be extremely difficult >> for all of us since we all have our favourite tool. Hint: learn more than >> one tool! >> >> It triggers me to take a step back and take another stab at the original >> topic of this thread. >> >> When I did my grass root (read guerilla warfare) work to be allowed to use >> Erlang for a project in Motorola the question of performance never became a >> serious issue. >> >> Why not? Because it was easy to see that if Ericsson could write telephone >> switches in Erlang it would have sufficient horsepower to do the things >> needed in another telecom system. >> >> The big worry was: "Who is using Erlang?" which comes back to the "Nobody >> has been fired for choosing Microsoft | IBM | Oracle"-maxim. Eventually I >> managed to gather enough evidence through the Erlang community (I am >> eternally grateful for the not-to-be-made-public inputs that I received from >> other Erlang warriors) to convince management that it would not be a >> dead-end to try out Erlang. >> >> And now I will allow myself to digress from the original topic for a very >> personal experience... working with Erlang has been the best experience of >> my professional life. >> Why? Because it was simply the right tool for the job at hand! It allowed us >> to get a lot done since the semantic gap between the domain and the >> programming language was so small. Not only did we get a lot done - we also >> had very few errors and when we had errors it was very easy to debug, fix >> and an re-deploy them. (During Interoperability testing with our competitors >> we had turn-around times on bugs of 15 minutes versus their 1 day... I rest >> my case!). >> >> So I am all for Joe's approach to project funding: get a prototype going and >> then evolve it into a product in small increments. You will get a prototype >> very fast - this mailing list is great for advice on how to wire things >> together and it is not that difficult to get a quick'n'dirty solution done >> in Erlang even without being an expert. > > Actually this is how to make money :-) > > "All" you have to do is: > > ? a) - choose a new and "likely to be trendy" standard > ? b) - implement it *all* in Erlang > ? c) - do significant work with the standardization committee > ? d) - make a product > ? e) - give the product away or sell it > ? f) - sell support > > Examples of this are (xmpp - process one) (netconf - tail-f) (AMQP - rabbit MQ) > (this is a list of (protocol, company) pairs) > > The tricky bit is choosing a) (many choices) b) (more tricky than you > think) and in the c) .. f) parts much can go wrong. > > Note - the above three standards all involved communication and > protocol implementation > which Erlang is pretty good at. > > /Joe > >> If you want to do something cool with Erlang you have burn for it - at least >> if you are not deciding what you should work on direcly - and then go and >> execute it! >> >> Enjoy the ride!! >> Torben >> >> >>> >>> The example that comes to mind is simulation. ?In a previous life, I >>> worked for a company that made military simulation software (massively >>> multiplayer games for folks who shoot real bullets). >>> >>> Having a networking background, where the natural inclination is to spawn >>> a process for every incoming task, I assumed that our simulators operated in >>> a similar way - simulate 1000s tanks, spawn 1000 processes and do everything >>> asynchronously. ?But no... as our coders informed me (I'm a systems guy), >>> you can't do that in Java (or C++, which was the bulk of our code) - the >>> context switching would bring everything to its knees. >>> >>> Instead, each tank was implemented as an object - with 4 main threads >>> winding their way through every object, 40 times per second. ?The code was >>> fast, but not surprisingly, it was amazingly brittle - change one object >>> (say add a new weapon to a particular tank) and all kinds of things would >>> have to be rewritten. >>> >>> My instinct that yes, indeed, one could implement each entity as an actor >>> is what led me to discover Erlang - as pretty much the ONLY run-time >>> environment optimized for massive concurrency. >>> >>> On the other hand, inherently serial applications probably run faster in >>> languages and run-time environments optimized for small numbers of threads. >>> >>> Miles Fidelman >>> >>> >>> >> >> -- >> http://www.linkedin.com/in/torbenhoffmann >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.eugene.turner@REDACTED Thu Mar 15 13:17:01 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 15 Mar 2012 21:17:01 +0900 Subject: [erlang-questions] [ANN] Brim - HTML templating library In-Reply-To: <6E4F3076-B078-4C47-B33E-561B43A1EEE2@gmail.com> References: <6E4F3076-B078-4C47-B33E-561B43A1EEE2@gmail.com> Message-ID: I think this sort of thing will always be ugly, no matter how you do it. My suggestion is to hide the ugliness down in a smart algorithm that figures out some of the more obvious things, and requires relatively unobtrusive hints for the subtler and more error-prone transformations. The idea of a pure-vanilla-HTML template is really just one half of an idea that used to be called "Programming by Example." )PBE) You provide sample input and corresponding sample output. Then some magical thingie infers what you (probably) want to do. It figures out some rules or code or both, to make appropriate output happen more routinely, for other input of the same form. Dan Halbert pioneered PBE at Xerox PARC. It was later taken up as a research project at at Apple. Then it was championed by none other than Melinda Gates, at Microsoft. It never took off. Not for lack of trying: we hapless Windows users got Microsoft Bob, then "Clippy," then that annoying dolphin swimming around on our desktops. In each case, I think the idea came to grief because of Dan Halbert's utopian ambition: to make Programming by Example "programming for the masses" -- no code skills required. But what if code skills *were* required? Start simple: here's your HTML target. The titleThe body Here's an Erlang input term: { <<"The title">>, <<"The body">> } How would you write code that took those two as input and figured out how to convert all terms of that form to HTML like that target? The matching problem might not be so hard for simple, constant-length data structures. But more often, on the Web, we want to generate content dynamically, not just hand a static data structure to a translator. This is where PBE went astray: by making it an active agent snooping in on the GUI, one that watches a user, and tries to guess the next iteration of what seems to be a loop. Result: one annoyed user. (In the case of programmers, one *really* annoyed user.) What would a programmer prefer? Perhaps some sort of parse transform that sees hints in the generator code about beginnings and endings, and substitutes calls that generate part of the appropriate HTML. Or maybe the API for the input-output matching algorithm might have "hint" calls that you could sprinkle into your code until algorithm "got it." Anyway, I'm just hinting at a possible approach. I'm a little concerned, because at this point it looks like you're starting to implement a whole little language, with code passed in string parameters (e.g., "tr:not([id^=ker])". Erlang offers so much power as a language itself, it seems there really should be some way to leverage it, so that the user is required to learn only a small Erlang API, not a whole embedded mini-language. -michael turner On Thu, Mar 15, 2012 at 7:40 PM, Per Melin wrote: > Brim is inspired by Enlive for Clojure, meaning that the HTML templates are completely free of the embedded code, annotations, special attributes, etc that we usually see. It also uses CSS selectors to reference the parts of the HTML we want to transform with Erlang. > > https://github.com/permelin/brim > > It is version 0.0.1 and while usable it's too early to be very useful yet. I'm announcing it anyway because I really want input on the API, which I am struggling with. > > An example (since there is no documentation): A web page that shows a table with the applications running on our Erlang node. Here is the template, filled with some dummy text so that we can test what it will look like in a browser. > > > ? > ? > ? ? > ? ? ? > ? ? ? > ? ? ? > ? ? > ? >
node name will go here
applicationyada yada yadaversion
> > > Then the code that will transform the above HTML and fill it with data. > > -module(running_apps). > > -export([html/0]). > > html() -> > ? ?% Read the template > ? ?Doc0 = brim:document("priv/apps.html"), > > ? ?% Set the caption to the output from node() > ? ?Doc1 = brim:content(Doc0, "#apps > caption", node()), > > ? ?% Fill the table tbody -- see rows/1 below > ? ?Doc2 = brim:content(Doc1, "#apps > tbody", rows(Doc1)), > > ? ?% And to really show off the CSS selector support we'll add the > ? ?% class "pointless" on all whose id does not start with "ker" > ? ?Doc3 = brim:add_class(Doc2, "tr:not([id^=ker])", "pointless"), > > ? ?% Print to screen -- use brim:render/1 instead to get io list > ? ?brim:print(Doc3). > > rows(Doc) -> > ? ?% Create one clone of the for each item returned from > ? ?% application:which_application() and transform them as specified > ? ?% by the fun. > ? ?brim:map( > ? ? ? ?Doc, "#apps tr", application:which_applications(), > ? ? ? ?fun(TR0, {App, Description, Version}) -> > ? ? ? ? ? ?TR1 = brim:content(TR0, "td:first-child", App), > ? ? ? ? ? ?TR2 = brim:content(TR1, "td:nth-child(2)", Description), > ? ? ? ? ? ?TR3 = brim:content(TR2, "td:last-child", Version), > ? ? ? ? ? ?% And finally set the id of each to the name of the > ? ? ? ? ? ?% corresponding application, for no good reason > ? ? ? ? ? ?brim:id(TR3, "tr", atom_to_list(App)) > ? ? ? ?end). > > > And here is the output (formatted for readability): > > > ? > ? > ? ? > ? ? ? > ? ? ? > ? ? ? > ? ? > ? ? > ? ? ? > ? ? ? > ? ? ? > ? ? > ? ? > ? ? ? > ? ? ? > ? ? ? > ? ? > ? >
nonode@REDACTED
inetsINETS ?CXC 138 495.7.1
stdlibERTS ?CXC 138 101.17.5
kernelERTS ?CXC 138 102.14.5
> > > As you can see, there is a lot of explicit state threading (Doc0, Doc1, Doc2, Doc3, TR0, TR1, TR2, TR3) which I fear can get annoying when dealing with larger templates. I'm trying out different ways to minimize that (and verbosity in general). The most obvious (to me) is to move the state argument to last position to facilitate chaining. > > html() -> > ? ?Doc = brim:document("priv/apps.html"), > ? ?brim:print( > ? ? ?brim:add_class("tr:not([id^=ker])", "pointless", > ? ? ? ?brim:content("#apps > caption", node(), > ? ? ? ? ?brim:content("#apps > tbody", app_rows(Doc), Doc)))). > > Another alternative below. But I think it has too many limitations to be feasible. > > html() -> > ? ?Doc = brim:document("priv/apps.html"), > ? ?brim:do(Doc, > ? ? ? ?[brim:content("#apps > caption", node()), > ? ? ? ? brim:content("#apps > tbody", app_rows(Doc)), > ? ? ? ? brim:add_class("tr:not([id^=ker])", "pointless"), > ? ? ? ? brim:print()]). > > As I said, all input is appreciated. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andre@REDACTED Thu Mar 15 14:00:47 2012 From: andre@REDACTED (=?ISO-8859-1?Q?Andr=E9_Graf?=) Date: Thu, 15 Mar 2012 14:00:47 +0100 Subject: [erlang-questions] Documentation error in Diameter AVP specification? In-Reply-To: References: Message-ID: Hi Jeroen I think you are right! However, the new draft for RFC 3588 http://datatracker.ietf.org/doc/draft-ietf-dime-rfc3588bis/ uses without the dash. - Andr? On 15 March 2012 11:44, Jeroen Koops wrote: > Hi all, > > In the documentation for the Diameter dict-file format, > at?http://www.erlang.org/doc/man/diameter_dict.html, it says, under the > @messages tag: > >> Defines the messages of the application. The section content consists of >> definitions of the form specified in >> section 3.2 of RFC 3588, "Command Code ABNF specification". > > The examples given show a diameter specified as follows:?< Diameter Header: > 287, REQ, PXY > > > However, RFC3588 specifies that a header should be specified as: > > ? ? header = "<" Diameter-Header:" command-id?[r-bit] [p-bit] [e-bit] > [application-id]">" > > Note the dash in Diameter-Header. Using a dash in a .dia file causes an > error when compiling the file with diameterc, so it seems that diameterc > does not completely follow RFC3588. > > Am I misunderstanding something here, or is this a bug for which I can > submit a patch? The best patch I can think of is to modify diameterc in such > a way that both 'Diameter-Header' and 'Diameter Header' are accepted, with a > note in the document saying that 'Diameter Header' is accepted but > deprecated. An easier patch would jus add a note to the documentation > pointing out the difference with the RFC. > > > -- > Jeroen Koops > > M: koops.j@REDACTED > T: +31-6-55590300 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mfidelman@REDACTED Thu Mar 15 14:19:55 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Thu, 15 Mar 2012 09:19:55 -0400 Subject: [erlang-questions] [OT] Re: GPL vs. whatever [was: Erlang UUID] In-Reply-To: References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> <4F5D736A.8070901@meetinghouse.net> Message-ID: <4F61EC7B.1050503@meetinghouse.net> Can you elaborate? Tim McNamara wrote: > If you're interested in using this approach for your business model, > then the AGPL can be a better option. > > On 12 March 2012 16:54, Miles Fidelman wrote: >> Just to throw in a different aspect of the GPL vs. BSD discussion: >> >> IMHO, GPL is a far better license than BSD for a developer that intends to >> commercialize a product. >> >> The initial developer (copyright holder) always has the option to release >> code under a dual license - GPL, BSD, or whatever for an open source >> release, something more restrictive for the commercial product (potentially >> with proprietary extensions). >> >> With GPL, you pretty much eliminate any competition - anybody else who >> extends the code is faced with copyleft considerations, they CAN'T take your >> code, combine it with their own code, and slap a proprietary license around >> the assemblage. With BSD, or Apache, (or LGPL for that matter), they can. >> >> Of course, if you dual-license your code under GPL and a proprietary >> license, things can come back to haunt you if you want to incorporate >> community-generated extensions into your upstream code base. In that case >> the GPL and copyleft apply to you. >> >> -- >> In theory, there is no difference between theory and practice. >> In practice, there is. .... Yogi Berra >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From pablo.platt@REDACTED Thu Mar 15 14:36:11 2012 From: pablo.platt@REDACTED (Pablo Platt) Date: Thu, 15 Mar 2012 06:36:11 -0700 (PDT) Subject: [erlang-questions] Erlang meets physics In-Reply-To: <961AF67D-1E9C-41B1-A5D8-DFACB6AD63C8@gmail.com> References: <1331666426.33915.YahooMailNeo@web112614.mail.gq1.yahoo.com> <1331739205.24889.YahooMailNeo@web112620.mail.gq1.yahoo.com> <961AF67D-1E9C-41B1-A5D8-DFACB6AD63C8@gmail.com> Message-ID: <1331818571.41505.YahooMailNeo@web112605.mail.gq1.yahoo.com> What I'm asking is how you actually send data over the wire. How does eprologix_cmdr:send_query/3 works? https://github.com/kofron/dripline/blob/master/src/hp8340b.erl#L100 Thanks ----- Original Message ----- From: Jared Kofron To: Pablo Platt Cc: Erlang Questions Sent: Thursday, March 15, 2012 2:32 AM Subject: Re: [erlang-questions] Erlang meets physics Hi Pablo- The translation is a simple pattern match - locator_to_ch_data(<<"cw_freq">>) -> "CW", which is then transformed into the final string which is passed over the wire. If you'd like to take a look at the code it's at http://github.com/kofron/dripline.? It's very very alpha, although deployed in limited production it works quite well. Unfortunately my first project, called hmhj, is closed source due to concerns from the SNO+ collaboration. JK On Mar 14, 2012, at 8:33 AM, Pablo Platt wrote: >> which the device module (in this case hp8340b) translates into the GPIB command "OPCW", and based on the >> device address (which is governed by the atom in the first argument), dispatches it over a bus handle that it owns >> in a state variable via bus:send_query/2.? > > Can you explain how the device module translate to the GPIB command "OPCW"? > Is it pure erlang? > Maybe you can share some code? > > Thanks > > > > > ________________________________ > From: Jared Kofron > To: Erlang Questions > Sent: Wednesday, March 14, 2012 1:43 AM > Subject: Re: [erlang-questions] Erlang meets physics > > > Hi Pablo- > My first project was on embedded hardware, and basically consisted of NIFs and Webmachine dispatching. > Pretty fun stuff.? > > My current project is a little more involved, but is also pretty interesting: > > Right now the bus over which communication takes place is abstracted away by having hardware modules > which translate API functions into their appropriate wire representation and then transmit those representations > over a handle that they have to the correct bus.? > > In essence what I've done is taken instruments and buses and given them something like behaviors.? > Hardware can perform read,write,or configure, for example.? So if I want to read the center frequency of a sweeper, > I might say > > hp8340b:read(high_frequency_sweeper,<<"cw_freq">>). > > which the device module (in this case hp8340b) translates into the GPIB command "OPCW", and based on the > device address (which is governed by the atom in the first argument), dispatches it over a bus handle that it owns > in a state variable via bus:send_query/2.? > > At the moment, we only communicate with things over GPIB via ethernet using prologix devices to do the translation > for us - basically glorified telnet - but it gets the job done.? Everything is in a very alpha stage right now for this project, > but it is working really nicely so far. > > JK > > On Mar 13, 2012, at 12:20 PM, Pablo Platt wrote: > > How do you interact with the hardware? >> Do you use GPIB C libr and wrap it with a NIF? >> >> >> >> ________________________________ >> From: Joe Armstrong >> To: Jared Kofron >> Cc: Erlang Questions >> Sent: Tuesday, March 13, 2012 12:34 PM >> Subject: Re: [erlang-questions] Erlang meets physics >> >> Great news - spread the word ! >> >> Just for the record Erlang programmers numbers 1 and 2 (ie myself and >> Robert Virding) >> are both ex physicists. >> >> When I lecture I often point out the similarity between causality and >> message reception. >> You don't know that something has happened until you get a message >> telling that it has happened. >> >> (In physics it's a ray of light, or a photon, or something - >> forgetting entanglement for the moment) >> >> In computing it's the reception of a message. >> >> As a ex physicist I know that we can't say anything about simultaneous >> events occurring >> at different places in space-time - turn this into computer science >> and the same arguments >> apply to things like making sure replicated data is consistent on >> remote sites - well you can't >> - at least if you want to change it - Brewer's CAP theorem applies - >> which for a physicist makes >> perfect sense. >> >> Also as an ex physicist > I realize that things do actually happen in >> parallel in the real world, >> so modelling them in a sequential programming language (if I wanted to do that) >> is big time crazy - just describe the parallel stuff in a concurrent >> language and the program >> writes itself. Wait a few years till we have million core computers >> and the parallel problems >> can be solved 1:1 on parallel computers - and programming simulations >> and so on will be >> really easy - but don't even think about doing it in a sequential language... >> >> Cheers >> >> /Joe >> >> >> On Mon, Mar 12, 2012 at 2:34 AM, Jared Kofron wrote: >>> Hi All, >>> I've been using Erlang at work for a few years now, and I thought I'd throw my experience out there, as >>> my application is a little different than what you usually see on the list - I am a graduate > student at the >>> Center for Nuclear Physics and Astrophysics at the University of Washington, and use Erlang extensively >>> in my work. >>> >>> In my experience, something that Erlang is really great at but doesn't receive much attention for these days >>> is managing and interacting with hardware.? In any physics experiment of even modest sizes, you wind up >>> having to keep track of the state of various pieces of equipment, often modify that state, and constantly >>> interrogate particular values.? For example, we might want to change the current in a magnetic trap, turn >>> that trap off altogether, or simply read back the voltage drop across our superconducting magnet. >>> >>> So far, I have deployed Erlang in this zone for two separate experiments (SNO+, a large particle physics >>> experiment in Canada) and Project 8 (a small nuclear physics experiment here in Seattle).? Both times > have >>> been great successes, and I have found the reception of Erlang in this market to be great.? In general, what >>> I have done is wrap a hardware management layer with some kind of outside world interface. For SNO+, we >>> used Webmachine and RESTful control, and for Project 8 we actually conduct all communication >>> by using CouchDB as a message passing interface. >>> >>> Physicists are suspicious creatures, but once you demonstrate the feature set that you get for practically >>> free with OTP, they see the advantage pretty quickly.? On top of that, the development cycle for sophisticated >>> applications can be greatly reduced - more than once it made my group float to the top in terms of meeting >>> goals. >>> >>> In short, as far as I am concerned, Erlang has found a new niche in the world of Physics, and I intend to >>> spread the word as much as I can! >>> >>> Jared > Kofron >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From anders.otp@REDACTED Thu Mar 15 14:43:21 2012 From: anders.otp@REDACTED (Anders Svensson) Date: Thu, 15 Mar 2012 14:43:21 +0100 Subject: [erlang-questions] Documentation error in Diameter AVP specification? In-Reply-To: References: Message-ID: Is this with an R14 diameterc? R15B diameterc should accept both "Diameter-Header" and "Diameter Header". RFC 3588 is inconsistent in its usage, specifying "Diameter-Header" in the ABNF but "Diameter Header" in all of its command definitions. The current draft RFC fixes this. /Anders, Erlang/OTP Ericsson RFC 3588 uses the former in its ABNF but the latter in all of its command definitions On Thu, Mar 15, 2012 at 11:44 AM, Jeroen Koops wrote: > Hi all, > > In the documentation for the Diameter dict-file format, > at?http://www.erlang.org/doc/man/diameter_dict.html, it says, under the > @messages tag: > >> Defines the messages of the application. The section content consists of >> definitions of the form specified in >> section 3.2 of RFC 3588, "Command Code ABNF specification". > > The examples given show a diameter specified as follows:?< Diameter Header: > 287, REQ, PXY > > > However, RFC3588 specifies that a header should be specified as: > > ? ? header = "<" Diameter-Header:" command-id?[r-bit] [p-bit] [e-bit] > [application-id]">" > > Note the dash in Diameter-Header. Using a dash in a .dia file causes an > error when compiling the file with diameterc, so it seems that diameterc > does not completely follow RFC3588. > > Am I misunderstanding something here, or is this a bug for which I can > submit a patch? The best patch I can think of is to modify diameterc in such > a way that both 'Diameter-Header' and 'Diameter Header' are accepted, with a > note in the document saying that 'Diameter Header' is accepted but > deprecated. An easier patch would jus add a note to the documentation > pointing out the difference with the RFC. > > > -- > Jeroen Koops > > M: koops.j@REDACTED > T: +31-6-55590300 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From koops.j@REDACTED Thu Mar 15 14:48:16 2012 From: koops.j@REDACTED (Jeroen Koops) Date: Thu, 15 Mar 2012 14:48:16 +0100 Subject: [erlang-questions] Documentation error in Diameter AVP specification? In-Reply-To: References: Message-ID: Yes, it's R14B03 -- so for now I'll just do nothing, move to R15, or wait for RFC3588bis to become current. Thanks all! On Thu, Mar 15, 2012 at 2:43 PM, Anders Svensson wrote: > Is this with an R14 diameterc? R15B diameterc should accept both > "Diameter-Header" and "Diameter Header". > > RFC 3588 is inconsistent in its usage, specifying "Diameter-Header" in > the ABNF but "Diameter Header" in all of its command definitions. The > current draft RFC fixes this. > > /Anders, Erlang/OTP Ericsson > > > > RFC 3588 uses the former in its ABNF but the latter in all of its > command definitions > > On Thu, Mar 15, 2012 at 11:44 AM, Jeroen Koops wrote: > > Hi all, > > > > In the documentation for the Diameter dict-file format, > > at http://www.erlang.org/doc/man/diameter_dict.html, it says, under the > > @messages tag: > > > >> Defines the messages of the application. The section content consists of > >> definitions of the form specified in > >> section 3.2 of RFC 3588, "Command Code ABNF specification". > > > > The examples given show a diameter specified as follows: < Diameter > Header: > > 287, REQ, PXY > > > > > However, RFC3588 specifies that a header should be specified as: > > > > header = "<" Diameter-Header:" command-id [r-bit] [p-bit] [e-bit] > > [application-id]">" > > > > Note the dash in Diameter-Header. Using a dash in a .dia file causes an > > error when compiling the file with diameterc, so it seems that diameterc > > does not completely follow RFC3588. > > > > Am I misunderstanding something here, or is this a bug for which I can > > submit a patch? The best patch I can think of is to modify diameterc in > such > > a way that both 'Diameter-Header' and 'Diameter Header' are accepted, > with a > > note in the document saying that 'Diameter Header' is accepted but > > deprecated. An easier patch would jus add a note to the documentation > > pointing out the difference with the RFC. > > > > > > -- > > Jeroen Koops > > > > M: koops.j@REDACTED > > T: +31-6-55590300 > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -- Jeroen Koops M: koops.j@REDACTED T: +31-6-55590300 -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Thu Mar 15 14:48:46 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Thu, 15 Mar 2012 14:48:46 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F606D90.5070206@fastmail.fm> <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> <4F614661.6070105@meetinghouse.net> <4F61ADDB.6010604@gmail.com> Message-ID: <4F61F33E.2050007@gmail.com> On 15/3/12 10:52 , Joe Armstrong wrote: > On Thu, Mar 15, 2012 at 9:52 AM, Torben Hoffmann > wrote: >> >> On 15/3/12 2:31 , Miles Fidelman wrote: >>> Richard O'Keefe wrote: >>>> >>>> There is often surprisingly little connection between the speed of a >>>> _language_ (implementation) and the speed of _systems_ built using it. >>>> >>>> >>> Which is almost the point - there's a rather direct correlation between >>> the speed of a system, and how well the system architecture aligns with the >>> constructs and run-time environment of the language it's implemented in. >>> >>> If you're building a system that is highly concurrent in nature, >>> implementing it in Erlang is a big win, while implementing it in Java is a >>> big loss. Yes, you can take something that's inherently concurrent, and >>> design an efficient Java design, but you end up with something that's a >>> conceptual mess. >> That is a very good point and rhymes well with the maxim of choosing the >> right tool for the job at hand... something that can be extremely difficult >> for all of us since we all have our favourite tool. Hint: learn more than >> one tool! >> >> It triggers me to take a step back and take another stab at the original >> topic of this thread. >> >> When I did my grass root (read guerilla warfare) work to be allowed to use >> Erlang for a project in Motorola the question of performance never became a >> serious issue. >> >> Why not? Because it was easy to see that if Ericsson could write telephone >> switches in Erlang it would have sufficient horsepower to do the things >> needed in another telecom system. >> >> The big worry was: "Who is using Erlang?" which comes back to the "Nobody >> has been fired for choosing Microsoft | IBM | Oracle"-maxim. Eventually I >> managed to gather enough evidence through the Erlang community (I am >> eternally grateful for the not-to-be-made-public inputs that I received from >> other Erlang warriors) to convince management that it would not be a >> dead-end to try out Erlang. >> >> And now I will allow myself to digress from the original topic for a very >> personal experience... working with Erlang has been the best experience of >> my professional life. >> Why? Because it was simply the right tool for the job at hand! It allowed us >> to get a lot done since the semantic gap between the domain and the >> programming language was so small. Not only did we get a lot done - we also >> had very few errors and when we had errors it was very easy to debug, fix >> and an re-deploy them. (During Interoperability testing with our competitors >> we had turn-around times on bugs of 15 minutes versus their 1 day... I rest >> my case!). >> >> So I am all for Joe's approach to project funding: get a prototype going and >> then evolve it into a product in small increments. You will get a prototype >> very fast - this mailing list is great for advice on how to wire things >> together and it is not that difficult to get a quick'n'dirty solution done >> in Erlang even without being an expert. > Actually this is how to make money :-) You're right! > > "All" you have to do is: Let me fill our story from Motorola into these steps!! > > a) - choose a new and "likely to be trendy" standard We started with a prototype for a simple version of our big system. > b) - implement it *all* in Erlang Check. > c) - do significant work with the standardization committee Massaged senior management and ended up being asked to choose any of three dormant projects to implement. > d) - make a product Did that. It was a 4 year journey in total - well worth the ride. > e) - give the product away or sell it This was kinda out of my hands, but it did make some money for Motorola. > f) - sell support Not likely to happen - after I left it is most likely that the "real" product will be implemented in something that works, i.e., C/C++. But you have a great action plan!! Cheers, Torben -- http://www.linkedin.com/in/torbenhoffmann From anders.otp@REDACTED Thu Mar 15 14:55:26 2012 From: anders.otp@REDACTED (Anders Svensson) Date: Thu, 15 Mar 2012 14:55:26 +0100 Subject: [erlang-questions] Documentation error in Diameter AVP specification? In-Reply-To: References: Message-ID: R15B diameterc also has a major advantage in that it emits useful error messages that point at the offending input line. Earlier versions just crashed and it could be quite difficult to figure out why. /Anders, Erlang/OTP Ericsson On Thu, Mar 15, 2012 at 2:48 PM, Jeroen Koops wrote: > Yes, it's R14B03 -- so for now I'll just do nothing, move to R15, or wait > for RFC3588bis to become current. Thanks all! > > > On Thu, Mar 15, 2012 at 2:43 PM, Anders Svensson > wrote: >> >> Is this with an R14 diameterc? R15B diameterc should accept both >> "Diameter-Header" and "Diameter Header". >> >> RFC 3588 is inconsistent in its usage, specifying "Diameter-Header" in >> the ABNF but "Diameter Header" in all of its command definitions. The >> current draft RFC fixes this. >> >> /Anders, Erlang/OTP Ericsson >> >> >> >> RFC 3588 uses the former in its ABNF but the latter in all of its >> command definitions >> >> On Thu, Mar 15, 2012 at 11:44 AM, Jeroen Koops wrote: >> > Hi all, >> > >> > In the documentation for the Diameter dict-file format, >> > at?http://www.erlang.org/doc/man/diameter_dict.html, it says, under the >> > @messages tag: >> > >> >> Defines the messages of the application. The section content consists >> >> of >> >> definitions of the form specified in >> >> section 3.2 of RFC 3588, "Command Code ABNF specification". >> > >> > The examples given show a diameter specified as follows:?< Diameter >> > Header: >> > 287, REQ, PXY > >> > >> > However, RFC3588 specifies that a header should be specified as: >> > >> > ? ? header = "<" Diameter-Header:" command-id?[r-bit] [p-bit] [e-bit] >> > [application-id]">" >> > >> > Note the dash in Diameter-Header. Using a dash in a .dia file causes an >> > error when compiling the file with diameterc, so it seems that diameterc >> > does not completely follow RFC3588. >> > >> > Am I misunderstanding something here, or is this a bug for which I can >> > submit a patch? The best patch I can think of is to modify diameterc in >> > such >> > a way that both 'Diameter-Header' and 'Diameter Header' are accepted, >> > with a >> > note in the document saying that 'Diameter Header' is accepted but >> > deprecated. An easier patch would jus add a note to the documentation >> > pointing out the difference with the RFC. >> > >> > >> > -- >> > Jeroen Koops >> > >> > M: koops.j@REDACTED >> > T: +31-6-55590300 >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > > > > > > -- > Jeroen Koops > > M: koops.j@REDACTED > T: +31-6-55590300 > From koops.j@REDACTED Thu Mar 15 14:58:44 2012 From: koops.j@REDACTED (Jeroen Koops) Date: Thu, 15 Mar 2012 14:58:44 +0100 Subject: [erlang-questions] Documentation error in Diameter AVP specification? In-Reply-To: References: Message-ID: OK, you convinced me.... upgrading to R15 *now*! On Thu, Mar 15, 2012 at 2:55 PM, Anders Svensson wrote: > R15B diameterc also has a major advantage in that it emits useful > error messages that point at the offending input line. Earlier > versions just crashed and it could be quite difficult to figure out > why. > > /Anders, Erlang/OTP Ericsson > > > On Thu, Mar 15, 2012 at 2:48 PM, Jeroen Koops wrote: > > Yes, it's R14B03 -- so for now I'll just do nothing, move to R15, or wait > > for RFC3588bis to become current. Thanks all! > > > > > > On Thu, Mar 15, 2012 at 2:43 PM, Anders Svensson > > wrote: > >> > >> Is this with an R14 diameterc? R15B diameterc should accept both > >> "Diameter-Header" and "Diameter Header". > >> > >> RFC 3588 is inconsistent in its usage, specifying "Diameter-Header" in > >> the ABNF but "Diameter Header" in all of its command definitions. The > >> current draft RFC fixes this. > >> > >> /Anders, Erlang/OTP Ericsson > >> > >> > >> > >> RFC 3588 uses the former in its ABNF but the latter in all of its > >> command definitions > >> > >> On Thu, Mar 15, 2012 at 11:44 AM, Jeroen Koops > wrote: > >> > Hi all, > >> > > >> > In the documentation for the Diameter dict-file format, > >> > at http://www.erlang.org/doc/man/diameter_dict.html, it says, under > the > >> > @messages tag: > >> > > >> >> Defines the messages of the application. The section content consists > >> >> of > >> >> definitions of the form specified in > >> >> section 3.2 of RFC 3588, "Command Code ABNF specification". > >> > > >> > The examples given show a diameter specified as follows: < Diameter > >> > Header: > >> > 287, REQ, PXY > > >> > > >> > However, RFC3588 specifies that a header should be specified as: > >> > > >> > header = "<" Diameter-Header:" command-id [r-bit] [p-bit] [e-bit] > >> > [application-id]">" > >> > > >> > Note the dash in Diameter-Header. Using a dash in a .dia file causes > an > >> > error when compiling the file with diameterc, so it seems that > diameterc > >> > does not completely follow RFC3588. > >> > > >> > Am I misunderstanding something here, or is this a bug for which I can > >> > submit a patch? The best patch I can think of is to modify diameterc > in > >> > such > >> > a way that both 'Diameter-Header' and 'Diameter Header' are accepted, > >> > with a > >> > note in the document saying that 'Diameter Header' is accepted but > >> > deprecated. An easier patch would jus add a note to the documentation > >> > pointing out the difference with the RFC. > >> > > >> > > >> > -- > >> > Jeroen Koops > >> > > >> > M: koops.j@REDACTED > >> > T: +31-6-55590300 > >> > > >> > > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > > > > > > > > -- > > Jeroen Koops > > > > M: koops.j@REDACTED > > T: +31-6-55590300 > > > -- Jeroen Koops M: koops.j@REDACTED T: +31-6-55590300 -------------- next part -------------- An HTML attachment was scrubbed... URL: From shahrdad1@REDACTED Thu Mar 15 15:34:45 2012 From: shahrdad1@REDACTED (Shahrdad Shadab) Date: Thu, 15 Mar 2012 10:34:45 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F606D90.5070206@fastmail.fm> <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> <4F614661.6070105@meetinghouse.net> <4F61ADDB.6010604@gmail.com> Message-ID: Within past 25 years of experience in this field working with various languages and technologies (Fortran, PL/I, C, C++, Java J2ee, .net, clojure, scala, groovy, Ruby, R, Erlang , Oracle SOA suit etc.) I found Erlang/OTP the best. I believe Erlang currently doesn't get the publicity that it deserves. As a concrete example what a monster like Oracle SOA suite (an ugly bloated technology placed on top of J2ee) offers can be implemented in Erlang/OTP + javascript + HTML5 + Db many times more efficient. Trust me folks , I worked and was trained in Oracle SAO suite. An this bothers me that no company even think about using Erlang before paying thousands of dollars to buy Licensees from Oracle. No one knows Erlang can be used in realms way different than telecom industry. (For instance: http://www.erlang-factory.com/upload/presentations/384/GeneSher.pdf) Along with Joe's guidelines I suggest the prototype work in following fields (I know some works are already done, but they are mostly in Europe): _ Statistical Machine learning engine (applicable for pattern recognitions, control systems, ...). _ Social insects behavioral model (as a form of collective intelligence to accomplish a task by large scale network of computers) _ Genetic Algorithms (particularly emergence and altruistic behaviors) In these applications concurrency, multicore , distribution and fault tolerance are key points that make Erlang first choice on the table. However Strong math library (something similar to that of R or Mathlab) needs to be added. Again Erlang's Distribution and message passing simplicity makes us able to distribute intensive math work cross nodes cross boxes. (Any comments?) On Thu, Mar 15, 2012 at 7:30 AM, Michael Turner < michael.eugene.turner@REDACTED> wrote: > Joe writes: > > "All" you have to do is: > > > > a) - choose a new and "likely to be trendy" standard > > b) - implement it *all* in Erlang > > c) - do significant work with the standardization committee > > d) - make a product > > e) - give the product away or sell it > > f) - sell support > > > > Examples of this are (xmpp - process one) (netconf - tail-f) (AMQP - > rabbit MQ) > > (this is a list of (protocol, company) pairs) > > > > The tricky bit is choosing a) (many choices) b) (more tricky than you > > think) and in the c) .. f) parts much can go wrong. > > (a) A long time ago, in a galaxy far, far away, a company called > Autodesk was just a gang of friends who all chose to hack on different > product ideas, while also pledging fealty to whichever idea really > took off. What took off was Autocad. I think all those Autodesk > startup people got rich. So if a bunch of good Erlang programmers got > together under a similar agreement, and each chose a different "likely > to be trendy" standard, the chances of catching a wave might be > greatly improved. Erlang's features make each programmer significantly > more productive than others attempting the same with less-suitable > languages/platforms, so the intuitively obvious "dilution of focus" > counter-argument isn't quite as strong here. > > (b) Let's say they all worked in a collegial "open door" manner where > they could pick each others' brains. This would be more likely if they > all keep each other briefed on their progress at regularly scheduled > demos and presentations, and always at least went to lunch together > several times a week even if they mostly worked at home. Then the > risks in "implement it *all* in Erlang" might also be reduced: > *somebody* in the group is likely to know how to do what you don't > know how to do. > > (c) Let's say this group appoints one of their number to be nothing > but a liaison to standards committees. (There are people who actually > thrive on this kind of work, believe it or not.) This reduces the > effort per programmer on the legwork and wrangling related to staying > abreast of standards and contributing to them. > > (d,e,f) Many a startup has come to grief on issues of execution. If > your pick for the multi-tasking liaison is also someone with > significant product management experience in protocol software > products, you've got a key member who gives your startup a fighting > chance at defying the harrowing infant-mortality statistics for young > companies. What happens when the clear winner emerges among the > varying efforts? Your standards-committee liaison is almost completely > up to speed on a whole spectrum of issues in marketing, product > management, technical publications, etc. for the *specific* idea that > has become the company's main focus. Maybe not a CEO type. But he/she > might do until the real thing comes along. And venture capitalists are > often only too happy to give you "the real thing," as a condition of > first-round financing. (But watch out for that, because sometimes they > regard certain fledgling endeavors as little more than training wheels > or reality tests for startup-CEO-wannabe types -- i.e., the real > "product" for them might be the CEO pick, with your company's failure > supposedly at least providing some instructive and/or > character-building education.) > > ... is how I'd strategize it, anyway. > > -michael turner > > On Thu, Mar 15, 2012 at 6:52 PM, Joe Armstrong wrote: > > On Thu, Mar 15, 2012 at 9:52 AM, Torben Hoffmann > > wrote: > >> > >> > >> On 15/3/12 2:31 , Miles Fidelman wrote: > >>> > >>> Richard O'Keefe wrote: > >>>> > >>>> > >>>> There is often surprisingly little connection between the speed of a > >>>> _language_ (implementation) and the speed of _systems_ built using it. > >>>> > >>>> > >>> Which is almost the point - there's a rather direct correlation between > >>> the speed of a system, and how well the system architecture aligns > with the > >>> constructs and run-time environment of the language it's implemented > in. > >>> > >>> If you're building a system that is highly concurrent in nature, > >>> implementing it in Erlang is a big win, while implementing it in Java > is a > >>> big loss. Yes, you can take something that's inherently concurrent, > and > >>> design an efficient Java design, but you end up with something that's a > >>> conceptual mess. > >> > >> That is a very good point and rhymes well with the maxim of choosing the > >> right tool for the job at hand... something that can be extremely > difficult > >> for all of us since we all have our favourite tool. Hint: learn more > than > >> one tool! > >> > >> It triggers me to take a step back and take another stab at the original > >> topic of this thread. > >> > >> When I did my grass root (read guerilla warfare) work to be allowed to > use > >> Erlang for a project in Motorola the question of performance never > became a > >> serious issue. > >> > >> Why not? Because it was easy to see that if Ericsson could write > telephone > >> switches in Erlang it would have sufficient horsepower to do the things > >> needed in another telecom system. > >> > >> The big worry was: "Who is using Erlang?" which comes back to the > "Nobody > >> has been fired for choosing Microsoft | IBM | Oracle"-maxim. Eventually > I > >> managed to gather enough evidence through the Erlang community (I am > >> eternally grateful for the not-to-be-made-public inputs that I received > from > >> other Erlang warriors) to convince management that it would not be a > >> dead-end to try out Erlang. > >> > >> And now I will allow myself to digress from the original topic for a > very > >> personal experience... working with Erlang has been the best experience > of > >> my professional life. > >> Why? Because it was simply the right tool for the job at hand! It > allowed us > >> to get a lot done since the semantic gap between the domain and the > >> programming language was so small. Not only did we get a lot done - we > also > >> had very few errors and when we had errors it was very easy to debug, > fix > >> and an re-deploy them. (During Interoperability testing with our > competitors > >> we had turn-around times on bugs of 15 minutes versus their 1 day... I > rest > >> my case!). > >> > >> So I am all for Joe's approach to project funding: get a prototype > going and > >> then evolve it into a product in small increments. You will get a > prototype > >> very fast - this mailing list is great for advice on how to wire things > >> together and it is not that difficult to get a quick'n'dirty solution > done > >> in Erlang even without being an expert. > > > > Actually this is how to make money :-) > > > > "All" you have to do is: > > > > a) - choose a new and "likely to be trendy" standard > > b) - implement it *all* in Erlang > > c) - do significant work with the standardization committee > > d) - make a product > > e) - give the product away or sell it > > f) - sell support > > > > Examples of this are (xmpp - process one) (netconf - tail-f) (AMQP - > rabbit MQ) > > (this is a list of (protocol, company) pairs) > > > > The tricky bit is choosing a) (many choices) b) (more tricky than you > > think) and in the c) .. f) parts much can go wrong. > > > > Note - the above three standards all involved communication and > > protocol implementation > > which Erlang is pretty good at. > > > > /Joe > > > >> If you want to do something cool with Erlang you have burn for it - at > least > >> if you are not deciding what you should work on direcly - and then go > and > >> execute it! > >> > >> Enjoy the ride!! > >> Torben > >> > >> > >>> > >>> The example that comes to mind is simulation. In a previous life, I > >>> worked for a company that made military simulation software (massively > >>> multiplayer games for folks who shoot real bullets). > >>> > >>> Having a networking background, where the natural inclination is to > spawn > >>> a process for every incoming task, I assumed that our simulators > operated in > >>> a similar way - simulate 1000s tanks, spawn 1000 processes and do > everything > >>> asynchronously. But no... as our coders informed me (I'm a systems > guy), > >>> you can't do that in Java (or C++, which was the bulk of our code) - > the > >>> context switching would bring everything to its knees. > >>> > >>> Instead, each tank was implemented as an object - with 4 main threads > >>> winding their way through every object, 40 times per second. The code > was > >>> fast, but not surprisingly, it was amazingly brittle - change one > object > >>> (say add a new weapon to a particular tank) and all kinds of things > would > >>> have to be rewritten. > >>> > >>> My instinct that yes, indeed, one could implement each entity as an > actor > >>> is what led me to discover Erlang - as pretty much the ONLY run-time > >>> environment optimized for massive concurrency. > >>> > >>> On the other hand, inherently serial applications probably run faster > in > >>> languages and run-time environments optimized for small numbers of > threads. > >>> > >>> Miles Fidelman > >>> > >>> > >>> > >> > >> -- > >> http://www.linkedin.com/in/torbenhoffmann > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Software Architect & Computer Scientist -------------- next part -------------- An HTML attachment was scrubbed... URL: From wmknapik@REDACTED Thu Mar 15 15:48:36 2012 From: wmknapik@REDACTED (Wojciech Knapik) Date: Thu, 15 Mar 2012 15:48:36 +0100 Subject: [erlang-questions] Boostrapping a clique of replicating mnesia nodes Message-ID: Hello everyone I have a set of nodes (discovered via net_adm:world(), but others will likely join in later) where I want to keep a fully replicated mnesia table(s). All the nodes are identical in terms of code and all of them start at roughly the same time on different machines. If the database doesn't exist, one of the nodes needs to create the table with {disc_copies, nodes()}. If it does exist and it's fully replicated, then we're done, but if it's not fully replicated, the nodes that are not replicating need to join in. I put together this code http://pastie.org/3601148 to illustrate my idea. It might be bad for whatever reason, but it's just to get my message across. Does this algorithm make sense ? Is this the right direction I'm headed ? I'd be grateful for any hints. thanks, WK -------------- next part -------------- An HTML attachment was scrubbed... URL: From jodie.burch@REDACTED Thu Mar 15 15:57:49 2012 From: jodie.burch@REDACTED (Jodie Burch) Date: Thu, 15 Mar 2012 14:57:49 +0000 Subject: [erlang-questions] Erlang User Conference - Talk Submission Message-ID: Hi Everyone, This is just a quick reminder that today is the deadline for you to submit your talk for this year?s EUC in Stockholm, 28-30th May. Amongst speakers confirmed already are the 3 inventors of Erlang: Mike Williams, Robert Virding and Joe Armstrong. Do not miss your chance to have your name listed alongside these Erlang legends. You can find all the details for the Conference here: http://www.erlang-factory.com/conference/ErlangUserConference2012 Submit your talk here: http://www.erlang-factory.com/conference/ErlangUserConference2012/submit_tal k Thanks, Jodie -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.melin@REDACTED Thu Mar 15 15:59:31 2012 From: per.melin@REDACTED (Per Melin) Date: Thu, 15 Mar 2012 15:59:31 +0100 Subject: [erlang-questions] [ANN] Brim - HTML templating library In-Reply-To: References: <6E4F3076-B078-4C47-B33E-561B43A1EEE2@gmail.com> Message-ID: On Mar 15, 2012, at 13:17 , Michael Turner wrote: > Not for lack of trying: we hapless Windows users got Microsoft Bob, > then "Clippy," then that annoying dolphin swimming around on our > desktops. Of all the possible reactions, evoking Clippy was not one I could have foreseen. > Anyway, I'm just hinting at a possible approach. I'm a little > concerned, because at this point it looks like you're starting to > implement a whole little language, with code passed in string > parameters (e.g., "tr:not([id^=ker])". This little language-in-a-string is not something I necessarily want, but it is also not something that I invented; http://www.w3.org/TR/selectors/#selectors Forcing your users to learn yet another thing is not what I strive for, but I hope CSS selectors are at least well known to anyone who works with HTML. I'd happily do away with selectors for Erlang, but I doubt I could create an API that would be nearly as expressive in a way that takes any less effort to learn. E.g. "div#foo span.bar:first-child" matches any with class "bar" that is the first child of its parent element and also a descendant of a
with id "foo". I don't know yet if this kind of power is overkill for this application. The state that I'm passing around is a zipper tree representing the HTML. I could of course give you the exported functions of my zipper module and let you navigate that way. From jack@REDACTED Thu Mar 15 17:05:44 2012 From: jack@REDACTED (Jack Moffitt) Date: Thu, 15 Mar 2012 10:05:44 -0600 Subject: [erlang-questions] [ANN] Brim - HTML templating library In-Reply-To: References: <6E4F3076-B078-4C47-B33E-561B43A1EEE2@gmail.com> Message-ID: > This little language-in-a-string is not something I necessarily want, but it is also not something that I invented; http://www.w3.org/TR/selectors/#selectors I'll point out that language-in-a-string has worked pretty well for jQuery. It has its warts, but it is a practical alternative to trying to encode selector syntax into some crazy in-language DSL. jack. From mjtruog@REDACTED Thu Mar 15 17:23:18 2012 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 15 Mar 2012 09:23:18 -0700 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F61F33E.2050007@gmail.com> References: <4F606D90.5070206@fastmail.fm> <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> <4F614661.6070105@meetinghouse.net> <4F61ADDB.6010604@gmail.com> <4F61F33E.2050007@gmail.com> Message-ID: <4F621776.30303@gmail.com> On 03/15/2012 06:48 AM, Torben Hoffmann wrote: > > > On 15/3/12 10:52 , Joe Armstrong wrote: >> On Thu, Mar 15, 2012 at 9:52 AM, Torben Hoffmann >> wrote: >>> >>> On 15/3/12 2:31 , Miles Fidelman wrote: >>>> Richard O'Keefe wrote: >>>>> >>>>> There is often surprisingly little connection between the speed of a >>>>> _language_ (implementation) and the speed of _systems_ built using it. >>>>> >>>>> >>>> Which is almost the point - there's a rather direct correlation between >>>> the speed of a system, and how well the system architecture aligns with the >>>> constructs and run-time environment of the language it's implemented in. >>>> >>>> If you're building a system that is highly concurrent in nature, >>>> implementing it in Erlang is a big win, while implementing it in Java is a >>>> big loss. Yes, you can take something that's inherently concurrent, and >>>> design an efficient Java design, but you end up with something that's a >>>> conceptual mess. >>> That is a very good point and rhymes well with the maxim of choosing the >>> right tool for the job at hand... something that can be extremely difficult >>> for all of us since we all have our favourite tool. Hint: learn more than >>> one tool! >>> >>> It triggers me to take a step back and take another stab at the original >>> topic of this thread. >>> >>> When I did my grass root (read guerilla warfare) work to be allowed to use >>> Erlang for a project in Motorola the question of performance never became a >>> serious issue. >>> >>> Why not? Because it was easy to see that if Ericsson could write telephone >>> switches in Erlang it would have sufficient horsepower to do the things >>> needed in another telecom system. >>> >>> The big worry was: "Who is using Erlang?" which comes back to the "Nobody >>> has been fired for choosing Microsoft | IBM | Oracle"-maxim. Eventually I >>> managed to gather enough evidence through the Erlang community (I am >>> eternally grateful for the not-to-be-made-public inputs that I received from >>> other Erlang warriors) to convince management that it would not be a >>> dead-end to try out Erlang. >>> >>> And now I will allow myself to digress from the original topic for a very >>> personal experience... working with Erlang has been the best experience of >>> my professional life. >>> Why? Because it was simply the right tool for the job at hand! It allowed us >>> to get a lot done since the semantic gap between the domain and the >>> programming language was so small. Not only did we get a lot done - we also >>> had very few errors and when we had errors it was very easy to debug, fix >>> and an re-deploy them. (During Interoperability testing with our competitors >>> we had turn-around times on bugs of 15 minutes versus their 1 day... I rest >>> my case!). >>> >>> So I am all for Joe's approach to project funding: get a prototype going and >>> then evolve it into a product in small increments. You will get a prototype >>> very fast - this mailing list is great for advice on how to wire things >>> together and it is not that difficult to get a quick'n'dirty solution done >>> in Erlang even without being an expert. >> Actually this is how to make money :-) > You're right! >> >> "All" you have to do is: > Let me fill our story from Motorola into these steps!! >> >> a) - choose a new and "likely to be trendy" standard > We started with a prototype for a simple version of our big system. >> b) - implement it *all* in Erlang > Check. >> c) - do significant work with the standardization committee > Massaged senior management and ended up being asked to choose any of three dormant projects to implement. >> d) - make a product > Did that. It was a 4 year journey in total - well worth the ride. >> e) - give the product away or sell it > This was kinda out of my hands, but it did make some money for Motorola. > >> f) - sell support > Not likely to happen - after I left it is most likely that the "real" product will be implemented in something that works, i.e., C/C++. > > But you have a great action plan!! > > Cheers, > Torben > > I agree that there has been lots of negativity in the past connected with statements at previous startups like: 1) "Erlang is just a prototype language" 2) "Erlang is only a testing language" 3) "Erlang is a ghetto" I don't believe these statements, but I believe that their existence and continuation is unfortunate. I think the only thing that may perpetuate these types of statements, is a lack of clear statements about Erlang's intent by current practitioners (as it is meant to exist in current products). I understand that we have newer products, like RabbitMQ and Riak which can help provide examples of Erlang success within production (much of the previous success coming from ejabberd). I am just disappointed by how these statements are very persistent, and may often prevent the use of Erlang. - Michael From mfidelman@REDACTED Thu Mar 15 17:30:30 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Thu, 15 Mar 2012 12:30:30 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F621776.30303@gmail.com> References: <4F606D90.5070206@fastmail.fm> <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> <4F614661.6070105@meetinghouse.net> <4F61ADDB.6010604@gmail.com> <4F61F33E.2050007@gmail.com> <4F621776.30303@gmail.com> Message-ID: <4F621926.7040809@meetinghouse.net> Michael Truog wrote: > I agree that there has been lots of negativity in the past connected with statements at previous startups like: > 1) "Erlang is just a prototype language" > 2) "Erlang is only a testing language" > 3) "Erlang is a ghetto" > > I don't believe these statements, but I believe that their existence and continuation is unfortunate. I think the only thing that may perpetuate these types of statements, is a lack of clear statements about Erlang's intent by current practitioners (as it is meant to exist in current products). I understand that we have newer products, like RabbitMQ and Riak which can help provide examples of Erlang success within production (much of the previous success coming from ejabberd). I am just disappointed by how these statements are very persistent, and may often prevent the use of Erlang. Personally, if Erlang shortens time-to-market and software maintenance costs, and improves scalability, then I'm all for the competition continuing to believe the myths. :-) Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From mfidelman@REDACTED Thu Mar 15 17:32:22 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Thu, 15 Mar 2012 12:32:22 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F621926.7040809@meetinghouse.net> References: <4F606D90.5070206@fastmail.fm> <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> <4F614661.6070105@meetinghouse.net> <4F61ADDB.6010604@gmail.com> <4F61F33E.2050007@gmail.com> <4F621776.30303@gmail.com> <4F621926.7040809@meetinghouse.net> Message-ID: <4F621996.7010102@meetinghouse.net> Miles Fidelman wrote: > Michael Truog wrote: >> I agree that there has been lots of negativity in the past connected >> with statements at previous startups like: >> 1) "Erlang is just a prototype language" >> 2) "Erlang is only a testing language" >> 3) "Erlang is a ghetto" >> >> I don't believe these statements, but I believe that their existence >> and continuation is unfortunate. I think the only thing that may >> perpetuate these types of statements, is a lack of clear statements >> about Erlang's intent by current practitioners (as it is meant to >> exist in current products). I understand that we have newer >> products, like RabbitMQ and Riak which can help provide examples of >> Erlang success within production (much of the previous success coming >> from ejabberd). I am just disappointed by how these statements are >> very persistent, and may often prevent the use of Erlang. > > Personally, if Erlang shortens time-to-market and software maintenance > costs, and improves scalability, then I'm all for the competition > continuing to believe the myths. :-) I should add... particularly if larger, better-funded competition continues to waste time and money and build/maintain overly-complicated, brittle systems! -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From mononcqc@REDACTED Thu Mar 15 19:01:29 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 15 Mar 2012 14:01:29 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F621926.7040809@meetinghouse.net> References: <4F606D90.5070206@fastmail.fm> <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> <4F614661.6070105@meetinghouse.net> <4F61ADDB.6010604@gmail.com> <4F61F33E.2050007@gmail.com> <4F621776.30303@gmail.com> <4F621926.7040809@meetinghouse.net> Message-ID: On Thu, Mar 15, 2012 at 12:30 PM, Miles Fidelman wrote: > > Personally, if Erlang shortens time-to-market and software maintenance > costs, and improves scalability, then I'm all for the competition > continuing to believe the myths. :-) The maintainability is a great one for me, but it's also a thing that I don't see mentioned too often. I'll attribute a lot of it to OTP and its well defined design practices, but it is usually not too hard for me to dive in any Erlang project and understand how it works in a matter of a few hours, maybe days for more complex or less standard stuff. This is something I cannot say is true of PHP, Python, Javascript, Scheme or many other languages where I've had or wanted to toy with the code a bit. You have to usually go for higher level frameworks to understand that, but Erlang's framework is sitting at a level low enough that it's rather universal to all code bases. Know your OTP and you can likely understand a crapload of Erlang projects without trying too hard. This is a very interesting property to emphasize, in my opinion. -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.melin@REDACTED Thu Mar 15 19:54:05 2012 From: per.melin@REDACTED (Per Melin) Date: Thu, 15 Mar 2012 19:54:05 +0100 Subject: [erlang-questions] [ANN] Brim - HTML templating library In-Reply-To: References: <6E4F3076-B078-4C47-B33E-561B43A1EEE2@gmail.com> Message-ID: <8D0C933B-4231-4A82-A3F5-B253332C155A@gmail.com> I wrote: > I'd happily do away with selectors for Erlang, but I doubt I could create an API that would be nearly as expressive in a way that takes any less effort to learn. I wish I hadn't been quite so quick to make that statement. > E.g. "div#foo span.bar:first-child" matches any with class "bar" that is the first child of its parent element and also a descendant of a
with id "foo". This (expressing the same rule as above) would arguably take less effort to learn at least: brim:descendant( brim:id(foo, brim:element(div)), brim:first_child(brim:class(bar, brim:element(span))) ) But it is too verbose. So we are tempted to start down this road (Jack Moffitt saw it coming): {descendant, {id, foo, {element, div}}}, {first_child, {class, bar, {element, span}}} If either is easier to read comes down to whether you know the (small) grammar of selectors, I guess. We'd do away with the string parsing, and both of the above are composable. Double big win. But still too long I think, even if we shorten the keywords. On the other hand, because they are composable we would not have to write the full thing every time. From pieter.rijken@REDACTED Thu Mar 15 19:56:44 2012 From: pieter.rijken@REDACTED (pietje) Date: Thu, 15 Mar 2012 11:56:44 -0700 (PDT) Subject: [erlang-questions] Erlang meets physics In-Reply-To: References: Message-ID: <16736994.2.1331837804364.JavaMail.geo-discussion-forums@ynkv16> > > Hi All, > Just my 2 cents. I am an ex physicist too. Used to do research on QCD (Drell-Yan process) calculating Feynman diagrams. For a couple of years I've been busy writing an erlang program to handle Feynman diagrams algebraically. Lots of fun. regards, Pieter Rijken > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > On Monday, 12 March 2012 02:34:04 UTC+1, Jared Kofron wrote: > > Hi All, > I've been using Erlang at work for a few years now, and I thought I'd > throw my experience out there, as > my application is a little different than what you usually see on the list > - I am a graduate student at the > Center for Nuclear Physics and Astrophysics at the University of > Washington, and use Erlang extensively > in my work. > > In my experience, something that Erlang is really great at but doesn't > receive much attention for these days > is managing and interacting with hardware. In any physics experiment of > even modest sizes, you wind up > having to keep track of the state of various pieces of equipment, often > modify that state, and constantly > interrogate particular values. For example, we might want to change the > current in a magnetic trap, turn > that trap off altogether, or simply read back the voltage drop across our > superconducting magnet. > > So far, I have deployed Erlang in this zone for two separate experiments > (SNO+, a large particle physics > experiment in Canada) and Project 8 (a small nuclear physics experiment > here in Seattle). Both times have > been great successes, and I have found the reception of Erlang in this > market to be great. In general, what > I have done is wrap a hardware management layer with some kind of outside > world interface. For SNO+, we > used Webmachine and RESTful control, and for Project 8 we actually conduct > all communication > by using CouchDB as a message passing interface. > > Physicists are suspicious creatures, but once you demonstrate the feature > set that you get for practically > free with OTP, they see the advantage pretty quickly. On top of that, the > development cycle for sophisticated > applications can be greatly reduced - more than once it made my group > float to the top in terms of meeting > goals. > > In short, as far as I am concerned, Erlang has found a new niche in the > world of Physics, and I intend to > spread the word as much as I can! > > Jared Kofron > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Mar 15 20:58:48 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 15 Mar 2012 20:58:48 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F612FF3.7030804@fastmail.fm> References: <4F606D90.5070206@fastmail.fm> <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> <4F612FF3.7030804@fastmail.fm> Message-ID: <4F6249F8.2040508@erlang-solutions.com> On 3/15/12 12:55 AM, Jan Burse wrote: > > But the Java performance also surprised me. Especially since > the code contains many new expressions. But rumors have > it that new is just a pointer decrement in Java. > It may be more than a rumor. Most GC'ed languages turn out to do something like that, and the allocation only costs for deallocation if you keep a reference to the allocated object. Java can routinely outperform Erlang if the problem at hand is CPU bound. It is exactly the kind of jobs that Erlang does not excel at, one way or the other. But take a larger system, and the performance of the language matter less and less, the larger the system. Then all hinges on the programmers and how quickly they can express their ideas. And in the game of expressing problems quickly and succinctly, you definitely need languages far from Java. Erlang, Ocaml and Haskell are my typical tools for quickly expressing problems. Real world problems are constrained by programmer time. So if you spend all your time debugging pesky null problems, state errors and writing boilerplate, then you are not going to get far. -- Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen, DK From paperless@REDACTED Thu Mar 15 21:02:35 2012 From: paperless@REDACTED (Tim McNamara) Date: Fri, 16 Mar 2012 09:02:35 +1300 Subject: [erlang-questions] [OT] Re: GPL vs. whatever [was: Erlang UUID] In-Reply-To: <4F61EC7B.1050503@meetinghouse.net> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> <4F5D736A.8070901@meetinghouse.net> <4F61EC7B.1050503@meetinghouse.net> Message-ID: >>> IMHO, GPL is a far better license than BSD for a developer that intends >>> to commercialize a product. >>> >>> The initial developer (copyright holder) always has the option to release >>> code under a dual license - GPL, BSD, or whatever for an open source >>> release, something more restrictive for the commercial product >>> (potentially with proprietary extensions). > Tim McNamara wrote: >> >> If you're interested in using this approach for your business model, >> then the AGPL can be a better option. On 16 March 2012 02:19, Miles Fidelman wrote: > Can you elaborate? The AGPL adds copyleft provisions for services provided via a network. That is, if you create a service and deliver it via WWW, then you need to make the source of your service available. The GPL has a smaller reach. As an example, Nimbus.io went live today. It is an Amazon S3 competitor, providing USD0.06/GB/month storage. One of its taglines i "100% Open source hardware, client libraries and server software". However, much (perhaps all, I haven't had a detailed look) of their software is available via the AGPL. Any competitors who wish to freeride on the code need to release everything. From mattevans123@REDACTED Thu Mar 15 21:16:43 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Thu, 15 Mar 2012 16:16:43 -0400 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: , <4F606D90.5070206@fastmail.fm>, <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz>, <4F614661.6070105@meetinghouse.net> <4F61ADDB.6010604@gmail.com>, , Message-ID: One problem Erlang has is a lack of marketing. Oracle makes money out of Java, it's in their interest to develop, grow, and then sell the language as much as possible. Although I think Ericsson is doing an awesome job with open-source Erlang, they don't make any money from it as a language. Obviously, Ericsson are only going to push new features and improvements that are of direct benefit to Ericsson for their own products (unless I am missing something). I've often wondered if it would make sense to have Erlang owned as a consortium of sorts. Get Ericsson, Klarna, Erlang Solutions, Basho, RabbitMQ etc together pushing progress in the language. I think it's sort of like that already in an unofficial way, but having direction from a group effort could really focus on evolving the language in new ways. My 2c ... Matt > Date: Thu, 15 Mar 2012 20:30:09 +0900 > From: michael.eugene.turner@REDACTED > To: erlang@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Erlang is the best choice for building commercial application servers > > Joe writes: > > "All" you have to do is: > > > > a) - choose a new and "likely to be trendy" standard > > b) - implement it *all* in Erlang > > c) - do significant work with the standardization committee > > d) - make a product > > e) - give the product away or sell it > > f) - sell support > > > > Examples of this are (xmpp - process one) (netconf - tail-f) (AMQP - rabbit MQ) > > (this is a list of (protocol, company) pairs) > > > > The tricky bit is choosing a) (many choices) b) (more tricky than you > > think) and in the c) .. f) parts much can go wrong. > > (a) A long time ago, in a galaxy far, far away, a company called > Autodesk was just a gang of friends who all chose to hack on different > product ideas, while also pledging fealty to whichever idea really > took off. What took off was Autocad. I think all those Autodesk > startup people got rich. So if a bunch of good Erlang programmers got > together under a similar agreement, and each chose a different "likely > to be trendy" standard, the chances of catching a wave might be > greatly improved. Erlang's features make each programmer significantly > more productive than others attempting the same with less-suitable > languages/platforms, so the intuitively obvious "dilution of focus" > counter-argument isn't quite as strong here. > > (b) Let's say they all worked in a collegial "open door" manner where > they could pick each others' brains. This would be more likely if they > all keep each other briefed on their progress at regularly scheduled > demos and presentations, and always at least went to lunch together > several times a week even if they mostly worked at home. Then the > risks in "implement it *all* in Erlang" might also be reduced: > *somebody* in the group is likely to know how to do what you don't > know how to do. > > (c) Let's say this group appoints one of their number to be nothing > but a liaison to standards committees. (There are people who actually > thrive on this kind of work, believe it or not.) This reduces the > effort per programmer on the legwork and wrangling related to staying > abreast of standards and contributing to them. > > (d,e,f) Many a startup has come to grief on issues of execution. If > your pick for the multi-tasking liaison is also someone with > significant product management experience in protocol software > products, you've got a key member who gives your startup a fighting > chance at defying the harrowing infant-mortality statistics for young > companies. What happens when the clear winner emerges among the > varying efforts? Your standards-committee liaison is almost completely > up to speed on a whole spectrum of issues in marketing, product > management, technical publications, etc. for the *specific* idea that > has become the company's main focus. Maybe not a CEO type. But he/she > might do until the real thing comes along. And venture capitalists are > often only too happy to give you "the real thing," as a condition of > first-round financing. (But watch out for that, because sometimes they > regard certain fledgling endeavors as little more than training wheels > or reality tests for startup-CEO-wannabe types -- i.e., the real > "product" for them might be the CEO pick, with your company's failure > supposedly at least providing some instructive and/or > character-building education.) > > ... is how I'd strategize it, anyway. > > -michael turner > > On Thu, Mar 15, 2012 at 6:52 PM, Joe Armstrong wrote: > > On Thu, Mar 15, 2012 at 9:52 AM, Torben Hoffmann > > wrote: > >> > >> > >> On 15/3/12 2:31 , Miles Fidelman wrote: > >>> > >>> Richard O'Keefe wrote: > >>>> > >>>> > >>>> There is often surprisingly little connection between the speed of a > >>>> _language_ (implementation) and the speed of _systems_ built using it. > >>>> > >>>> > >>> Which is almost the point - there's a rather direct correlation between > >>> the speed of a system, and how well the system architecture aligns with the > >>> constructs and run-time environment of the language it's implemented in. > >>> > >>> If you're building a system that is highly concurrent in nature, > >>> implementing it in Erlang is a big win, while implementing it in Java is a > >>> big loss. Yes, you can take something that's inherently concurrent, and > >>> design an efficient Java design, but you end up with something that's a > >>> conceptual mess. > >> > >> That is a very good point and rhymes well with the maxim of choosing the > >> right tool for the job at hand... something that can be extremely difficult > >> for all of us since we all have our favourite tool. Hint: learn more than > >> one tool! > >> > >> It triggers me to take a step back and take another stab at the original > >> topic of this thread. > >> > >> When I did my grass root (read guerilla warfare) work to be allowed to use > >> Erlang for a project in Motorola the question of performance never became a > >> serious issue. > >> > >> Why not? Because it was easy to see that if Ericsson could write telephone > >> switches in Erlang it would have sufficient horsepower to do the things > >> needed in another telecom system. > >> > >> The big worry was: "Who is using Erlang?" which comes back to the "Nobody > >> has been fired for choosing Microsoft | IBM | Oracle"-maxim. Eventually I > >> managed to gather enough evidence through the Erlang community (I am > >> eternally grateful for the not-to-be-made-public inputs that I received from > >> other Erlang warriors) to convince management that it would not be a > >> dead-end to try out Erlang. > >> > >> And now I will allow myself to digress from the original topic for a very > >> personal experience... working with Erlang has been the best experience of > >> my professional life. > >> Why? Because it was simply the right tool for the job at hand! It allowed us > >> to get a lot done since the semantic gap between the domain and the > >> programming language was so small. Not only did we get a lot done - we also > >> had very few errors and when we had errors it was very easy to debug, fix > >> and an re-deploy them. (During Interoperability testing with our competitors > >> we had turn-around times on bugs of 15 minutes versus their 1 day... I rest > >> my case!). > >> > >> So I am all for Joe's approach to project funding: get a prototype going and > >> then evolve it into a product in small increments. You will get a prototype > >> very fast - this mailing list is great for advice on how to wire things > >> together and it is not that difficult to get a quick'n'dirty solution done > >> in Erlang even without being an expert. > > > > Actually this is how to make money :-) > > > > "All" you have to do is: > > > > a) - choose a new and "likely to be trendy" standard > > b) - implement it *all* in Erlang > > c) - do significant work with the standardization committee > > d) - make a product > > e) - give the product away or sell it > > f) - sell support > > > > Examples of this are (xmpp - process one) (netconf - tail-f) (AMQP - rabbit MQ) > > (this is a list of (protocol, company) pairs) > > > > The tricky bit is choosing a) (many choices) b) (more tricky than you > > think) and in the c) .. f) parts much can go wrong. > > > > Note - the above three standards all involved communication and > > protocol implementation > > which Erlang is pretty good at. > > > > /Joe > > > >> If you want to do something cool with Erlang you have burn for it - at least > >> if you are not deciding what you should work on direcly - and then go and > >> execute it! > >> > >> Enjoy the ride!! > >> Torben > >> > >> > >>> > >>> The example that comes to mind is simulation. In a previous life, I > >>> worked for a company that made military simulation software (massively > >>> multiplayer games for folks who shoot real bullets). > >>> > >>> Having a networking background, where the natural inclination is to spawn > >>> a process for every incoming task, I assumed that our simulators operated in > >>> a similar way - simulate 1000s tanks, spawn 1000 processes and do everything > >>> asynchronously. But no... as our coders informed me (I'm a systems guy), > >>> you can't do that in Java (or C++, which was the bulk of our code) - the > >>> context switching would bring everything to its knees. > >>> > >>> Instead, each tank was implemented as an object - with 4 main threads > >>> winding their way through every object, 40 times per second. The code was > >>> fast, but not surprisingly, it was amazingly brittle - change one object > >>> (say add a new weapon to a particular tank) and all kinds of things would > >>> have to be rewritten. > >>> > >>> My instinct that yes, indeed, one could implement each entity as an actor > >>> is what led me to discover Erlang - as pretty much the ONLY run-time > >>> environment optimized for massive concurrency. > >>> > >>> On the other hand, inherently serial applications probably run faster in > >>> languages and run-time environments optimized for small numbers of threads. > >>> > >>> Miles Fidelman > >>> > >>> > >>> > >> > >> -- > >> http://www.linkedin.com/in/torbenhoffmann > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From edward.yu.wang@REDACTED Thu Mar 15 21:32:07 2012 From: edward.yu.wang@REDACTED (Edward Wang) Date: Fri, 16 Mar 2012 04:32:07 +0800 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F6249F8.2040508@erlang-solutions.com> References: <4F606D90.5070206@fastmail.fm> <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> <4F612FF3.7030804@fastmail.fm> <4F6249F8.2040508@erlang-solutions.com> Message-ID: On Mar 16, 2012 3:59 AM, "Jesper Louis Andersen" < jesper.louis.andersen@REDACTED> wrote: > > On 3/15/12 12:55 AM, Jan Burse wrote: >> >> >> But the Java performance also surprised me. Especially since >> the code contains many new expressions. But rumors have >> it that new is just a pointer decrement in Java. >> > It may be more than a rumor. Most GC'ed languages turn out to do something like that, and the allocation only costs for deallocation if you keep a reference to the allocated object. > It becomes even better since copy collectors only need to copy and move live objects, not dead ones. So there's no deallocation cost for them. More details here in case you are interested: http://www.ibm.com/developerworks/java/library/j-jtp09275/index.html Regards, Edward -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Thu Mar 15 21:42:14 2012 From: bob@REDACTED (Bob Ippolito) Date: Thu, 15 Mar 2012 13:42:14 -0700 Subject: [erlang-questions] [ANN] Brim - HTML templating library In-Reply-To: <8D0C933B-4231-4A82-A3F5-B253332C155A@gmail.com> References: <6E4F3076-B078-4C47-B33E-561B43A1EEE2@gmail.com> <8D0C933B-4231-4A82-A3F5-B253332C155A@gmail.com> Message-ID: On Thu, Mar 15, 2012 at 11:54 AM, Per Melin wrote: > I wrote: > > > I'd happily do away with selectors for Erlang, but I doubt I could > create an API that would be nearly as expressive in a way that takes any > less effort to learn. > > I wish I hadn't been quite so quick to make that statement. > > > > E.g. "div#foo span.bar:first-child" matches any with class "bar" > that is the first child of its parent element and also a descendant of a >
with id "foo". > > This (expressing the same rule as above) would arguably take less effort > to learn at least: > > brim:descendant( > brim:id(foo, brim:element(div)), > brim:first_child(brim:class(bar, brim:element(span))) > ) > > But it is too verbose. > > So we are tempted to start down this road (Jack Moffitt saw it coming): > > {descendant, > {id, foo, {element, div}}}, > {first_child, {class, bar, {element, span}}} > > If either is easier to read comes down to whether you know the (small) > grammar of selectors, I guess. > > We'd do away with the string parsing, and both of the above are > composable. Double big win. But still too long I think, even if we shorten > the keywords. On the other hand, because they are composable we would not > have to write the full thing every time. You could also consider going down the road of parse transforms to keep the string syntax but get rid of the runtime string parsing. Personally I like the CSS selector syntax. It's terse but not hard to understand, and you already know it if you've done pretty much any client-side web development. Even with an alternative syntax you're still going to need to learn a subset of the same keywords (id, first_child, etc.) which is I think much more work than picking up the grammar. -bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Mar 15 21:57:37 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 16 Mar 2012 09:57:37 +1300 Subject: [erlang-questions] [ANN] Brim - HTML templating library In-Reply-To: References: <6E4F3076-B078-4C47-B33E-561B43A1EEE2@gmail.com> Message-ID: <746F15C0-458B-4E1D-A7D2-71F6A96037C3@cs.otago.ac.nz> Just for grins, let's look at another way of doing it, using another programming language. On 16/03/2012, at 1:17 AM, Michael Turner wrote: > The titleThe body Change it to <span class="C:replace">TITLE</span>

BODY

% qh -hew sample.xml | page-gen >sample.c % cat sample.c #include "dvm2.h" extern void replace(xml); void generate(xml e) { x_begin("html"); x_begin("head"); x_begin("title"); x_begin("span"); x_pcdata("TITLE"); x_end("span"); replace(x_pop()); x_end("title"); x_end("head"); x_begin("body"); x_begin("p"); x_begin("span"); x_pcdata("BODY"); x_end("span"); replace(x_pop()); x_end("p"); x_end("body"); x_end("html"); } #ifdef TEST int main(void) { generate(pcdata_str("")); write_xml(stdout, "", x_pop(), XML_SHOW_HTML|XML_SHOW_LATIN1); return 0; } #endif/*TEST*/ Now plug in a definition: void replace(xml e) { char const *k = text_of(pcdata_text(e)); if (0 == strcmp(k, "TITLE")) { x_pcdata("The title"); } else if (0 == strcmp(k, "BODY")) { x_pcdata("The body"); } else { x_begin1("font", "color", "red"); x_pcdata("No value known for: "); x_push(e); x_end("font"); } } and you're away laughing. The idea here is that the input is turned into C code that will reproduce it, except that elements with the attribute class = "C:<>" will be handed to the function of the given name for processing. The Erlang version would translate <span class="E:mod:replace">TITLE</span>

BODY

to f() -> {html,[],[ {head,[],[ {title,[],[ ]++mod:replace({span,[],[<<"TITLE">>]})++[ ]}]}, {body,[],[ {p,[],[ ]++mod:replace({span,[],[<<"BODY">>]})++[ ]}]}]). This kind of stuff is _so_ easy to do... (The C example is real; the Erlang one is not.) From ok@REDACTED Thu Mar 15 22:06:46 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 16 Mar 2012 10:06:46 +1300 Subject: [erlang-questions] [ANN] Brim - HTML templating library In-Reply-To: References: <6E4F3076-B078-4C47-B33E-561B43A1EEE2@gmail.com> Message-ID: <3C4BF661-FAA1-4CF4-A70A-8C917A2E45B8@cs.otago.ac.nz> On 16/03/2012, at 1:17 AM, Michael Turner wrote: > > Dan Halbert pioneered PBE at Xerox PARC. It was later taken up as a > research project at at Apple. Then it was championed by none other > than Melinda Gates, at Microsoft. > > It never took off. No? There are programmers who swear by "Intellisense" and similar automatic completion whizz-bang in their IDE. One of the Smalltalk systems I use, Pharo, does this. I swear *at* it, but that's another story. The web browser I mainly use offers an autocomplete list when I start to type a URL. Unlike Pharo, this doesn't visually obscure what I'm trying to type, so I just use it, and don't swear at it. Start typing a query in the Google box, and again an autocomplete list pops up (but NOT in the way!) and very handy it is. That's small-scale programming by example. I don't know what key it is that I'm pressing by mistake, but it seems as if every time I have to use vim and want to quit I end up in macro recording and have to spend a long time finger-dancing on the keyboard just to quit. Macro recording is precisely programming by example. The thing that Windows victims got unhappy about was *not* the system watching what they were trying to do and getting *ready* to be helpful about it but the system seizing control. If Clippy had stayed entirely off the screen unless and until people pressed a special Ctrl-Alt-Help chord, it would still be here. I had to use Explorer the other day on someone's netbook, and I started typing a URL and it seized control from me, saying in effect, "no, I know better than you what you want, you really want to go to your home page, and to prove how much better I know than you do, I'm going to delete your typing." PBE is not the problem, interfering is the problem. From joelr1@REDACTED Thu Mar 15 22:29:02 2012 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 15 Mar 2012 21:29:02 +0000 Subject: [erlang-questions] OpenPoker and Erlang powers EA's World Series of Poker Message-ID: EA World Series of Poker (WSOP) [1] is powered by Erlang and my OpenPoker [2]. Just wanted to give you all a heads-up ;-). [1] http://toucharcade.com/2012/03/08/gdc-2012-ea-and-chillingo-showcasing-flight-control-rocket-burnout-crash-air-mail-and-more/ [2] https://github.com/wagerlabs/openpoker -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From ulf@REDACTED Thu Mar 15 22:37:35 2012 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 15 Mar 2012 22:37:35 +0100 Subject: [erlang-questions] OpenPoker and Erlang powers EA's World Series of Poker In-Reply-To: References: Message-ID: <20CF9248-F514-427B-8906-F27703AB0296@feuerlabs.com> Congratulations, Joel! Well earned. BR, Ulf W On 15 Mar 2012, at 22:29, Joel Reymont wrote: > EA World Series of Poker (WSOP) [1] is powered by Erlang and my OpenPoker [2]. > > Just wanted to give you all a heads-up ;-). > > [1] http://toucharcade.com/2012/03/08/gdc-2012-ea-and-chillingo-showcasing-flight-control-rocket-burnout-crash-air-mail-and-more/ > [2] https://github.com/wagerlabs/openpoker > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mjtruog@REDACTED Thu Mar 15 22:43:07 2012 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 15 Mar 2012 14:43:07 -0700 Subject: [erlang-questions] OpenPoker and Erlang powers EA's World Series of Poker In-Reply-To: References: Message-ID: <4F62626B.8020403@gmail.com> On 03/15/2012 02:29 PM, Joel Reymont wrote: > EA World Series of Poker (WSOP) [1] is powered by Erlang and my OpenPoker [2]. > > Just wanted to give you all a heads-up ;-). > > [1] http://toucharcade.com/2012/03/08/gdc-2012-ea-and-chillingo-showcasing-flight-control-rocket-burnout-crash-air-mail-and-more/ > [2] https://github.com/wagerlabs/openpoker > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Very cool! From joelr1@REDACTED Thu Mar 15 22:46:04 2012 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 15 Mar 2012 21:46:04 +0000 Subject: [erlang-questions] OpenPoker and Erlang powers EA's World Series of Poker In-Reply-To: <20CF9248-F514-427B-8906-F27703AB0296@feuerlabs.com> References: <20CF9248-F514-427B-8906-F27703AB0296@feuerlabs.com> Message-ID: <851DC03726454A15A7076387008067CD@gmail.com> Ulf has the distinction of directly helping to make this happen. I wrote the first version of OpenPoker in the mansard of his house in V?rby g?rd ;-). -- http://www.linkedin.com/in/joelreymont On Thursday, March 15, 2012 at 9:37 PM, Ulf Wiger wrote: > > Congratulations, Joel! > > Well earned. > > BR, > Ulf W > > On 15 Mar 2012, at 22:29, Joel Reymont wrote: > > > EA World Series of Poker (WSOP) [1] is powered by Erlang and my OpenPoker [2]. > > > > Just wanted to give you all a heads-up ;-). > > > > [1] http://toucharcade.com/2012/03/08/gdc-2012-ea-and-chillingo-showcasing-flight-control-rocket-burnout-crash-air-mail-and-more/ > > [2] https://github.com/wagerlabs/openpoker > > > > -------------------------------------------------------------------------- > > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > > ---------------------+------------+--------------------------------------- > > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > > ---------------------+------------+--------------------------------------- > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Thu Mar 15 22:48:35 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Thu, 15 Mar 2012 22:48:35 +0100 Subject: [erlang-questions] OpenPoker and Erlang powers EA's World Series of Poker In-Reply-To: References: Message-ID: Kudos to you, Joel!! A great success story for making money on a open source project. Cheers, Torben P.S. Did you use my list comprehensions for Omaha or did you find something better? On Thu, Mar 15, 2012 at 22:29, Joel Reymont wrote: > EA World Series of Poker (WSOP) [1] is powered by Erlang and my OpenPoker > [2]. > > Just wanted to give you all a heads-up ;-). > > [1] > http://toucharcade.com/2012/03/08/gdc-2012-ea-and-chillingo-showcasing-flight-control-rocket-burnout-crash-air-mail-and-more/ > [2] https://github.com/wagerlabs/openpoker > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Fri Mar 16 01:20:45 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 15 Mar 2012 17:20:45 -0700 (PDT) Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F6249F8.2040508@erlang-solutions.com> References: <4F606D90.5070206@fastmail.fm> <30A62AF7-07C2-4600-86E1-78F780BE8AE0@cs.otago.ac.nz> <4F612FF3.7030804@fastmail.fm> <4F6249F8.2040508@erlang-solutions.com> Message-ID: <22694422.225.1331857245453.JavaMail.geo-discussion-forums@vbck9> +10 On Thursday, March 15, 2012 2:58:48 PM UTC-5, Jesper Louis Andersen wrote: > > Real world problems are constrained by programmer time. So if you spend > all your time debugging pesky null problems, state errors and writing > boilerplate, then you are not going to get far. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Fri Mar 16 01:46:40 2012 From: gumm@REDACTED (Jesse Gumm) Date: Thu, 15 Mar 2012 19:46:40 -0500 Subject: [erlang-questions] OpenPoker and Erlang powers EA's World Series of Poker In-Reply-To: References: Message-ID: Much congrats Joel, Very cool! -Jesse On Thu, Mar 15, 2012 at 4:48 PM, Torben Hoffmann wrote: > Kudos to you, Joel!! > > A great success story for making money on a open source project. > > Cheers, > Torben > > P.S. Did you use my list comprehensions for Omaha or did you find something > better? > > > On Thu, Mar 15, 2012 at 22:29, Joel Reymont wrote: >> >> EA World Series of Poker (WSOP) [1] is powered by Erlang and my OpenPoker >> [2]. >> >> Just wanted to give you all a heads-up ;-). >> >> [1] >> http://toucharcade.com/2012/03/08/gdc-2012-ea-and-chillingo-showcasing-flight-control-rocket-burnout-crash-air-mail-and-more/ >> [2] https://github.com/wagerlabs/openpoker >> >> -------------------------------------------------------------------------- >> - for hire: mac osx device driver ninja, kernel extensions and usb drivers >> ---------------------+------------+--------------------------------------- >> http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont >> ---------------------+------------+--------------------------------------- >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > http://www.linkedin.com/in/torbenhoffmann > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm From jared.nance@REDACTED Fri Mar 16 02:18:47 2012 From: jared.nance@REDACTED (Jared Kofron) Date: Thu, 15 Mar 2012 18:18:47 -0700 Subject: [erlang-questions] Erlang meets physics In-Reply-To: <16736994.2.1331837804364.JavaMail.geo-discussion-forums@ynkv16> References: <16736994.2.1331837804364.JavaMail.geo-discussion-forums@ynkv16> Message-ID: <9B85B9B4-BA18-4FB5-8168-E4275AB036F0@gmail.com> Very cool... and unusual for Erlang. Well, I will be at Erlang Factory this year, so maybe some of us physicists (ex and otherwise) can discuss it over a beer. Jared K On Mar 15, 2012, at 11:56 AM, pietje wrote: > Hi All, > > Just my 2 cents. > I am an ex physicist too. Used to do research on QCD (Drell-Yan process) calculating Feynman diagrams. > > For a couple of years I've been busy writing an erlang program to handle Feynman diagrams algebraically. Lots of fun. > > regards, Pieter Rijken > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > On Monday, 12 March 2012 02:34:04 UTC+1, Jared Kofron wrote: > Hi All, > I've been using Erlang at work for a few years now, and I thought I'd throw my experience out there, as > my application is a little different than what you usually see on the list - I am a graduate student at the > Center for Nuclear Physics and Astrophysics at the University of Washington, and use Erlang extensively > in my work. > In my experience, something that Erlang is really great at but doesn't receive much attention for these days > is managing and interacting with hardware. In any physics experiment of even modest sizes, you wind up > having to keep track of the state of various pieces of equipment, often modify that state, and constantly > interrogate particular values. For example, we might want to change the current in a magnetic trap, turn > that trap off altogether, or simply read back the voltage drop across our superconducting magnet. > > So far, I have deployed Erlang in this zone for two separate experiments (SNO+, a large particle physics > experiment in Canada) and Project 8 (a small nuclear physics experiment here in Seattle). Both times have > been great successes, and I have found the reception of Erlang in this market to be great. In general, what > I have done is wrap a hardware management layer with some kind of outside world interface. For SNO+, we > used Webmachine and RESTful control, and for Project 8 we actually conduct all communication > by using CouchDB as a message passing interface. > > Physicists are suspicious creatures, but once you demonstrate the feature set that you get for practically > free with OTP, they see the advantage pretty quickly. On top of that, the development cycle for sophisticated > applications can be greatly reduced - more than once it made my group float to the top in terms of meeting > goals. > > In short, as far as I am concerned, Erlang has found a new niche in the world of Physics, and I intend to > spread the word as much as I can! > > Jared Kofron > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Fri Mar 16 03:35:47 2012 From: bob@REDACTED (Bob Ippolito) Date: Thu, 15 Mar 2012 19:35:47 -0700 Subject: [erlang-questions] OpenPoker and Erlang powers EA's World Series of Poker In-Reply-To: References: Message-ID: Nice! Congrats :) On Thursday, March 15, 2012, Joel Reymont wrote: > EA World Series of Poker (WSOP) [1] is powered by Erlang and my OpenPoker [2]. > > Just wanted to give you all a heads-up ;-). > > [1] http://toucharcade.com/2012/03/08/gdc-2012-ea-and-chillingo-showcasing-flight-control-rocket-burnout-crash-air-mail-and-more/ > [2] https://github.com/wagerlabs/openpoker > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave@REDACTED Fri Mar 16 05:25:15 2012 From: dave@REDACTED (David Goehrig) Date: Fri, 16 Mar 2012 00:25:15 -0400 Subject: [erlang-questions] OpenPoker and Erlang powers EA's World Series of Poker In-Reply-To: References: Message-ID: <76142DC1-785D-46B2-8E95-DA4B62F55BEF@nexttolast.com> Back when I did the ESPN WSOP we used Erlang. I also did a Friday Night Fights game for ESPN with the simulation in Ocaml & Erlang. It tends to do really well with distributed message passing apps for some reason :) -=-=- dave@REDACTED -=-=- On Mar 15, 2012, at 5:29 PM, Joel Reymont wrote: > EA World Series of Poker (WSOP) [1] is powered by Erlang and my OpenPoker [2]. > > Just wanted to give you all a heads-up ;-). > > [1] http://toucharcade.com/2012/03/08/gdc-2012-ea-and-chillingo-showcasing-flight-control-rocket-burnout-crash-air-mail-and-more/ > [2] https://github.com/wagerlabs/openpoker > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.eugene.turner@REDACTED Fri Mar 16 06:41:13 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Fri, 16 Mar 2012 14:41:13 +0900 Subject: [erlang-questions] [ANN] Brim - HTML templating library In-Reply-To: <3C4BF661-FAA1-4CF4-A70A-8C917A2E45B8@cs.otago.ac.nz> References: <6E4F3076-B078-4C47-B33E-561B43A1EEE2@gmail.com> <3C4BF661-FAA1-4CF4-A70A-8C917A2E45B8@cs.otago.ac.nz> Message-ID: Richard, I wouldn't consider autocompletion "programming by example", and especially not for purposes of discussion here. A successful result from autocompletion is still only a text-paste in a GUI, once and only once. A successful result in PBE is a step added to an algorithm specification that you'd (at least potentially) save and later reuse in many hands-off, unseen operations. Keyboard macros also aren't quite it, either, though they come much closer to the idea. Insofar as they contain parameterized search-replace commands, they involve recognition of explicitly described patterns (code) rather than inferring patterns and generating the code. Also, keyboard macros can't really do iteration (or did I miss something in their development?) The basic idea of PBE can work in a batch environment (and indeed was probably first applied there, if not under that rubric): here's the input, here's the output, give me back a best guess at a mapping specification. But thanks for pointing out autocompletion anyway. A non-interactive *kind* of autocompletion might work nicely for what I suggest. A web designer might want to include substantial amounts of text in order to get a sense of the balance of the layout. So for the web designer, my HTML example becomes The Title Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. But it would be ungainly, in the code, to include that whole greeked-text paragraph in the Erlang term for which you want to establish a mapping. As a coder, you'd probably prefer to supply the matching algorithm with a more concise input data structure like this: { <<"The Text">>, <<"Lorem ipsum ...">> } Here it should be understood that <<"[some text] ...">> says to match on Lorem ipsum * in the HTML template input, in effect a kind of autocompletion of the pattern specification. What's the point? It's to maximize results by taking complex communication between experts down to the bare minimum. There are people who can design gorgeous web layouts but who can't handle code at all -- including fairly minimal CSS selectors. There are programmers who can implement anti-gravity and controlled thermonuclear reactions in their sleep, but whose web page layouts are invariably, irretrievably ugly. It's like that line from that movie, "With my looks and your brains, we could go far." PBE could help more people go the distance, and in record time. -michael turner On Fri, Mar 16, 2012 at 6:06 AM, Richard O'Keefe wrote: > > On 16/03/2012, at 1:17 AM, Michael Turner wrote: > >> >> Dan Halbert pioneered PBE at Xerox PARC. It was later taken up as a >> research project at at Apple. Then it was championed by none other >> than Melinda Gates, at Microsoft. >> >> It never took off. > > No? ?There are programmers who swear by "Intellisense" and similar automatic > completion whizz-bang in their IDE. ?One of the Smalltalk systems I use, > Pharo, does this. ?I swear *at* it, but that's another story. ?The web browser > I mainly use offers an autocomplete list when I start to type a URL. ?Unlike > Pharo, this doesn't visually obscure what I'm trying to type, so I just use it, > and don't swear at it. ?Start typing a query in the Google box, and again an > autocomplete list pops up (but NOT in the way!) and very handy it is. > > That's small-scale programming by example. > > I don't know what key it is that I'm pressing by mistake, but it seems as if > every time I have to use vim and want to quit I end up in macro recording and > have to spend a long time finger-dancing on the keyboard just to quit. ?Macro > recording is precisely programming by example. > > The thing that Windows victims got unhappy about was *not* the system watching > what they were trying to do and getting *ready* to be helpful about it but the > system seizing control. ?If Clippy had stayed entirely off the screen unless > and until people pressed a special Ctrl-Alt-Help chord, it would still be here. > > I had to use Explorer the other day on someone's netbook, and I started typing > a URL and it seized control from me, saying in effect, "no, I know better than > you what you want, you really want to go to your home page, and to prove how > much better I know than you do, I'm going to delete your typing." ?PBE is not > the problem, interfering is the problem. > > From meetprashant007@REDACTED Fri Mar 16 09:55:00 2012 From: meetprashant007@REDACTED (Prashant Sharma) Date: Fri, 16 Mar 2012 14:25:00 +0530 Subject: [erlang-questions] OpenCL.framework cause my NIF library to crash In-Reply-To: References: <1331653757763-4469412.post@n4.nabble.com> Message-ID: was wondering, if everyone is already aware of already existing https://github.com/tonyrog/cl -P On Wed, Mar 14, 2012 at 1:39 PM, Alfredo Di Napoli wrote: > > On 13/mar/2012, at 17:54, Bob Ippolito wrote: > > Do you also have the -framework OpenCL flag? Static libraries don't > automatically link in their dynamic dependencies. > > > Yes I've compiled with this flag both my static framework library as well as > my shared NIF, but the problem still persist. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From alfredo.dinapoli@REDACTED Fri Mar 16 09:59:17 2012 From: alfredo.dinapoli@REDACTED (Alfredo Di Napoli) Date: Fri, 16 Mar 2012 09:59:17 +0100 Subject: [erlang-questions] OpenCL.framework cause my NIF library to crash In-Reply-To: References: <1331653757763-4469412.post@n4.nabble.com> Message-ID: <02565E01-CB38-4556-811E-6107BA3252E8@gmail.com> On 16/mar/2012, at 09:55, Prashant Sharma wrote: > was wondering, if everyone is already aware of already existing > https://github.com/tonyrog/cl > -P > Sure, but the strange thing is that I've discovered that even the "cl" project above cause the NIF to crash. In other words, I suspect it can be an Erlang VM bug. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Fri Mar 16 10:12:01 2012 From: tony@REDACTED (Tony Rogvall) Date: Fri, 16 Mar 2012 10:12:01 +0100 Subject: [erlang-questions] Erlang meets physics In-Reply-To: <16736994.2.1331837804364.JavaMail.geo-discussion-forums@ynkv16> References: <16736994.2.1331837804364.JavaMail.geo-discussion-forums@ynkv16> Message-ID: <85A125C0-97EA-4017-92E7-E9F499EF2BA1@rogvall.se> Wow :-) I like to see that, is it anywhere on git hub? Very early in early in my Erlang career I help a math friend to calculate betti numbers (simply put, calculate number of holes in an object given a triangulation of a surface description ) The oldest file is dated Jun 11 1992 :-) I have also implemented the AKS (prime number) algorithm in, it beat the C++ code without even trying to use all the cores (linear speedup) Just to mention a few of the math related stuff I have been using Erlang for. I totally agree Lots of fun! /Tony On 15 mar 2012, at 19:56, pietje wrote: > Hi All, > > Just my 2 cents. > I am an ex physicist too. Used to do research on QCD (Drell-Yan process) calculating Feynman diagrams. > > For a couple of years I've been busy writing an erlang program to handle Feynman diagrams algebraically. Lots of fun. > > regards, Pieter Rijken > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > On Monday, 12 March 2012 02:34:04 UTC+1, Jared Kofron wrote: > Hi All, > I've been using Erlang at work for a few years now, and I thought I'd throw my experience out there, as > my application is a little different than what you usually see on the list - I am a graduate student at the > Center for Nuclear Physics and Astrophysics at the University of Washington, and use Erlang extensively > in my work. > In my experience, something that Erlang is really great at but doesn't receive much attention for these days > is managing and interacting with hardware. In any physics experiment of even modest sizes, you wind up > having to keep track of the state of various pieces of equipment, often modify that state, and constantly > interrogate particular values. For example, we might want to change the current in a magnetic trap, turn > that trap off altogether, or simply read back the voltage drop across our superconducting magnet. > > So far, I have deployed Erlang in this zone for two separate experiments (SNO+, a large particle physics > experiment in Canada) and Project 8 (a small nuclear physics experiment here in Seattle). Both times have > been great successes, and I have found the reception of Erlang in this market to be great. In general, what > I have done is wrap a hardware management layer with some kind of outside world interface. For SNO+, we > used Webmachine and RESTful control, and for Project 8 we actually conduct all communication > by using CouchDB as a message passing interface. > > Physicists are suspicious creatures, but once you demonstrate the feature set that you get for practically > free with OTP, they see the advantage pretty quickly. On top of that, the development cycle for sophisticated > applications can be greatly reduced - more than once it made my group float to the top in terms of meeting > goals. > > In short, as far as I am concerned, Erlang has found a new niche in the world of Physics, and I intend to > spread the word as much as I can! > > Jared Kofron > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Fri Mar 16 10:29:34 2012 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 16 Mar 2012 09:29:34 +0000 Subject: [erlang-questions] OpenPoker and Erlang powers EA's World Series of Poker In-Reply-To: References: Message-ID: On Thu, Mar 15, 2012 at 9:48 PM, Torben Hoffmann wrote: > > P.S. Did you use my list comprehensions for Omaha or did you find something > better? I think I did use your suggestion ;-). Props to Torben for contributing to the success of OpenPoker! -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From janburse@REDACTED Fri Mar 16 10:30:22 2012 From: janburse@REDACTED (Jan Burse) Date: Fri, 16 Mar 2012 10:30:22 +0100 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: References: <4F606D90.5070206@fastmail.fm> Message-ID: <4F63082E.2000409@fastmail.fm> Hi, Looks my mac book pro is much slower... Sniff I don't have Erlang on it, since I do not build. But here are some figures that show the difference between JDK 1.6 / JDK 1.7 on a mac. I didn't use the micro benchmark I posted, I used my Prolog interpreter test suite: java version "1.6.0_27" Java(TM) SE Runtime Environment (build 1.6.0_27-b07-393-10M3527) Java HotSpot(TM) 64-Bit Server VM (build 20.2-b06-393, mixed mode) Total in 12049 (206 gc) ms openjdk version "1.7.0-u4-b225" OpenJDK Runtime Environment (build 1.7.0-u4-b225-20120119) OpenJDK 64-Bit Server VM (build 23.0-b10, mixed mode) Total in 10388 (92 gc) ms I don't see a difference -client versus -server for the test suite. So difference is not that big, but still! Bye P.S.: Correction of Richard O'Keefe Java measure done with JDK 1.6: 19.2 ms * 10388 / 12049 = 16.5 ms, how funny, on par. Richard O'Keefe schrieb: > Jan Burse's naive reverse for Erlang is not quite what we usually mean > by nrev; he measures the time to create the initial data list as well, > which the Java version doesn't. > > Revising the Erlang code slightly, I got 2.8 usec per call. > I also revised the Java code slightly, for example, NOT to > use accessor methods, and NOT to require any casting during > the benchmark. Also, both benchmarks were revised by > doing > > t0 = now > run a dummy benchmark calling an identity function > t1 = now > run the real benchmark > t2 = now > report ((t2 - t1) - (t1 - t0))/(double)number-of-iterations > > as is traditional in the naive reverse benchmark. > > The final score: > > Erlang: 2.8 usec per call > Java : 3.2 usec per call (reported as 3226.977 nsec) > > Machine: a MacBook pro with a 2GHz Intel core 2 Duo cpu running > Mac OS X 10.6.8. > > Multiplying these numbers by 6000 to match Jan Burse's: > Me Him Him, scaled > Erlang 16.8 m ~60 ms ~ 72.9 ms > Java 19.2 ms ~30 ms ~ 36.5 ms > (The scaled column assumes that the 2430 number is CPU > speed; if that's not what it is, ignore that column.) > > There's one tiny change that accounts for this. > > And it's the reason this benchmark really tells us nothing worth knowing. > From tony@REDACTED Fri Mar 16 11:10:49 2012 From: tony@REDACTED (Tony Rogvall) Date: Fri, 16 Mar 2012 11:10:49 +0100 Subject: [erlang-questions] OpenCL.framework cause my NIF library to crash In-Reply-To: <02565E01-CB38-4556-811E-6107BA3252E8@gmail.com> References: <1331653757763-4469412.post@n4.nabble.com> <02565E01-CB38-4556-811E-6107BA3252E8@gmail.com> Message-ID: I just tried it, and you are totally correct. The cl nif failed to load on R15B Darwin 11.3.0. What Erlang release and OS/release are you using ? /Tony On 16 mar 2012, at 09:59, Alfredo Di Napoli wrote: > > On 16/mar/2012, at 09:55, Prashant Sharma wrote: > >> was wondering, if everyone is already aware of already existing >> https://github.com/tonyrog/cl >> -P >> > > Sure, but the strange thing is that I've discovered that even the "cl" project above cause the NIF to crash. > In other words, I suspect it can be an Erlang VM bug. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From andy.richards.iit@REDACTED Fri Mar 16 11:13:56 2012 From: andy.richards.iit@REDACTED (Andy Richards) Date: Fri, 16 Mar 2012 10:13:56 +0000 Subject: [erlang-questions] downgrade / rollback a hot deployment without restarting OTP Message-ID: Hi All, I'm struggling to find documentation on how I perform a rollback of a unsuccessful OTP hot deployment. I'm currently manually creating my .appup file with upgrade and downgrade instructions and when I perform a release_handler:install_release("someApp") my application hot deploys successfully, excellent! What i cant figure out however is if there were a problem following an upgrade how I downgrade / rollback to the previous release? Ive tried release_handler:remove_release("someApp") unsuccessfully as my upgrade remains active and I understand that I can restart OTP before making the upgrade permanent however in the environment I'm working in a restart would be unacceptable. How do I downgrade / rollback a hot deployment without restarting OTP? Many thanks, Andy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfredo.dinapoli@REDACTED Fri Mar 16 11:21:52 2012 From: alfredo.dinapoli@REDACTED (Alfredo Di Napoli) Date: Fri, 16 Mar 2012 11:21:52 +0100 Subject: [erlang-questions] OpenCL.framework cause my NIF library to crash In-Reply-To: References: <1331653757763-4469412.post@n4.nabble.com> <02565E01-CB38-4556-811E-6107BA3252E8@gmail.com> Message-ID: <006A4A21-6F06-4D9B-993E-D1C3F7D6C647@gmail.com> On 16/mar/2012, at 11:10, Tony Rogvall wrote: > I just tried it, and you are totally correct. > > The cl nif failed to load on R15B Darwin 11.3.0. > What Erlang release and OS/release are you using ? > > /Tony > Hi Tony, I'm on a Mac Os X Lion (so SDK 10.7) with the latest Erlang distribution R15B. I suspect this isn't a problem with your bindings, though, because I'm the same guy who asked you help about OpenCL binding with NIF via email :) I've also tried a brain-dead simple NIF application (a stupid app that performs a square of a number), compiling it from command line (so no Xcode involved :) ) The NIF works fine, until I link ANY Mac OS Framework. In my example I've ONLY linked the AppKit.framework: bare in mind that actually the NIF does not uses it in any function call! The result? Abort trap! So I guess this could be a Erlang VM bug, related to Mac Os Frameworks dynamic linking process. Regards, Alfredo -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Fri Mar 16 11:34:26 2012 From: tony@REDACTED (Tony Rogvall) Date: Fri, 16 Mar 2012 11:34:26 +0100 Subject: [erlang-questions] OpenCL.framework cause my NIF library to crash In-Reply-To: <006A4A21-6F06-4D9B-993E-D1C3F7D6C647@gmail.com> References: <1331653757763-4469412.post@n4.nabble.com> <02565E01-CB38-4556-811E-6107BA3252E8@gmail.com> <006A4A21-6F06-4D9B-993E-D1C3F7D6C647@gmail.com> Message-ID: <582EF8C6-8605-44D3-B7B2-6FAD826EA69E@rogvall.se> The problem seem to be dlopen, that it must be called from the main thread, this used to work. Googling around I can see other people have this problem, the work around for some projects is to run erlang as: erl -smp disable However, this is not possible with the cl binding since it require SMP( sending event messages from various threads ) You can verify that the cl application loads ok with non smp, but then crash with enif_send: env==NULL on non-SMP VMAbort trap: 6 as it should :-) Any OTP takers ? Sverker / Bj?rn ? /Tony On 16 mar 2012, at 11:21, Alfredo Di Napoli wrote: > > On 16/mar/2012, at 11:10, Tony Rogvall wrote: > >> I just tried it, and you are totally correct. >> >> The cl nif failed to load on R15B Darwin 11.3.0. >> What Erlang release and OS/release are you using ? >> >> /Tony >> > > Hi Tony, I'm on a Mac Os X Lion (so SDK 10.7) with the latest Erlang distribution R15B. > I suspect this isn't a problem with your bindings, though, because I'm the same guy who asked you help about OpenCL binding with NIF via email :) > I've also tried a brain-dead simple NIF application (a stupid app that performs a square of a number), compiling it from command line (so no Xcode involved :) ) > The NIF works fine, until I link ANY Mac OS Framework. In my example I've ONLY linked the AppKit.framework: bare in mind that actually the NIF does not uses it in any function call! > The result? Abort trap! > > So I guess this could be a Erlang VM bug, related to Mac Os Frameworks dynamic linking process. > > Regards, > Alfredo > > > "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Fri Mar 16 12:09:54 2012 From: dangud@REDACTED (Dan Gudmundsson) Date: Fri, 16 Mar 2012 12:09:54 +0100 Subject: [erlang-questions] OpenCL.framework cause my NIF library to crash In-Reply-To: <582EF8C6-8605-44D3-B7B2-6FAD826EA69E@rogvall.se> References: <1331653757763-4469412.post@n4.nabble.com> <02565E01-CB38-4556-811E-6107BA3252E8@gmail.com> <006A4A21-6F06-4D9B-993E-D1C3F7D6C647@gmail.com> <582EF8C6-8605-44D3-B7B2-6FAD826EA69E@rogvall.se> Message-ID: That is a pain ... You can grab the main thread within erlang (undocumented ?) because the gui must also be run from the main thread, which means that you can not combine OpenCL with wx (or esdl). Or is it only the dlopen call that must be run from that thread? Apple sometimes not so great.. /Dan On Fri, Mar 16, 2012 at 11:34 AM, Tony Rogvall wrote: > The problem seem to be dlopen, that it must be called from the main thread, > this used to work. Googling around I can see other people have this problem, > the work around for some projects is to run erlang as: > > erl -smp disable > > However, this is not possible with the cl binding since it require SMP( > sending event messages from > various threads ) > You can verify that the cl application loads ok with non smp, but then crash > with > > enif_send: env==NULL on non-SMP VMAbort trap: 6 > > as it should :-) > > Any OTP takers ? Sverker / Bj?rn ? > > /Tony > > > > > > On 16 mar 2012, at 11:21, Alfredo Di Napoli wrote: > > > On 16/mar/2012, at 11:10, Tony Rogvall wrote: > > I just tried it, and you are totally correct. > > The cl nif failed to load on R15B Darwin 11.3.0. > What Erlang release and OS/release are you using ? > > /Tony > > > Hi Tony, I'm on a Mac Os X Lion (so SDK 10.7) with the latest Erlang > distribution R15B. > I suspect this isn't a problem with your bindings, though, because I'm the > same guy who asked you help about OpenCL binding with NIF via email :) > I've also tried a brain-dead simple NIF application (a stupid app that > performs a square of a number), compiling it from command line (so no Xcode > involved :) ) > The NIF works fine, until I link ANY Mac OS Framework. In my example I've > ONLY linked the AppKit.framework: bare in mind that actually the NIF does > not uses it in any function call! > The result? Abort trap! > > So I guess this could be a Erlang VM bug, related to Mac Os Frameworks > dynamic linking process. > > Regards, > Alfredo > > > > > "Installing applications can lead to corruption over time.?Applications > gradually write over each other's libraries, partial upgrades occur, user > and system errors happen, and minute changes may be unnoticeable and > difficult to fix" > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ward@REDACTED Fri Mar 16 12:13:49 2012 From: ward@REDACTED (Ward Bekker) Date: Fri, 16 Mar 2012 12:13:49 +0100 Subject: [erlang-questions] [ANN] Erltricity: An Erlang EEG/brainwave recording / charting app. Message-ID: <20120316111349.c65046f7@groupware.tty.nl> Erltricity is an Erlang/OTP Application for recording and displaying your EEG brainwaves. It implements the ThinkGear Socket Protocol from NeuroSky which is supported by the Mindwave headset http://store.neurosky.com/products/mindwave-1 . The status of the code is 'proof of concept'. More information: https://github.com/wardbekker/Erltricity Example output: https://img.skitch.com/20120316-f7yngmu28b5yq8xmusgabc13r3.jpg -------------- next part -------------- An HTML attachment was scrubbed... URL: From freeakk@REDACTED Fri Mar 16 12:57:33 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Fri, 16 Mar 2012 14:57:33 +0300 Subject: [erlang-questions] Google summer of code Message-ID: Hello, Are Erlang projects planning to participate at GSoC 2012? -- Best regards, Uvarov Michael From zabrane3@REDACTED Fri Mar 16 13:33:54 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 16 Mar 2012 13:33:54 +0100 Subject: [erlang-questions] [ANN] Erltricity: An Erlang EEG/brainwave recording / charting app. In-Reply-To: <20120316111349.c65046f7@groupware.tty.nl> References: <20120316111349.c65046f7@groupware.tty.nl> Message-ID: Hi Ward, On Mar 16, 2012, at 12:13 PM, Ward Bekker wrote: > https://github.com/wardbekker/Erltricity Maybe you've to choose another name for it (to avoid confusion with Erltricity): https://github.com/wardbekker/Erltricity Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From ward@REDACTED Fri Mar 16 13:37:23 2012 From: ward@REDACTED (Ward Bekker) Date: Fri, 16 Mar 2012 13:37:23 +0100 Subject: [erlang-questions] [ANN] Erltricity: An Erlang EEG/brainwave recording / charting app. In-Reply-To: Message-ID: <20120316123723.5ba2db06@groupware.tty.nl> Hi Zabrane, Confused here. Is there another project called Erltricity? _____ From: Zabrane Mickael [mailto:zabrane3@REDACTED] To: Ward Bekker [mailto:ward@REDACTED] Cc: erlang-questions@REDACTED Sent: Fri, 16 Mar 2012 13:33:54 +0100 Subject: Re: [erlang-questions] [ANN] Erltricity: An Erlang EEG/brainwave recording / charting app. Hi Ward, On Mar 16, 2012, at 12:13 PM, Ward Bekker wrote:https://github.com/wardbekker/Erltricity Maybe you've to choose another name for it (to avoid confusion with Erltricity): https://github.com/wardbekker/Erltricity Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela@REDACTED Fri Mar 16 13:40:07 2012 From: ingela@REDACTED (Ingela Andin) Date: Fri, 16 Mar 2012 13:40:07 +0100 Subject: [erlang-questions] ODBC Connectivity with Netezza In-Reply-To: References: Message-ID: Hi! I have never used Netezza have you tried Googling? First hit I got was http://www.connectionstrings.com/netezza Regards Ingela Erlang/OTP team - Ericsson AB 2012/3/15 Avinash Dhumane : > We have an integration glue application in Erlang (on Windows) that connects > to heterogeneous data sources SQL Server, AS/400, and Netezza via ODBC > application provided in the standard Erlang distribution. While the drill > works perfectly for SQL Server and AS/400, Netezza is encountering following > connectivity problem: > > Erlang R14B02 (erts-5.8.3) [smp:2:2] [rq:2] [async-threads:0] > > Eshell V5.8.3 ?(abort with ^G) > 1> odbc:start(). > ok > 2> odbc:connect("DSN=NZSQL;UID=TEST;PWD=TEST", []). > {error,"[Microsoft][ODBC Driver Manager] Data source name not found and no > default driver specified SQLSTATE IS: IM002 Connection to database failed."} > 3> > > Microsoft help link: > http://msdn.microsoft.com/en-us/library/windows/desktop/ms711810(v=vs.85).aspx > says: > > ? ? If the data source name cannot be found or ServerName is a null pointer, > ? ? and the default data source specification does not exist, the Driver > Manager > ? ? returns SQL_ERROR with SQLSTATE IM002 (Data source name not found and > ? ? no default driver specified). > > Obviously, this is not the case with our setup. We tried with both "User > DSN" and "System DSN" in Windows "ODBC Data Source Administrator" tool, but > the same error persists in both the cases. > > Does Netezza require different connection string? > > Please advise. > > Thanks > Avinash > > PS: We have NetezzaSQL driver (NSSQLODBC.DLL) version 4.06.05.10119 > installed on the Windows box that runs the Erlang application. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Fri Mar 16 13:41:51 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 16 Mar 2012 13:41:51 +0100 Subject: [erlang-questions] [ANN] Erltricity: An Erlang EEG/brainwave recording / charting app. In-Reply-To: <20120316123723.5ba2db06@groupware.tty.nl> References: <20120316123723.5ba2db06@groupware.tty.nl> Message-ID: I'm aware of only this one as I used it for year or two now. Regards, Zabrane On Mar 16, 2012, at 1:37 PM, Ward Bekker wrote: > Hi Zabrane, > > Confused here. Is there another project called Erltricity? > From: Zabrane Mickael [mailto:zabrane3@REDACTED] > To: Ward Bekker [mailto:ward@REDACTED] > Cc: erlang-questions@REDACTED > Sent: Fri, 16 Mar 2012 13:33:54 +0100 > Subject: Re: [erlang-questions] [ANN] Erltricity: An Erlang EEG/brainwave recording / charting app. > > Hi Ward, > > On Mar 16, 2012, at 12:13 PM, Ward Bekker wrote: >> https://github.com/wardbekker/Erltricity > > > > Maybe you've to choose another name for it (to avoid confusion with Erltricity): > https://github.com/wardbekker/Erltricity > > > Regards, > Zabrane > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela@REDACTED Fri Mar 16 13:44:25 2012 From: ingela@REDACTED (Ingela Andin) Date: Fri, 16 Mar 2012 13:44:25 +0100 Subject: [erlang-questions] SSH module and Agent Forwarding In-Reply-To: References: Message-ID: 2012/3/15 Ingela Andin : > ?Hi! > > ?I do not have an answer for agent-forwarding question at the moment, > ?but I would like to mention that if you want to use the key_cb option > ?you should use the upcoming version ssh-2.1 > > ?Regards Ingela Erlang/OTP team - Ericsson AB > >> 2012/3/13, Jeffrey Massung : >>> Has anyone been able to accomplish this? I've been scouring Google, >>> documentation, and not finding much help. Does it require using the >>> key_cb >>> option in ssh:connect? I don't care if this takes a chunk of code to get >>> working, but if someone has done this in the past and has some pointers >>> or >>> same code to work from, I'd be very appreciative. >>> >>> Thanks! >>> >>> Jeff M. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> From ward@REDACTED Fri Mar 16 13:47:09 2012 From: ward@REDACTED (Ward Bekker) Date: Fri, 16 Mar 2012 13:47:09 +0100 Subject: [erlang-questions] [ANN] Erltricity: An Erlang EEG/brainwave recording / charting app. In-Reply-To: Message-ID: <20120316124709.41badf37@groupware.tty.nl> Hi Zabrane, Wasn't aware of that. Do you have a project URL for me? Googling "Erltricity" only returns a single result. Regards, Ward From: Zabrane Mickael [mailto:zabrane3@REDACTED] To: Ward Bekker [mailto:ward@REDACTED] Cc: erlang-questions@REDACTED Sent: Fri, 16 Mar 2012 13:41:51 +0100 Subject: Re: [erlang-questions] [ANN] Erltricity: An Erlang EEG/brainwave recording / charting app. I'm aware of only this one as I used it for year or two now. Regards, Zabrane On Mar 16, 2012, at 1:37 PM, Ward Bekker wrote: Hi Zabrane, Confused here. Is there another project called Erltricity? _____ From: Zabrane Mickael [mailto:zabrane3@REDACTED] To: Ward Bekker [mailto:ward@REDACTED] Cc: erlang-questions@REDACTED Sent: Fri, 16 Mar 2012 13:33:54 +0100 Subject: Re: [erlang-questions] [ANN] Erltricity: An Erlang EEG/brainwave recording / charting app. Hi Ward, On Mar 16, 2012, at 12:13 PM, Ward Bekker wrote:https://github.com/wardbekker/Erltricity Maybe you've to choose another name for it (to avoid confusion with Erltricity): https://github.com/wardbekker/Erltricity Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Fri Mar 16 14:01:25 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 16 Mar 2012 14:01:25 +0100 Subject: [erlang-questions] [ANN] Erltricity: An Erlang EEG/brainwave recording / charting app. In-Reply-To: References: <20120316123723.5ba2db06@groupware.tty.nl> Message-ID: <4F6339A5.7020408@gmail.com> Most confused thread ever? I would guess that Zabrane is referring to https://github.com/mojombo/erlectricity /Richard On 03/16/2012 01:41 PM, Zabrane Mickael wrote: > I'm aware of only this one as I used it for year or two now. > > Regards, > Zabrane > > On Mar 16, 2012, at 1:37 PM, Ward Bekker wrote: > >> Hi Zabrane, >> >> Confused here. Is there another project called Erltricity? >> >> ------------------------------------------------------------------------ >> *From:*Zabrane Mickael [mailto:zabrane3@REDACTED] >> *To:*Ward Bekker [mailto:ward@REDACTED] >> *Cc:*erlang-questions@REDACTED >> *Sent:*Fri, 16 Mar 2012 13:33:54 +0100 >> *Subject:*Re: [erlang-questions] [ANN] Erltricity: An Erlang >> EEG/brainwave recording / charting app. >> >> Hi Ward, >> >> On Mar 16, 2012, at 12:13 PM, Ward Bekker wrote: >>> https://github.com/wardbekker/Erltricity >> >> >> Maybe you've to choose another name for it (to avoid confusion >> with Erltricity): >> https://github.com/wardbekker/Erltricity >> >> >> Regards, >> Zabrane >> >> > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From avinash@REDACTED Fri Mar 16 14:04:20 2012 From: avinash@REDACTED (Avinash Dhumane) Date: Fri, 16 Mar 2012 18:34:20 +0530 Subject: [erlang-questions] ODBC Connectivity with Netezza In-Reply-To: References: Message-ID: Hi Ingela: The below lines from the Erlang Documentation helped to narrow down to the cause: >>> The Erlang ODBC application consists of both Erlang and C code. The C code is delivered as a precompiled executable for windows, solaris and linux (SLES10) in the commercial build. In the open source distribution it is built the same way as all other application using configure and make. You may want to provide the the path to your ODBC libraries using --with-odbc=PATH. >>>>> Since ODBC connectivity with AS/400 worked fine, I checked why it worked. Apparently, on "Windows 2008 Server" installation (in virtualised environment), there are driver DLLs with *same* name but different content, placed in 2 different directories - System32 and SysWOW64. Netezza driver DLL was found only in System32 but not in SysWOW64, unlike for AS/400 driver. That was the only difference in the setup. Upon highlighting this difference to IBM Support, we were advised of installing both 32-bit and 64-bit drivers of Netezza, and that resolved the issue. The connection string was not the issue; the same set of 3 parameters - DSN, UID and PWD - are working fine across all ODBC-interfaced DBMS that we encountered so far. The Erlang's standard ODBC application is cool! Regards, Avinash On Fri, 16 Mar 2012 18:10:07 +0530, Ingela Andin wrote: > Hi! > > I have never used Netezza have you tried Googling? > > First hit I got was > > http://www.connectionstrings.com/netezza > > Regards Ingela Erlang/OTP team - Ericsson AB > > 2012/3/15 Avinash Dhumane : >> We have an integration glue application in Erlang (on Windows) that >> connects >> to heterogeneous data sources SQL Server, AS/400, and Netezza via ODBC >> application provided in the standard Erlang distribution. While the >> drill >> works perfectly for SQL Server and AS/400, Netezza is encountering >> following >> connectivity problem: >> >> Erlang R14B02 (erts-5.8.3) [smp:2:2] [rq:2] [async-threads:0] >> >> Eshell V5.8.3 (abort with ^G) >> 1> odbc:start(). >> ok >> 2> odbc:connect("DSN=NZSQL;UID=TEST;PWD=TEST", []). >> {error,"[Microsoft][ODBC Driver Manager] Data source name not found and >> no >> default driver specified SQLSTATE IS: IM002 Connection to database >> failed."} >> 3> >> >> Microsoft help link: >> http://msdn.microsoft.com/en-us/library/windows/desktop/ms711810(v=vs.85).aspx >> says: >> >> If the data source name cannot be found or ServerName is a null >> pointer, >> and the default data source specification does not exist, the Driver >> Manager >> returns SQL_ERROR with SQLSTATE IM002 (Data source name not found >> and >> no default driver specified). >> >> Obviously, this is not the case with our setup. We tried with both "User >> DSN" and "System DSN" in Windows "ODBC Data Source Administrator" tool, >> but >> the same error persists in both the cases. >> >> Does Netezza require different connection string? >> >> Please advise. >> >> Thanks >> Avinash >> >> PS: We have NetezzaSQL driver (NSSQLODBC.DLL) version 4.06.05.10119 >> installed on the Windows box that runs the Erlang application. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -- Using Opera's revolutionary email client: http://www.opera.com/mail/ From wmknapik@REDACTED Fri Mar 16 14:15:48 2012 From: wmknapik@REDACTED (Wojciech Knapik) Date: Fri, 16 Mar 2012 14:15:48 +0100 Subject: [erlang-questions] Boostrapping a clique of replicating mnesia nodes In-Reply-To: References: Message-ID: On Thu, Mar 15, 2012 at 3:48 PM, Wojciech Knapik wrote: I have a set of nodes (discovered via net_adm:world(), but others will > likely join in later) where I want to keep a fully replicated mnesia > table(s). > > All the nodes are identical in terms of code and all of them start at > roughly the same time on different machines. > > If the database doesn't exist, one of the nodes needs to create the table > with {disc_copies, nodes()}. If it does exist and it's fully replicated, > then we're done, but if it's not fully replicated, the nodes that are not > replicating need to join in. > > I put together this code http://pastie.org/3601148 to illustrate my idea. > It might be bad for whatever reason, but it's just to get my message across. > > Does this algorithm make sense ? Is this the right direction I'm headed ? > I'd be grateful for any hints. > Anyone ? Even a yes/no would be better than nothing... WK -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Fri Mar 16 14:43:46 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 16 Mar 2012 14:43:46 +0100 Subject: [erlang-questions] [ANN] Erltricity: An Erlang EEG/brainwave recording / charting app. In-Reply-To: <4F6339A5.7020408@gmail.com> References: <20120316123723.5ba2db06@groupware.tty.nl> <4F6339A5.7020408@gmail.com> Message-ID: On Mar 16, 2012, at 2:01 PM, Richard Carlsson wrote: > Most confused thread ever? I would guess that Zabrane is referring to https://github.com/mojombo/erlectricity Yep ... right Richard ;-) Regards, Zabrane From per.melin@REDACTED Fri Mar 16 14:54:38 2012 From: per.melin@REDACTED (Per Melin) Date: Fri, 16 Mar 2012 14:54:38 +0100 Subject: [erlang-questions] downgrade / rollback a hot deployment without restarting OTP In-Reply-To: References: Message-ID: On Mar 16, 2012, at 11:13 , Andy Richards wrote: > I'm struggling to find documentation on how I perform a rollback of a > unsuccessful OTP hot deployment. I'm currently manually creating my .appup > file with upgrade and downgrade instructions and when I perform a > release_handler:install_release("someApp") my application hot deploys > successfully, excellent! What i cant figure out however is if there were a > problem following an upgrade how I downgrade / rollback to the previous > release? It's been a long time since I last did this by hand, but if I remember correctly you simply do release_handler:install_release("OldVer") to roll back. From tony@REDACTED Fri Mar 16 15:41:11 2012 From: tony@REDACTED (Tony Rogvall) Date: Fri, 16 Mar 2012 15:41:11 +0100 Subject: [erlang-questions] OpenCL.framework cause my NIF library to crash In-Reply-To: References: <1331653757763-4469412.post@n4.nabble.com> <02565E01-CB38-4556-811E-6107BA3252E8@gmail.com> <006A4A21-6F06-4D9B-993E-D1C3F7D6C647@gmail.com> <582EF8C6-8605-44D3-B7B2-6FAD826EA69E@rogvall.se> Message-ID: Hi Dan! I think (and hope) that it is only dlopen that needs to be run from the main thread, is the undocumented feature accessible from the NIF's ? Otherwise what about having a special async thread you could run in the main thread? Apple still have not fixed poll ! Lets start a Facebook group ;-) /Tony On 16 mar 2012, at 12:09, Dan Gudmundsson wrote: > That is a pain ... > > You can grab the main thread within erlang (undocumented ?) because > the gui must also be run from the > main thread, which means that you can not combine OpenCL with wx (or esdl). > > Or is it only the dlopen call that must be run from that thread? > > Apple sometimes not so great.. > > /Dan > > On Fri, Mar 16, 2012 at 11:34 AM, Tony Rogvall wrote: >> The problem seem to be dlopen, that it must be called from the main thread, >> this used to work. Googling around I can see other people have this problem, >> the work around for some projects is to run erlang as: >> >> erl -smp disable >> >> However, this is not possible with the cl binding since it require SMP( >> sending event messages from >> various threads ) >> You can verify that the cl application loads ok with non smp, but then crash >> with >> >> enif_send: env==NULL on non-SMP VMAbort trap: 6 >> >> as it should :-) >> >> Any OTP takers ? Sverker / Bj?rn ? >> >> /Tony >> >> >> >> >> >> On 16 mar 2012, at 11:21, Alfredo Di Napoli wrote: >> >> >> On 16/mar/2012, at 11:10, Tony Rogvall wrote: >> >> I just tried it, and you are totally correct. >> >> The cl nif failed to load on R15B Darwin 11.3.0. >> What Erlang release and OS/release are you using ? >> >> /Tony >> >> >> Hi Tony, I'm on a Mac Os X Lion (so SDK 10.7) with the latest Erlang >> distribution R15B. >> I suspect this isn't a problem with your bindings, though, because I'm the >> same guy who asked you help about OpenCL binding with NIF via email :) >> I've also tried a brain-dead simple NIF application (a stupid app that >> performs a square of a number), compiling it from command line (so no Xcode >> involved :) ) >> The NIF works fine, until I link ANY Mac OS Framework. In my example I've >> ONLY linked the AppKit.framework: bare in mind that actually the NIF does >> not uses it in any function call! >> The result? Abort trap! >> >> So I guess this could be a Erlang VM bug, related to Mac Os Frameworks >> dynamic linking process. >> >> Regards, >> Alfredo >> >> >> >> >> "Installing applications can lead to corruption over time. Applications >> gradually write over each other's libraries, partial upgrades occur, user >> and system errors happen, and minute changes may be unnoticeable and >> difficult to fix" >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From straightflush@REDACTED Fri Mar 16 17:11:35 2012 From: straightflush@REDACTED (AD) Date: Fri, 16 Mar 2012 12:11:35 -0400 Subject: [erlang-questions] Push a message to cowboy websockets Message-ID: Hello, I have cowboy setup and working to send/receive websockets over a socket connection. I am trying to figure out how to initiate a message server side over that channel (without it just being a reply). I am envisioning exposing a webservice on the cowboy service that would then push a message to all connected users (or maybe a subset based on some criteria). Does anyone know if this is possible and if so how to implement? Cheers, -AD -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Mar 16 17:20:01 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Fri, 16 Mar 2012 17:20:01 +0100 Subject: [erlang-questions] Push a message to cowboy websockets In-Reply-To: References: Message-ID: <4F636831.1020204@ninenines.eu> Hey! The general idea is to have your websocket handler register itself using gproc[1] (or through a central gen_server) inside the websocket_init/3 callback, and then have your service push messages to all registered handlers which you can then catch in websocket_info/3. I mention gproc because it's infinitely easier with it, as you only have to register all your processes under one property and then send a message to that property which will multicast it to all registered processes. And when your websocket closes, gproc takes care of removing your process from the list of registered processes, so you really have to worry only about 2 lines of code to do everything you need. Good luck! [1] https://github.com/uwiger/gproc On 03/16/2012 05:11 PM, AD wrote: > Hello, > > I have cowboy setup and working to send/receive websockets over a > socket connection. I am trying to figure out how to initiate a message > server side over that channel (without it just being a reply). I am > envisioning exposing a webservice on the cowboy service that would then > push a message to all connected users (or maybe a subset based on some > criteria). > > Does anyone know if this is possible and if so how to implement? > > Cheers, > -AD > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Erlang Cowboy Nine Nines From sam@REDACTED Fri Mar 16 18:49:24 2012 From: sam@REDACTED (Samuel Elliott) Date: Fri, 16 Mar 2012 17:49:24 +0000 Subject: [erlang-questions] downgrade / rollback a hot deployment without restarting OTP In-Reply-To: References: Message-ID: On Fri, Mar 16, 2012 at 1:54 PM, Per Melin wrote: > On Mar 16, 2012, at 11:13 , Andy Richards wrote: > >> I'm struggling to find documentation on how I perform a rollback of a >> unsuccessful OTP hot deployment. I'm currently manually creating my .appup >> file with upgrade and downgrade instructions and when I perform a >> release_handler:install_release("someApp") my application hot deploys >> successfully, excellent! What i cant figure out however is if there were a >> problem following an upgrade how I downgrade / rollback to the previous >> release? > > It's been a long time since I last did this by hand, but if I remember correctly you simply do release_handler:install_release("OldVer") to roll back. Seems so, reading Ferd's Guide: http://learnyousomeerlang.com/relups#upgrading-the-release Sam > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Samuel Elliott sam@REDACTED http://lenary.co.uk/ +44 (0)7891 993 664 From straightflush@REDACTED Fri Mar 16 19:01:03 2012 From: straightflush@REDACTED (AD) Date: Fri, 16 Mar 2012 14:01:03 -0400 Subject: [erlang-questions] Push a message to cowboy websockets In-Reply-To: <4F636831.1020204@ninenines.eu> References: <4F636831.1020204@ninenines.eu> Message-ID: Thanks, so in websocket/init gproc:reg({p, l, {?MODULE, WSBroadcast}}). Then in my webservice i can do Msg = "Test broadcast". gproc:send({p, l, {?MODULE,WSBroadcast}}, {self(), {?MODULE,WSBroadcast}, Msg}). How do you catch this in websocket_info/3 to formulate a reply? Also noticed a new pub/sub module for gproc not sure if this helps simplify some of this https://github.com/uwiger/gproc/blob/master/src/gproc_ps.erl Thanks! -AD On Fri, Mar 16, 2012 at 12:20 PM, Lo?c Hoguin wrote: > Hey! > > The general idea is to have your websocket handler register itself using > gproc[1] (or through a central gen_server) inside the websocket_init/3 > callback, and then have your service push messages to all registered > handlers which you can then catch in websocket_info/3. > > I mention gproc because it's infinitely easier with it, as you only have > to register all your processes under one property and then send a message > to that property which will multicast it to all registered processes. And > when your websocket closes, gproc takes care of removing your process from > the list of registered processes, so you really have to worry only about 2 > lines of code to do everything you need. > > Good luck! > > [1] https://github.com/uwiger/**gproc > > > On 03/16/2012 05:11 PM, AD wrote: > >> Hello, >> >> I have cowboy setup and working to send/receive websockets over a >> socket connection. I am trying to figure out how to initiate a message >> server side over that channel (without it just being a reply). I am >> envisioning exposing a webservice on the cowboy service that would then >> push a message to all connected users (or maybe a subset based on some >> criteria). >> >> Does anyone know if this is possible and if so how to implement? >> >> Cheers, >> -AD >> >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Mar 16 19:06:57 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Fri, 16 Mar 2012 19:06:57 +0100 Subject: [erlang-questions] Push a message to cowboy websockets In-Reply-To: References: <4F636831.1020204@ninenines.eu> Message-ID: <4F638141.4040405@ninenines.eu> Just like you would catch any message, you match: websocket_info({Pid, {Module, WSBroadcast}, Msg}, Req, State) When using gproc:send/2, the second arg is what you want to match in the first arg of websocket_info/3. All messages the websocket process receives are given to you in websocket_info/3. Didn't try the new pubsub, it looks interesting though. On 03/16/2012 07:01 PM, AD wrote: > Thanks, so in websocket/init > > gproc:reg({p, l, {?MODULE, WSBroadcast}}). > > Then in my webservice i can do > > Msg = "Test broadcast". > gproc:send({p, l, {?MODULE,WSBroadcast}}, {self(), > {?MODULE,WSBroadcast}, Msg}). > > How do you catch this in websocket_info/3 to formulate a reply? > > Also noticed a new pub/sub module for gproc not sure if this helps > simplify some of this > https://github.com/uwiger/gproc/blob/master/src/gproc_ps.erl > > Thanks! > -AD > > On Fri, Mar 16, 2012 at 12:20 PM, Lo?c Hoguin > wrote: > > Hey! > > The general idea is to have your websocket handler register itself > using gproc[1] (or through a central gen_server) inside the > websocket_init/3 callback, and then have your service push messages > to all registered handlers which you can then catch in websocket_info/3. > > I mention gproc because it's infinitely easier with it, as you only > have to register all your processes under one property and then send > a message to that property which will multicast it to all registered > processes. And when your websocket closes, gproc takes care of > removing your process from the list of registered processes, so you > really have to worry only about 2 lines of code to do everything you > need. > > Good luck! > > [1] https://github.com/uwiger/__gproc > > > On 03/16/2012 05:11 PM, AD wrote: > > Hello, > > I have cowboy setup and working to send/receive websockets over a > socket connection. I am trying to figure out how to initiate a > message > server side over that channel (without it just being a reply). I am > envisioning exposing a webservice on the cowboy service that > would then > push a message to all connected users (or maybe a subset based > on some > criteria). > > Does anyone know if this is possible and if so how to implement? > > Cheers, > -AD > > > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/__listinfo/erlang-questions > > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > -- Lo?c Hoguin Erlang Cowboy Nine Nines From straightflush@REDACTED Fri Mar 16 21:05:32 2012 From: straightflush@REDACTED (AD) Date: Fri, 16 Mar 2012 16:05:32 -0400 Subject: [erlang-questions] Push a message to cowboy websockets In-Reply-To: <4F638141.4040405@ninenines.eu> References: <4F636831.1020204@ninenines.eu> <4F638141.4040405@ninenines.eu> Message-ID: Thanks, getting some stacktraces though... ** Handler ad_test terminating in websocket_init/3 for the reason error:badarg i have *-define(WSBroadcast,"wsbroadcast").* then in websocket_init *gproc:reg({p, l, {?MODULE, ?WSBroadcast}})* * * then in my handle() method in cowboy for a certain path *{URLPath,_Req2} = cowboy_http_req:path(Req),* * case lists:nth(1,URLPath) of* * <<"sendit">> -> * * %io:format("Sending websocket !!~n"),* * Msg = "Test broadcast",* * gproc:send({p, l, {?MODULE,?WSBroadcast}}, {self(), {?MODULE,?WSBroadcast}, Msg});* * * then in websocket_info w*ebsocket_info({_PID,{_MODULE,_WSBroadcast},Msg},Req,State) ->* * {reply, {text, <>}, Req, State, hibernate}.* Any thoughts ? Looks like something with ets:insert ** Stacktrace: [{ets,insert_new, [gproc, [{{{p,l,{ad_test,"wsbroadcast"}},<0.153.0>}, <0.153.0>,undefined}, {{<0.153.0>,{p,l,{ad_test,"wsbroadcast"}}},[]}]], []}, Thanks again for the help -AD On Fri, Mar 16, 2012 at 2:06 PM, Lo?c Hoguin wrote: > Just like you would catch any message, you match: > > websocket_info({Pid, {Module, WSBroadcast}, Msg}, Req, State) > > When using gproc:send/2, the second arg is what you want to match in the > first arg of websocket_info/3. All messages the websocket process receives > are given to you in websocket_info/3. > > Didn't try the new pubsub, it looks interesting though. > > > On 03/16/2012 07:01 PM, AD wrote: > >> Thanks, so in websocket/init >> >> gproc:reg({p, l, {?MODULE, WSBroadcast}}). >> >> Then in my webservice i can do >> >> Msg = "Test broadcast". >> gproc:send({p, l, {?MODULE,WSBroadcast}}, {self(), >> {?MODULE,WSBroadcast}, Msg}). >> >> How do you catch this in websocket_info/3 to formulate a reply? >> >> Also noticed a new pub/sub module for gproc not sure if this helps >> simplify some of this >> https://github.com/uwiger/**gproc/blob/master/src/gproc_**ps.erl >> >> Thanks! >> -AD >> >> On Fri, Mar 16, 2012 at 12:20 PM, Lo?c Hoguin > > wrote: >> >> Hey! >> >> The general idea is to have your websocket handler register itself >> using gproc[1] (or through a central gen_server) inside the >> websocket_init/3 callback, and then have your service push messages >> to all registered handlers which you can then catch in >> websocket_info/3. >> >> I mention gproc because it's infinitely easier with it, as you only >> have to register all your processes under one property and then send >> a message to that property which will multicast it to all registered >> processes. And when your websocket closes, gproc takes care of >> removing your process from the list of registered processes, so you >> really have to worry only about 2 lines of code to do everything you >> need. >> >> Good luck! >> >> [1] https://github.com/uwiger/__**gproc< >> https://github.com/uwiger/**gproc > >> >> >> >> On 03/16/2012 05:11 PM, AD wrote: >> >> Hello, >> >> I have cowboy setup and working to send/receive websockets over a >> socket connection. I am trying to figure out how to initiate a >> message >> server side over that channel (without it just being a reply). I >> am >> envisioning exposing a webservice on the cowboy service that >> would then >> push a message to all connected users (or maybe a subset based >> on some >> criteria). >> >> Does anyone know if this is possible and if so how to implement? >> >> Cheers, >> -AD >> >> >> ______________________________**___________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > >> http://erlang.org/mailman/__**listinfo/erlang-questions >> >> >> > >> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> >> >> > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Mar 16 21:18:12 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Fri, 16 Mar 2012 21:18:12 +0100 Subject: [erlang-questions] Push a message to cowboy websockets In-Reply-To: References: <4F636831.1020204@ninenines.eu> <4F638141.4040405@ninenines.eu> Message-ID: <4F63A004.7030802@ninenines.eu> Did you start gproc? application:start(gproc) On 03/16/2012 09:05 PM, AD wrote: > Thanks, getting some stacktraces though... > > ** Handler ad_test terminating in websocket_init/3 > for the reason error:badarg > > i have > > /-define(WSBroadcast,"wsbroadcast")./ > > then in websocket_init > > /gproc:reg({p, l, {?MODULE, ?WSBroadcast}})/ > / > / > then in my handle() method in cowboy for a certain path > > /{URLPath,_Req2} = cowboy_http_req:path(Req),/ > / case lists:nth(1,URLPath) of/ > /<<"sendit">> -> / > /%io:format("Sending websocket !!~n"),/ > /Msg = "Test broadcast",/ > /gproc:send({p, l, {?MODULE,?WSBroadcast}}, {self(), > {?MODULE,?WSBroadcast}, Msg});/ > / > / > then in websocket_info > > w/ebsocket_info({_PID,{_MODULE,_WSBroadcast},Msg},Req,State) ->/ > /{reply, {text, <>}, Req, State, hibernate}./ > > Any thoughts ? Looks like something with ets:insert > > ** Stacktrace: [{ets,insert_new, > [gproc, > [{{{p,l,{ad_test,"wsbroadcast"}},<0.153.0>}, > <0.153.0>,undefined}, > {{<0.153.0>,{p,l,{ad_test,"wsbroadcast"}}},[]}]], > []}, > > Thanks again for the help > -AD > > On Fri, Mar 16, 2012 at 2:06 PM, Lo?c Hoguin > wrote: > > Just like you would catch any message, you match: > > websocket_info({Pid, {Module, WSBroadcast}, Msg}, Req, State) > > When using gproc:send/2, the second arg is what you want to match in > the first arg of websocket_info/3. All messages the websocket > process receives are given to you in websocket_info/3. > > Didn't try the new pubsub, it looks interesting though. > > > On 03/16/2012 07:01 PM, AD wrote: > > Thanks, so in websocket/init > > gproc:reg({p, l, {?MODULE, WSBroadcast}}). > > Then in my webservice i can do > > Msg = "Test broadcast". > gproc:send({p, l, {?MODULE,WSBroadcast}}, {self(), > {?MODULE,WSBroadcast}, Msg}). > > How do you catch this in websocket_info/3 to formulate a reply? > > Also noticed a new pub/sub module for gproc not sure if this helps > simplify some of this > https://github.com/uwiger/__gproc/blob/master/src/gproc___ps.erl > > > Thanks! > -AD > > On Fri, Mar 16, 2012 at 12:20 PM, Lo?c Hoguin > > >> wrote: > > Hey! > > The general idea is to have your websocket handler register > itself > using gproc[1] (or through a central gen_server) inside the > websocket_init/3 callback, and then have your service push > messages > to all registered handlers which you can then catch in > websocket_info/3. > > I mention gproc because it's infinitely easier with it, as > you only > have to register all your processes under one property and > then send > a message to that property which will multicast it to all > registered > processes. And when your websocket closes, gproc takes care of > removing your process from the list of registered processes, > so you > really have to worry only about 2 lines of code to do > everything you > need. > > Good luck! > > [1] https://github.com/uwiger/____gproc > > > > > > > On 03/16/2012 05:11 PM, AD wrote: > > Hello, > > I have cowboy setup and working to send/receive > websockets over a > socket connection. I am trying to figure out how to > initiate a > message > server side over that channel (without it just being a > reply). I am > envisioning exposing a webservice on the cowboy service that > would then > push a message to all connected users (or maybe a subset > based > on some > criteria). > > Does anyone know if this is possible and if so how to > implement? > > Cheers, > -AD > > > ___________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > > > http://erlang.org/mailman/____listinfo/erlang-questions > > > > > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > -- Lo?c Hoguin Erlang Cowboy Nine Nines From straightflush@REDACTED Fri Mar 16 21:56:22 2012 From: straightflush@REDACTED (AD) Date: Fri, 16 Mar 2012 16:56:22 -0400 Subject: [erlang-questions] Push a message to cowboy websockets In-Reply-To: <4F63A004.7030802@ninenines.eu> References: <4F636831.1020204@ninenines.eu> <4F638141.4040405@ninenines.eu> <4F63A004.7030802@ninenines.eu> Message-ID: Sigh, no. Thanks, its working perfectly now. -AD On Fri, Mar 16, 2012 at 4:18 PM, Lo?c Hoguin wrote: > Did you start gproc? application:start(gproc) > > > On 03/16/2012 09:05 PM, AD wrote: > >> Thanks, getting some stacktraces though... >> >> ** Handler ad_test terminating in websocket_init/3 >> for the reason error:badarg >> >> i have >> >> /-define(WSBroadcast,"**wsbroadcast")./ >> >> then in websocket_init >> >> /gproc:reg({p, l, {?MODULE, ?WSBroadcast}})/ >> / >> >> / >> then in my handle() method in cowboy for a certain path >> >> /{URLPath,_Req2} = cowboy_http_req:path(Req),/ >> / case lists:nth(1,URLPath) of/ >> /<<"sendit">> -> / >> /%io:format("Sending websocket !!~n"),/ >> /Msg = "Test broadcast",/ >> /gproc:send({p, l, {?MODULE,?WSBroadcast}}, {self(), >> {?MODULE,?WSBroadcast}, Msg});/ >> / >> / >> then in websocket_info >> >> w/ebsocket_info({_PID,{_**MODULE,_WSBroadcast},Msg},Req,**State) ->/ >> /{reply, {text, <>}, Req, State, hibernate}./ >> >> >> Any thoughts ? Looks like something with ets:insert >> >> ** Stacktrace: [{ets,insert_new, >> [gproc, >> [{{{p,l,{ad_test,"wsbroadcast"**}},<0.153.0>}, >> <0.153.0>,undefined}, >> {{<0.153.0>,{p,l,{ad_test,"**wsbroadcast"}}},[]}]], >> []}, >> >> Thanks again for the help >> -AD >> >> On Fri, Mar 16, 2012 at 2:06 PM, Lo?c Hoguin > > wrote: >> >> Just like you would catch any message, you match: >> >> websocket_info({Pid, {Module, WSBroadcast}, Msg}, Req, State) >> >> When using gproc:send/2, the second arg is what you want to match in >> the first arg of websocket_info/3. All messages the websocket >> process receives are given to you in websocket_info/3. >> >> Didn't try the new pubsub, it looks interesting though. >> >> >> On 03/16/2012 07:01 PM, AD wrote: >> >> Thanks, so in websocket/init >> >> gproc:reg({p, l, {?MODULE, WSBroadcast}}). >> >> Then in my webservice i can do >> >> Msg = "Test broadcast". >> gproc:send({p, l, {?MODULE,WSBroadcast}}, {self(), >> {?MODULE,WSBroadcast}, Msg}). >> >> How do you catch this in websocket_info/3 to formulate a reply? >> >> Also noticed a new pub/sub module for gproc not sure if this helps >> simplify some of this >> https://github.com/uwiger/__**gproc/blob/master/src/gproc___** >> ps.erl >> >> >> > >> >> Thanks! >> -AD >> >> On Fri, Mar 16, 2012 at 12:20 PM, Lo?c Hoguin >> >> >> wrote: >> >> Hey! >> >> The general idea is to have your websocket handler register >> itself >> using gproc[1] (or through a central gen_server) inside the >> websocket_init/3 callback, and then have your service push >> messages >> to all registered handlers which you can then catch in >> websocket_info/3. >> >> I mention gproc because it's infinitely easier with it, as >> you only >> have to register all your processes under one property and >> then send >> a message to that property which will multicast it to all >> registered >> processes. And when your websocket closes, gproc takes care of >> removing your process from the list of registered processes, >> so you >> really have to worry only about 2 lines of code to do >> everything you >> need. >> >> Good luck! >> >> [1] https://github.com/uwiger/____**gproc >> >> > >> >> >> >> >> >> >> >> >> On 03/16/2012 05:11 PM, AD wrote: >> >> Hello, >> >> I have cowboy setup and working to send/receive >> websockets over a >> socket connection. I am trying to figure out how to >> initiate a >> message >> server side over that channel (without it just being a >> reply). I am >> envisioning exposing a webservice on the cowboy service >> that >> would then >> push a message to all connected users (or maybe a subset >> based >> on some >> criteria). >> >> Does anyone know if this is possible and if so how to >> implement? >> >> Cheers, >> -AD >> >> >> ______________________________**_____________________ >> >> erlang-questions mailing list >> erlang-questions@REDACTED >> > >> >> >> >> >> http://erlang.org/mailman/____**listinfo/erlang-questions >> >> > >> >> >> >> >> >> >> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> >> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> >> >> > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.hughes@REDACTED Sat Mar 17 10:31:12 2012 From: john.hughes@REDACTED (John Hughes) Date: Sat, 17 Mar 2012 10:31:12 +0100 Subject: [erlang-questions] Nitrogen start page Message-ID: I'm using nitrogen on inets to serve a mostly-static web site, with a little dynamic content here and there. In particular, the first page is static--index.html as usual. But when I just browse to the domain name, nitrogen uses index.beam to serve the request--and if there is no index.beam, then the request fails. I've hacked around this by providing an index.beam that uses wf:redirect to go to index.html, which is then served statically. But this means that the browser sees a brief "Redirecting..." message... and perhaps risks Google thinking that's what the page contains! Is there a way to configure nitrogen so that it uses index.html as the start page directly? John -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Sat Mar 17 12:16:51 2012 From: gumm@REDACTED (Jesse Gumm) Date: Sat, 17 Mar 2012 06:16:51 -0500 Subject: [erlang-questions] Nitrogen start page In-Reply-To: References: Message-ID: Hi John, Aside from my usual recommendation of "have the proxy deal with it", the solution would be to roll your own route handler. I'd say copy the current default route handler from nitrogen to a new module and modify the results of the route("/") call: https://github.com/nitrogen/nitrogen_core/blob/master/src/handlers/route/dynamic_route_handler.erl#L57 to be something like: {static_file, "/index.html"} You can install this new handler module in your src dir, and then to make sure nitrogen loads the handler, modify your src/nitrogen_inets.erl: ... nitrogen:init_request(RequestBridge, ResponseBridge), %% Obviously use whatever you name your module %% instead of my_custom_route_handler nitrogen:handler(my_custom_route_handler,[]), nitrogen:run(). Does that make any sense at all or is the late night/early morning distorting my thoughts? -Jesse On Sat, Mar 17, 2012 at 4:31 AM, John Hughes wrote: > I'm using nitrogen on inets?to serve a mostly-static web site, with a little > dynamic content here and there. In particular, the first page is > static--index.html as usual. But when I just browse to the domain name, > nitrogen uses index.beam to serve the request--and if there is no > index.beam, then the request fails. > > I've hacked around this by providing an index.beam that uses wf:redirect to > go to index.html, which is then served statically. But this means that the > browser sees a brief "Redirecting..." message... and perhaps risks Google > thinking that's what the page contains! Is there a way to configure nitrogen > so that it uses index.html as the start page directly? > > John > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm From tony@REDACTED Sat Mar 17 17:03:50 2012 From: tony@REDACTED (Tony Rogvall) Date: Sat, 17 Mar 2012 17:03:50 +0100 Subject: [erlang-questions] OpenCL.framework cause my NIF library to crash In-Reply-To: References: <1331653757763-4469412.post@n4.nabble.com> <02565E01-CB38-4556-811E-6107BA3252E8@gmail.com> <006A4A21-6F06-4D9B-993E-D1C3F7D6C647@gmail.com> <582EF8C6-8605-44D3-B7B2-6FAD826EA69E@rogvall.se> Message-ID: <79CD582A-6073-4ACF-9825-2F1E85C801EE@rogvall.se> Update. running cl works on 32 bit emulator. So that is a great workaround, while poking around in the subject. /Tony On 16 mar 2012, at 15:41, Tony Rogvall wrote: > Hi Dan! > > I think (and hope) that it is only dlopen that needs to be run from the main thread, > is the undocumented feature accessible from the NIF's ? > > Otherwise what about having a special async thread you could run in the main thread? > > Apple still have not fixed poll ! Lets start a Facebook group ;-) > > /Tony > > On 16 mar 2012, at 12:09, Dan Gudmundsson wrote: > >> That is a pain ... >> >> You can grab the main thread within erlang (undocumented ?) because >> the gui must also be run from the >> main thread, which means that you can not combine OpenCL with wx (or esdl). >> >> Or is it only the dlopen call that must be run from that thread? >> >> Apple sometimes not so great.. >> >> /Dan >> >> On Fri, Mar 16, 2012 at 11:34 AM, Tony Rogvall wrote: >>> The problem seem to be dlopen, that it must be called from the main thread, >>> this used to work. Googling around I can see other people have this problem, >>> the work around for some projects is to run erlang as: >>> >>> erl -smp disable >>> >>> However, this is not possible with the cl binding since it require SMP( >>> sending event messages from >>> various threads ) >>> You can verify that the cl application loads ok with non smp, but then crash >>> with >>> >>> enif_send: env==NULL on non-SMP VMAbort trap: 6 >>> >>> as it should :-) >>> >>> Any OTP takers ? Sverker / Bj?rn ? >>> >>> /Tony >>> >>> >>> >>> >>> >>> On 16 mar 2012, at 11:21, Alfredo Di Napoli wrote: >>> >>> >>> On 16/mar/2012, at 11:10, Tony Rogvall wrote: >>> >>> I just tried it, and you are totally correct. >>> >>> The cl nif failed to load on R15B Darwin 11.3.0. >>> What Erlang release and OS/release are you using ? >>> >>> /Tony >>> >>> >>> Hi Tony, I'm on a Mac Os X Lion (so SDK 10.7) with the latest Erlang >>> distribution R15B. >>> I suspect this isn't a problem with your bindings, though, because I'm the >>> same guy who asked you help about OpenCL binding with NIF via email :) >>> I've also tried a brain-dead simple NIF application (a stupid app that >>> performs a square of a number), compiling it from command line (so no Xcode >>> involved :) ) >>> The NIF works fine, until I link ANY Mac OS Framework. In my example I've >>> ONLY linked the AppKit.framework: bare in mind that actually the NIF does >>> not uses it in any function call! >>> The result? Abort trap! >>> >>> So I guess this could be a Erlang VM bug, related to Mac Os Frameworks >>> dynamic linking process. >>> >>> Regards, >>> Alfredo >>> >>> >>> >>> >>> "Installing applications can lead to corruption over time. Applications >>> gradually write over each other's libraries, partial upgrades occur, user >>> and system errors happen, and minute changes may be unnoticeable and >>> difficult to fix" >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> > > "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfredo.dinapoli@REDACTED Sat Mar 17 17:12:26 2012 From: alfredo.dinapoli@REDACTED (Alfredo Di Napoli) Date: Sat, 17 Mar 2012 17:12:26 +0100 Subject: [erlang-questions] OpenCL.framework cause my NIF library to crash In-Reply-To: <79CD582A-6073-4ACF-9825-2F1E85C801EE@rogvall.se> References: <1331653757763-4469412.post@n4.nabble.com> <02565E01-CB38-4556-811E-6107BA3252E8@gmail.com> <006A4A21-6F06-4D9B-993E-D1C3F7D6C647@gmail.com> <582EF8C6-8605-44D3-B7B2-6FAD826EA69E@rogvall.se> <79CD582A-6073-4ACF-9825-2F1E85C801EE@rogvall.se> Message-ID: So you think the problem may be caused by 64 bit? I suppose that, in order to work, it need a 32 bit version of Erlang, isn't it? Bye A. Il giorno 17 marzo 2012 17:03, Tony Rogvall ha scritto: > Update. > > running cl works on 32 bit emulator. > So that is a great workaround, while poking around in the subject. > > /Tony > > On 16 mar 2012, at 15:41, Tony Rogvall wrote: > > Hi Dan! > > I think (and hope) that it is only dlopen that needs to be run from the > main thread, > is the undocumented feature accessible from the NIF's ? > > Otherwise what about having a special async thread you could run in the > main thread? > > Apple still have not fixed poll ! Lets start a Facebook group ;-) > > /Tony > > On 16 mar 2012, at 12:09, Dan Gudmundsson wrote: > > That is a pain ... > > You can grab the main thread within erlang (undocumented ?) because > the gui must also be run from the > main thread, which means that you can not combine OpenCL with wx (or esdl). > > Or is it only the dlopen call that must be run from that thread? > > Apple sometimes not so great.. > > /Dan > > On Fri, Mar 16, 2012 at 11:34 AM, Tony Rogvall wrote: > > The problem seem to be dlopen, that it must be called from the main thread, > > this used to work. Googling around I can see other people have this > problem, > > the work around for some projects is to run erlang as: > > > erl -smp disable > > > However, this is not possible with the cl binding since it require SMP( > > sending event messages from > > various threads ) > > You can verify that the cl application loads ok with non smp, but then > crash > > with > > > enif_send: env==NULL on non-SMP VMAbort trap: 6 > > > as it should :-) > > > Any OTP takers ? Sverker / Bj?rn ? > > > /Tony > > > > > > > On 16 mar 2012, at 11:21, Alfredo Di Napoli wrote: > > > > On 16/mar/2012, at 11:10, Tony Rogvall wrote: > > > I just tried it, and you are totally correct. > > > The cl nif failed to load on R15B Darwin 11.3.0. > > What Erlang release and OS/release are you using ? > > > /Tony > > > > Hi Tony, I'm on a Mac Os X Lion (so SDK 10.7) with the latest Erlang > > distribution R15B. > > I suspect this isn't a problem with your bindings, though, because I'm the > > same guy who asked you help about OpenCL binding with NIF via email :) > > I've also tried a brain-dead simple NIF application (a stupid app that > > performs a square of a number), compiling it from command line (so no Xcode > > involved :) ) > > The NIF works fine, until I link ANY Mac OS Framework. In my example I've > > ONLY linked the AppKit.framework: bare in mind that actually the NIF does > > not uses it in any function call! > > The result? Abort trap! > > > So I guess this could be a Erlang VM bug, related to Mac Os Frameworks > > dynamic linking process. > > > Regards, > > Alfredo > > > > > > "Installing applications can lead to corruption over time. Applications > > gradually write over each other's libraries, partial upgrades occur, user > > and system errors happen, and minute changes may be unnoticeable and > > difficult to fix" > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > "Installing applications can lead to corruption over time. Applications > gradually write over each other's libraries, partial upgrades occur, user > and system errors happen, and minute changes may be unnoticeable and > difficult to fix" > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > "Installing applications can lead to corruption over time. Applications > gradually write over each other's libraries, partial upgrades occur, user > and system errors happen, and minute changes may be unnoticeable and > difficult to fix" > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Sat Mar 17 17:14:14 2012 From: tony@REDACTED (Tony Rogvall) Date: Sat, 17 Mar 2012 17:14:14 +0100 Subject: [erlang-questions] OpenCL.framework cause my NIF library to crash In-Reply-To: References: <1331653757763-4469412.post@n4.nabble.com> <02565E01-CB38-4556-811E-6107BA3252E8@gmail.com> <006A4A21-6F06-4D9B-993E-D1C3F7D6C647@gmail.com> <582EF8C6-8605-44D3-B7B2-6FAD826EA69E@rogvall.se> <79CD582A-6073-4ACF-9825-2F1E85C801EE@rogvall.se> Message-ID: <443FED8F-0DD1-4118-BF18-5728ABFEEC02@rogvall.se> On 17 mar 2012, at 17:12, Alfredo Di Napoli wrote: > So you think the problem may be caused by 64 bit? Yepp. > I suppose that, in order to work, it need a 32 bit version of Erlang, isn't it? ./configure --enable-m32-build; make /Tony > Bye > A. > > Il giorno 17 marzo 2012 17:03, Tony Rogvall ha scritto: > Update. > > running cl works on 32 bit emulator. > So that is a great workaround, while poking around in the subject. > > /Tony > > On 16 mar 2012, at 15:41, Tony Rogvall wrote: > >> Hi Dan! >> >> I think (and hope) that it is only dlopen that needs to be run from the main thread, >> is the undocumented feature accessible from the NIF's ? >> >> Otherwise what about having a special async thread you could run in the main thread? >> >> Apple still have not fixed poll ! Lets start a Facebook group ;-) >> >> /Tony >> >> On 16 mar 2012, at 12:09, Dan Gudmundsson wrote: >> >>> That is a pain ... >>> >>> You can grab the main thread within erlang (undocumented ?) because >>> the gui must also be run from the >>> main thread, which means that you can not combine OpenCL with wx (or esdl). >>> >>> Or is it only the dlopen call that must be run from that thread? >>> >>> Apple sometimes not so great.. >>> >>> /Dan >>> >>> On Fri, Mar 16, 2012 at 11:34 AM, Tony Rogvall wrote: >>>> The problem seem to be dlopen, that it must be called from the main thread, >>>> this used to work. Googling around I can see other people have this problem, >>>> the work around for some projects is to run erlang as: >>>> >>>> erl -smp disable >>>> >>>> However, this is not possible with the cl binding since it require SMP( >>>> sending event messages from >>>> various threads ) >>>> You can verify that the cl application loads ok with non smp, but then crash >>>> with >>>> >>>> enif_send: env==NULL on non-SMP VMAbort trap: 6 >>>> >>>> as it should :-) >>>> >>>> Any OTP takers ? Sverker / Bj?rn ? >>>> >>>> /Tony >>>> >>>> >>>> >>>> >>>> >>>> On 16 mar 2012, at 11:21, Alfredo Di Napoli wrote: >>>> >>>> >>>> On 16/mar/2012, at 11:10, Tony Rogvall wrote: >>>> >>>> I just tried it, and you are totally correct. >>>> >>>> The cl nif failed to load on R15B Darwin 11.3.0. >>>> What Erlang release and OS/release are you using ? >>>> >>>> /Tony >>>> >>>> >>>> Hi Tony, I'm on a Mac Os X Lion (so SDK 10.7) with the latest Erlang >>>> distribution R15B. >>>> I suspect this isn't a problem with your bindings, though, because I'm the >>>> same guy who asked you help about OpenCL binding with NIF via email :) >>>> I've also tried a brain-dead simple NIF application (a stupid app that >>>> performs a square of a number), compiling it from command line (so no Xcode >>>> involved :) ) >>>> The NIF works fine, until I link ANY Mac OS Framework. In my example I've >>>> ONLY linked the AppKit.framework: bare in mind that actually the NIF does >>>> not uses it in any function call! >>>> The result? Abort trap! >>>> >>>> So I guess this could be a Erlang VM bug, related to Mac Os Frameworks >>>> dynamic linking process. >>>> >>>> Regards, >>>> Alfredo >>>> >>>> >>>> >>>> >>>> "Installing applications can lead to corruption over time. Applications >>>> gradually write over each other's libraries, partial upgrades occur, user >>>> and system errors happen, and minute changes may be unnoticeable and >>>> difficult to fix" >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >> >> "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" > > > > "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Sat Mar 17 19:24:53 2012 From: jwatte@REDACTED (Jon Watte) Date: Sat, 17 Mar 2012 11:24:53 -0700 Subject: [erlang-questions] [OT] Re: GPL vs. whatever [was: Erlang UUID] In-Reply-To: <4F5D736A.8070901@meetinghouse.net> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> <4F5D736A.8070901@meetinghouse.net> Message-ID: The problem with GPL, even for a business that releases the source, is that it becomes a lot harder to accept contributions from the rest of the world. With GPL 3, the IP provisions make that pretty much a non-starter for a business operating in the US business climate. Thus, a GPL release (or AGPL release) from a commercial entity into the world pretty much guarantees that it will be a one-way street, where fixes won't work their way back up-stream. I think the restrictions the GPL puts on use and management of a source base make it significantly less free than BSD/MIT, or (my favorite) public domain. Sincerely, Jon Watte -- "I pledge allegiance to the flag of the United States of America, and to the republic for which it stands, one nation indivisible, with liberty and justice for all." ~ Adopted by U.S. Congress, June 22, 1942 On Sun, Mar 11, 2012 at 8:54 PM, Miles Fidelman wrote: > Just to throw in a different aspect of the GPL vs. BSD discussion: > > IMHO, GPL is a far better license than BSD for a developer that intends to > commercialize a product. > > The initial developer (copyright holder) always has the option to release > code under a dual license - GPL, BSD, or whatever for an open source > release, something more restrictive for the commercial product (potentially > with proprietary extensions). > > With GPL, you pretty much eliminate any competition - anybody else who > extends the code is faced with copyleft considerations, they CAN'T take > your code, combine it with their own code, and slap a proprietary license > around the assemblage. With BSD, or Apache, (or LGPL for that matter), > they can. > > Of course, if you dual-license your code under GPL and a proprietary > license, things can come back to haunt you if you want to incorporate > community-generated extensions into your upstream code base. In that case > the GPL and copyleft apply to you. > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Sat Mar 17 19:59:06 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 17 Mar 2012 19:59:06 +0100 Subject: [erlang-questions] [OT] Re: GPL vs. whatever [was: Erlang UUID] In-Reply-To: References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> <4F5D736A.8070901@meetinghouse.net> Message-ID: <4F64DEFA.6040104@cs.ntua.gr> On 03/17/2012 07:24 PM, Jon Watte wrote: > The problem with GPL, even for a business that releases the source, is > that it becomes a lot harder to accept contributions from the rest of > the world. With GPL 3, the IP provisions make that pretty much a > non-starter for a business operating in the US business climate. Thus, a > GPL release (or AGPL release) from a commercial entity into the world > pretty much guarantees that it will be a one-way street, where fixes > won't work their way back up-stream. I really do not understand what sort of situation and/or business climate you are describing. Suppose I use a software X from company/organization/some developers which was released under GPL and I find a bug in it and correct it. What exactly is it that prevents me from sending the fixes back to the company/organization/developers of X for possible inclusion in the next release? Similarly if I enhance X with some additional functionality. What does business climate have to do with sending bug reports, bug fixes or enhancements? Why is this a one-way street as you claim? Kostis From mattevans123@REDACTED Sun Mar 18 00:27:35 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Sat, 17 Mar 2012 19:27:35 -0400 Subject: [erlang-questions] Erlang shell - JCL Mode Message-ID: Hi I am working on a system where we could (hopefully) have deployments of many nodes (dimensioned to be in the 100's). It would be really nice if we could use JCL to switch between shells (using a central management shell) without having to enter JCL and doing r 'node@REDACTED' for each remote node in the system. Is there any magic that makes this possible? Thanks Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sun Mar 18 00:33:32 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Sun, 18 Mar 2012 00:33:32 +0100 Subject: [erlang-questions] [OT] Re: GPL vs. whatever [was: Erlang UUID] In-Reply-To: <4F64DEFA.6040104@cs.ntua.gr> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> <4F5D736A.8070901@meetinghouse.net> <4F64DEFA.6040104@cs.ntua.gr> Message-ID: <4F651F4C.9090000@ninenines.eu> On 03/17/2012 07:59 PM, Kostis Sagonas wrote: > On 03/17/2012 07:24 PM, Jon Watte wrote: >> The problem with GPL, even for a business that releases the source, is >> that it becomes a lot harder to accept contributions from the rest of >> the world. With GPL 3, the IP provisions make that pretty much a >> non-starter for a business operating in the US business climate. Thus, a >> GPL release (or AGPL release) from a commercial entity into the world >> pretty much guarantees that it will be a one-way street, where fixes >> won't work their way back up-stream. > > I really do not understand what sort of situation and/or business > climate you are describing. > > Suppose I use a software X from company/organization/some developers > which was released under GPL and I find a bug in it and correct it. What > exactly is it that prevents me from sending the fixes back to the > company/organization/developers of X for possible inclusion in the next > release? Similarly if I enhance X with some additional functionality. > What does business climate have to do with sending bug reports, bug > fixes or enhancements? Why is this a one-way street as you claim? Suppose you and your competitors were using the same open source project as a basis for their platform. Fixing bugs, or improving the software performance, and not contributing upstream gives you an advantage over your competition. You have the fixes, they don't. If it's (A)GPL, you need to publish these fixes, and lose your advantage. If it's BSD, you can use it and keep your fixes to yourself and gain an advantage over your competition. Either way the changes wouldn't be pushed back to upstream directly, because GPL only forces you to publish the changes, not feed them back to upstream. So chances are upstream wouldn't get the improvements anyway. Your competitors would, though, because they know you and they'll go get your sources directly. Your choice then becomes: do I want businesses to think twice before using my software? -- Lo?c Hoguin Erlang Cowboy Nine Nines From mjtruog@REDACTED Sun Mar 18 02:08:00 2012 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 17 Mar 2012 18:08:00 -0700 Subject: [erlang-questions] [OT] Re: GPL vs. whatever [was: Erlang UUID] In-Reply-To: <4F651F4C.9090000@ninenines.eu> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> <4F5D736A.8070901@meetinghouse.net> <4F64DEFA.6040104@cs.ntua.gr> <4F651F4C.9090000@ninenines.eu> Message-ID: <4F653570.90508@gmail.com> On 03/17/2012 04:33 PM, Lo?c Hoguin wrote: > On 03/17/2012 07:59 PM, Kostis Sagonas wrote: >> On 03/17/2012 07:24 PM, Jon Watte wrote: >>> The problem with GPL, even for a business that releases the source, is >>> that it becomes a lot harder to accept contributions from the rest of >>> the world. With GPL 3, the IP provisions make that pretty much a >>> non-starter for a business operating in the US business climate. Thus, a >>> GPL release (or AGPL release) from a commercial entity into the world >>> pretty much guarantees that it will be a one-way street, where fixes >>> won't work their way back up-stream. >> >> I really do not understand what sort of situation and/or business >> climate you are describing. >> >> Suppose I use a software X from company/organization/some developers >> which was released under GPL and I find a bug in it and correct it. What >> exactly is it that prevents me from sending the fixes back to the >> company/organization/developers of X for possible inclusion in the next >> release? Similarly if I enhance X with some additional functionality. >> What does business climate have to do with sending bug reports, bug >> fixes or enhancements? Why is this a one-way street as you claim? > > Suppose you and your competitors were using the same open source project as a basis for their platform. Fixing bugs, or improving the software performance, and not contributing upstream gives you an advantage over your competition. You have the fixes, they don't. > > If it's (A)GPL, you need to publish these fixes, and lose your advantage. If it's BSD, you can use it and keep your fixes to yourself and gain an advantage over your competition. > > Either way the changes wouldn't be pushed back to upstream directly, because GPL only forces you to publish the changes, not feed them back to upstream. So chances are upstream wouldn't get the improvements anyway. Your competitors would, though, because they know you and they'll go get your sources directly. > > Your choice then becomes: do I want businesses to think twice before using my software? > I agree that fixes could be seen as a competitive advantage. However, in practice, I think that becomes rare when you compare BSD projects to GPL. The BSD projects are generally known for their quality, whereas the GPL projects are generally known as feature-driven. So, the difference can be seen between projects like OpenBSD and Ubuntu Linux. OpenBSD is very concerned about code quality and eliminating any possible security concerns, so they are unlikely to have new features that open the potential for security problems (especially remote ones, but local ones also). Ubuntu Linux is best understood when broken apart. Ubuntu is about usability and a slow graphical interface to improve Linux usability. Linux is the kernel and supporting code that is known for security problems, though never as bad as Windows. When you consider the middle-of-the-road, you have FreeBSD which offers more features and decent usability, so closer to Linux, but with more concerns about security and code quality. NetBSD is another, but more of a focus on supporting many architectures. FreeBSD is the one that Apple saw fit to include within OSX. So, thinking about these different projects, the commercial users of the BSD software are unlikely to have the knowledge to fix errors easily unless they are paying one of the people involved in the project. If they are paying a person involved within the project, then the change will go back into the public source code. This is generally how it must work to provide the high quality software they have been during the past years. Your suggestion provides a conspiracy theory that is meant to justify GPL usage by assuming a greedy company that wanted to keep bugfixes as a competitive advantage would somehow care about the legality of the license. However, in reality, I believe the situation is very different. From alex.arnon@REDACTED Sun Mar 18 09:05:41 2012 From: alex.arnon@REDACTED (Alex Arnon) Date: Sun, 18 Mar 2012 10:05:41 +0200 Subject: [erlang-questions] OpenPoker and Erlang powers EA's World Series of Poker In-Reply-To: References: Message-ID: Congratulations!!! On Fri, Mar 16, 2012 at 11:29 AM, Joel Reymont wrote: > On Thu, Mar 15, 2012 at 9:48 PM, Torben Hoffmann > wrote: > > > > P.S. Did you use my list comprehensions for Omaha or did you find > something > > better? > > I think I did use your suggestion ;-). > > Props to Torben for contributing to the success of OpenPoker! > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattevans123@REDACTED Sun Mar 18 13:54:55 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Sun, 18 Mar 2012 08:54:55 -0400 Subject: [erlang-questions] Erlang shell - JCL Mode In-Reply-To: References: Message-ID: I managed to do it by changing user_drv.erl. You can connect the nodes to create a mesh (e.g. net_adm:ping/1) and then from JCL do: r all e.g. (manager@REDACTED)4> nodes().['bob@REDACTED','sue@REDACTED', 'matt@REDACTED','jane@REDACTED'](manager@REDACTED)5> User switch command --> r all --> j 1 {shell,start,[init]} 2 {'bob@REDACTED',shell,start,[]} 3 {'sue@REDACTED',shell,start,[]} 4 {'matt@REDACTED',shell,start,[]} 5* {'jane@REDACTED',shell,start,[]} --> Seems to work.... Attached is a patch file (Erlang R15B) for those that are interested... Matt From: mattevans123@REDACTED To: erlang-questions@REDACTED Date: Sat, 17 Mar 2012 19:27:35 -0400 Subject: [erlang-questions] Erlang shell - JCL Mode Hi I am working on a system where we could (hopefully) have deployments of many nodes (dimensioned to be in the 100's). It would be really nice if we could use JCL to switch between shells (using a central management shell) without having to enter JCL and doing r 'node@REDACTED' for each remote node in the system. Is there any magic that makes this possible? Thanks Matt _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: user_drv.erl.patch Type: application/octet-stream Size: 1069 bytes Desc: not available URL: From kostis@REDACTED Sun Mar 18 14:28:29 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Sun, 18 Mar 2012 14:28:29 +0100 Subject: [erlang-questions] [OT] Re: GPL vs. whatever [was: Erlang UUID] In-Reply-To: <4F651F4C.9090000@ninenines.eu> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> <4F5D736A.8070901@meetinghouse.net> <4F64DEFA.6040104@cs.ntua.gr> <4F651F4C.9090000@ninenines.eu> Message-ID: <4F65E2FD.9000001@cs.ntua.gr> On 03/18/2012 12:33 AM, Lo?c Hoguin wrote: > On 03/17/2012 07:59 PM, Kostis Sagonas wrote: >> On 03/17/2012 07:24 PM, Jon Watte wrote: >>> The problem with GPL, even for a business that releases the source, is >>> that it becomes a lot harder to accept contributions from the rest of >>> the world. With GPL 3, the IP provisions make that pretty much a >>> non-starter for a business operating in the US business climate. Thus, a >>> GPL release (or AGPL release) from a commercial entity into the world >>> pretty much guarantees that it will be a one-way street, where fixes >>> won't work their way back up-stream. >> >> I really do not understand what sort of situation and/or business >> climate you are describing. >> >> Suppose I use a software X from company/organization/some developers >> which was released under GPL and I find a bug in it and correct it. What >> exactly is it that prevents me from sending the fixes back to the >> company/organization/developers of X for possible inclusion in the next >> release? Similarly if I enhance X with some additional functionality. >> What does business climate have to do with sending bug reports, bug >> fixes or enhancements? Why is this a one-way street as you claim? > > Suppose you and your competitors were using the same open source project > as a basis for their platform. Fixing bugs, or improving the software > performance, and not contributing upstream gives you an advantage over > your competition. You have the fixes, they don't. Seems to me that you are writing about a completely different issue from the one in the original post. In any case, you are not replying to my question. The original post wrote: The problem with GPL, even for a business that releases the source, is that it becomes a lot harder to accept contributions from the rest of the world. With GPL 3, the IP provisions make that pretty much a non-starter for a business operating in the US business climate. So the situation we are discussing is one where some developers have released software X under GPL and company C has chosen to use that software and finds a bug in it. Why is it "harder (for the original developers, I guess) to accept contributions (from company C)"? I really do not understand this argument. This is the question I am asking Jon. Regarding the following: > If it's (A)GPL, you need to publish these fixes, and lose your > advantage. If it's BSD, you can use it and keep your fixes to yourself > and gain an advantage over your competition. > > Either way the changes wouldn't be pushed back to upstream directly, > because GPL only forces you to publish the changes, not feed them back > to upstream. So chances are upstream wouldn't get the improvements > anyway. Your competitors would, though, because they know you and > they'll go get your sources directly. > > Your choice then becomes: do I want businesses to think twice before > using my software? you are of course right, but these arguments concern the issue of what license developers of open source software choose for their software. If the goal is to maximize the users of a particular piece of software, then I agree that perhaps a BSD style license is a better choice than (A)GPL. But this is not what we are discussing here: we are discussing that a particular company has *chosen* to use some open source software and why the fact that this software comes with a GPL license makes the situation worse ("a one-way street" in the words of the original post) in terms of user contributions to it than a software released under a BSD style license. I do not understand the difference (if any). Kostis From dan@REDACTED Sun Mar 18 17:05:24 2012 From: dan@REDACTED (Daniel Dormont) Date: Sun, 18 Mar 2012 12:05:24 -0400 Subject: [erlang-questions] [OT] Re: GPL vs. whatever [was: Erlang UUID] In-Reply-To: <4F64DEFA.6040104@cs.ntua.gr> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> <4F5D736A.8070901@meetinghouse.net> <4F64DEFA.6040104@cs.ntua.gr> Message-ID: In the simple case, it's pretty straightforward: BSD - I can make fixes in your code and send them to you, and it's cool because both of us could release our own proprietary version based on those if we want GPL - I can make fixes in your code and send them to you, and it's cool because neither of us can make our own proprietary version Except that, as mentioned above, some companies use the "dual-license" approach so that they and they and they alone can produce a proprietary version of their own code. I assume RMS must find this terribly ironic, but it is only the GPL and not BSD that enables this. But in this case, upstream contributions are a problem unless the contributor is required to give permission to the original author to dual-license the code. But if you were a potential contributor to such a project, wouldn't you think twice about it? Dan On Sat, Mar 17, 2012 at 2:59 PM, Kostis Sagonas wrote: > On 03/17/2012 07:24 PM, Jon Watte wrote: > >> The problem with GPL, even for a business that releases the source, is >> that it becomes a lot harder to accept contributions from the rest of >> the world. With GPL 3, the IP provisions make that pretty much a >> non-starter for a business operating in the US business climate. Thus, a >> GPL release (or AGPL release) from a commercial entity into the world >> pretty much guarantees that it will be a one-way street, where fixes >> won't work their way back up-stream. >> > > I really do not understand what sort of situation and/or business climate > you are describing. > > Suppose I use a software X from company/organization/some developers which > was released under GPL and I find a bug in it and correct it. What exactly > is it that prevents me from sending the fixes back to the > company/organization/**developers of X for possible inclusion in the next > release? Similarly if I enhance X with some additional functionality. What > does business climate have to do with sending bug reports, bug fixes or > enhancements? Why is this a one-way street as you claim? > > Kostis > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.melin@REDACTED Sun Mar 18 18:34:59 2012 From: per.melin@REDACTED (Per Melin) Date: Sun, 18 Mar 2012 18:34:59 +0100 Subject: [erlang-questions] [ANN] Brim - HTML templating library In-Reply-To: References: <6E4F3076-B078-4C47-B33E-561B43A1EEE2@gmail.com> <8D0C933B-4231-4A82-A3F5-B253332C155A@gmail.com> Message-ID: <4364B0F5-F6E0-4423-91B5-35FADB4EB617@gmail.com> On Mar 15, 2012, at 21:42 , Bob Ippolito wrote: > You could also consider going down the road of parse transforms to keep the string syntax but get rid of the runtime string parsing. Good idea. I've written an optional parse transform now. The performance gain should in most cases be negligible. To parse a fairly complex selector takes ~5 ?s on my computer. Still, if e.g. you are filling a table with a large data set you could otherwise end up parsing the same selectors thousands of times at runtime for a single page. > Personally I like the CSS selector syntax. It's terse but not hard to understand, and you already know it if you've done pretty much any client-side web development. Even with an alternative syntax you're still going to need to learn a subset of the same keywords (id, first_child, etc.) which is I think much more work than picking up the grammar. Yes, I'm partial to CSS selectors too. But as O'Keefe likes to say; "STRINGS ARE WRONG". At least problems and inconveniences with escaping should be minimal in this case. From mfidelman@REDACTED Sun Mar 18 18:40:50 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Sun, 18 Mar 2012 13:40:50 -0400 Subject: [erlang-questions] [OT] Re: GPL vs. whatever [was: Erlang UUID] In-Reply-To: References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> <4F5D736A.8070901@meetinghouse.net> <4F64DEFA.6040104@cs.ntua.gr> Message-ID: <4F661E22.3020502@meetinghouse.net> Daniel Dormont wrote: > In the simple case, it's pretty straightforward: > > BSD - I can make fixes in your code and send them to you, and it's > cool because both of us could release our own proprietary version > based on those if we want > > GPL - I can make fixes in your code and send them to you, and it's > cool because neither of us can make our own proprietary version > > Except that, as mentioned above, some companies use the "dual-license" > approach so that they and they and they alone can produce a > proprietary version of their own code. I assume RMS must find this > terribly ironic, but it is only the GPL and not BSD that enables this. > But in this case, upstream contributions are a problem unless the > contributor is required to give permission to the original author to > dual-license the code. But if you were a potential contributor to such > a project, wouldn't you think twice about it? Not necessarily. If the original author is both doing most of the development, and being fairly responsible about releasing code as open source (i.e., the commercial version has serious value added capabilities, rather than the "community version" being a substantially crippled version of the commercial product), I certainly don't begrudge the developers the ability to generate revenue. Rather, it benefits me for them to be able to support their activities, and for contributions and fixes to make their way into the open source version. On other hand, contributing a fix to a crippled version of code, that enhances the commercial version - that I'd think twice about. Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From ok@REDACTED Sun Mar 18 22:39:01 2012 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 19 Mar 2012 10:39:01 +1300 Subject: [erlang-questions] [ANN] Brim - HTML templating library In-Reply-To: References: <6E4F3076-B078-4C47-B33E-561B43A1EEE2@gmail.com> <3C4BF661-FAA1-4CF4-A70A-8C917A2E45B8@cs.otago.ac.nz> Message-ID: <977C7697-BFBD-474C-A582-D2BE10F07B32@cs.otago.ac.nz> On 16/03/2012, at 6:41 PM, Michael Turner wrote: > Richard, I wouldn't consider autocompletion "programming by example", The book "Watch What I Do : Programming by Demonstration" does. http://acypher.com/wwid/ Ian Witten's Predictive Calculator (chapter 3) is basically an autocompleter. In the introduction to the successor volume, "Your Wish is my Command", Henry Lieberman wrote "But it also makes sense to view more conventional user-programming facilities, such as so-called "interface builders", macros and scripting systems as the "poor man's Programming by Example". http://web.media.mit.edu/~lieber/Your-Wish-Intro.html Ben Shneiderman's foreword makes it clear that a tool he built that "enabled users to record and view their actions, and then store and replay macros" was conceived as part of Programming by Example. From michael.eugene.turner@REDACTED Mon Mar 19 03:27:37 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Mon, 19 Mar 2012 11:27:37 +0900 Subject: [erlang-questions] [ANN] Brim - HTML templating library In-Reply-To: <977C7697-BFBD-474C-A582-D2BE10F07B32@cs.otago.ac.nz> References: <6E4F3076-B078-4C47-B33E-561B43A1EEE2@gmail.com> <3C4BF661-FAA1-4CF4-A70A-8C917A2E45B8@cs.otago.ac.nz> <977C7697-BFBD-474C-A582-D2BE10F07B32@cs.otago.ac.nz> Message-ID: On Mon, Mar 19, 2012 at 6:39 AM, Richard O'Keefe wrote: > > On 16/03/2012, at 6:41 PM, Michael Turner wrote: > >> Richard, I wouldn't consider autocompletion "programming by example", > > The book "Watch What I Do : Programming by Demonstration" does. > http://acypher.com/wwid/ > Ian Witten's Predictive Calculator (chapter 3) is basically an > autocompleter. There's overlap, perhaps but I've enjoyed the benefits of autocompletion without ever providing examples in the first place. And again, the output of most autocomplete isn't code, it's data. > In the introduction to the successor volume, "Your Wish is my > Command", Henry Lieberman wrote > "But it also makes sense to view more conventional user-programming facilities, > such as so-called "interface builders", macros and scripting systems as > the "poor man's Programming by Example". His scare quotes together with the qualification "poor man's" suggest to me that he knows these aren't really making the grade. There's no attempt at algorithm synthesis through implicit pattern matching with any of these. > http://web.media.mit.edu/~lieber/Your-Wish-Intro.html > Ben Shneiderman's foreword makes it clear that a tool he built that > "enabled users to record and view their actions, and then store and replay > macros" was conceived as part of Programming by Example. Would that he had actually provided a definition! Actually, he only exposes a terminological wilderness, biased toward direct-manipulation interfaces: "This important volume of papers carries forward the agenda of making direct manipulation programming (or programming-by-example, programming-by-demonstration, end-user-programming, programming-in-the-user-interface, etc.) a reality." The end result of programming by example is a program: a (potentially) reusable batch of instructions for doing much the same thing on different data. It doesn't require direct manipulation (the main concern of that book, back when direct-manipulation interfaces were hitting the mainstream); in fact, what I'm suggesting here is insensitive to where the examples come from, it just figures out a way to generalize from the examples to some kind of mapping. Maybe I should have called it something else. My point was, people spend a lot of time doing careful stitching to map between data and web pages, when much of the time, the correspondence is so obvious as to be significantly automatable. -michael turner From aleksandr.vin@REDACTED Mon Mar 19 09:19:05 2012 From: aleksandr.vin@REDACTED (Aleksandr Vinokurov) Date: Mon, 19 Mar 2012 12:19:05 +0400 Subject: [erlang-questions] Different behaviour of the erl shell and compile:file Message-ID: Hello all, I'm investigating Unicode support in Erlang. And have found the difference in behaviour of erl shell and compile:file. The erl shell: 1. We configure our terminal emulator to use UTF-8 2. We set io:setopts({encoding,unicode}). 3. Then in the prompt we enter a Unicode string and see the result -- a list of Unicode codepoints: (emacs@REDACTED)103> "?????". [1040,1041,1042,1043,1044] The compile:file: 1. We save a file foo in UTF-8 with a function that return a Unicode string bar() -> "?????". 2. Now we open an erl shell and run compile:file(foo). 3. Finally running foo:bar() we have a list of UTF-8 bytes. (emacs@REDACTED)111> foo:bar(). [208,144,208,145,208,146,208,147,208,148] I dug the compile:file/1 and found epp:server/4 where the file is opened with file:open(Name, [read]) and where encoding of the returned IoDevice is not configured. Why not to use one of the Unicode recipes from the Unicode usage doc (lib/stdlib-1.18/doc/html/unicode_usage.html) to set io:setopts(F,[{encoding,Type}]) or add an encoding option to the compile:file/2 and erlc? What would be your suggestions, honorable all? -- ????????? ????????? +7 (921) 982-21-43 @aleksandrvin -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.hughes@REDACTED Mon Mar 19 14:38:48 2012 From: john.hughes@REDACTED (John Hughes) Date: Mon, 19 Mar 2012 14:38:48 +0100 Subject: [erlang-questions] [OT] Re: GPL vs. whatever [was: Erlang UUID] In-Reply-To: <4F661E22.3020502@meetinghouse.net> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> <4F5D736A.8070901@meetinghouse.net> <4F64DEFA.6040104@cs.ntua.gr> <4F661E22.3020502@meetinghouse.net> Message-ID: Isn't the biggest difference likely to be that many companies won't contribute patches to GPL code, because they're not using it? At least, not as a component of their product. OK, some cowboys may be using it in violation of the licence... but then they're hardly going to advertise the fact by contributing patches! John Sent from my iPhone On 18 Mar 2012, at 18:40, Miles Fidelman wrote: > Daniel Dormont wrote: >> In the simple case, it's pretty straightforward: >> >> BSD - I can make fixes in your code and send them to you, and it's cool because both of us could release our own proprietary version based on those if we want >> >> GPL - I can make fixes in your code and send them to you, and it's cool because neither of us can make our own proprietary version >> >> Except that, as mentioned above, some companies use the "dual-license" approach so that they and they and they alone can produce a proprietary version of their own code. I assume RMS must find this terribly ironic, but it is only the GPL and not BSD that enables this. But in this case, upstream contributions are a problem unless the contributor is required to give permission to the original author to dual-license the code. But if you were a potential contributor to such a project, wouldn't you think twice about it? > > Not necessarily. If the original author is both doing most of the development, and being fairly responsible about releasing code as open source (i.e., the commercial version has serious value added capabilities, rather than the "community version" being a substantially crippled version of the commercial product), I certainly don't begrudge the developers the ability to generate revenue. Rather, it benefits me for them to be able to support their activities, and for contributions and fixes to make their way into the open source version. On other hand, contributing a fix to a crippled version of code, that enhances the commercial version - that I'd think twice about. > > Miles Fidelman > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From s.j.thompson@REDACTED Mon Mar 19 14:46:58 2012 From: s.j.thompson@REDACTED (Simon Thompson) Date: Mon, 19 Mar 2012 13:46:58 +0000 Subject: [erlang-questions] [OT] Re: GPL vs. whatever [was: Erlang UUID] In-Reply-To: <4F661E22.3020502@meetinghouse.net> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> <4F5D736A.8070901@meetinghouse.net> <4F64DEFA.6040104@cs.ntua.gr> <4F661E22.3020502@meetinghouse.net> Message-ID: <06A71466-5648-484B-B449-18DAF4E0B6DB@kent.ac.uk> Isn't the biggest difference likely to be that many companies won't contribute patches to GPL code, because they're not using it? At least, not as a component of their product. OK, some cowboys may be using it in violation of the licence... but then they're hardly going to advertise the fact by contributing patches! John Sent from my iPhone On 18 Mar 2012, at 18:40, Miles Fidelman wrote: > Daniel Dormont wrote: >> In the simple case, it's pretty straightforward: >> >> BSD - I can make fixes in your code and send them to you, and it's cool because both of us could release our own proprietary version based on those if we want >> >> GPL - I can make fixes in your code and send them to you, and it's cool because neither of us can make our own proprietary version >> >> Except that, as mentioned above, some companies use the "dual-license" approach so that they and they and they alone can produce a proprietary version of their own code. I assume RMS must find this terribly ironic, but it is only the GPL and not BSD that enables this. But in this case, upstream contributions are a problem unless the contributor is required to give permission to the original author to dual-license the code. But if you were a potential contributor to such a project, wouldn't you think twice about it? > > Not necessarily. If the original author is both doing most of the development, and being fairly responsible about releasing code as open source (i.e., the commercial version has serious value added capabilities, rather than the "community version" being a substantially crippled version of the commercial product), I certainly don't begrudge the developers the ability to generate revenue. Rather, it benefits me for them to be able to support their activities, and for contributions and fixes to make their way into the open source version. On other hand, contributing a fix to a crippled version of code, that enhances the commercial version - that I'd think twice about. > > Miles Fidelman > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Mon Mar 19 15:14:55 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 19 Mar 2012 15:14:55 +0100 Subject: [erlang-questions] Misultin 0.9 with "binary mode" enabled In-Reply-To: References: Message-ID: <562209BD-5E83-4E2E-AEAD-471AF0205C26@gmail.com> Hi Roberto, I'd like to you Misutlin 0.9 (from github) with "binary mode" enabled (http://erlang.org/pipermail/erlang-questions/2011-May/059048.html). How should I do? Any option for that? Regards, Zabrane On Jan 29, 2012, at 10:06 PM, Roberto Ostinelli wrote: > Dear all, > > Misultin 0.9 is out. Changelog is: > added SESSIONS state, persistent across requests > added access log callback function, so that main application can log HTTP access > added streaming input for big files or endless input, using a manual body_recv function in conjunction with the {auto_recv_body, false} option > added static directory support, so that GET requests to /static/* can automatically send files from a specified directory (thanks to egobrain suggestion) > added request redirection helper method > consistently improved memory usage by not copying by default to handler processes the full request or websocket record > added configuration option to set which websocket versions must be supported by the server > added support for websocket draft-hybi-10 > added support for websocket draft-hybi-17 (thanks to RJ) > added support for websockets on FireFox (thanks to Egobrain) > added support for 'If-Modified-Since' headers in file sending (thanks to davidgaleano) > added support for websockets when behind stunnel with {external_ssl, boolean()} option (thanks to RJ) > added support to see the correct client IP when behind stunnel, according to http://haproxy.1wt.eu/download/1.5/doc/proxy-protocol.txt (thanks to RJ) > added support for OPTIONS method (thanks to majek) > rebar-ized makefile > corrected minor bugs (thank you all - you know who you are!) > Link to repo: https://bit.ly/misultin09 > > Thank you, > > r. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Mon Mar 19 15:43:13 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 19 Mar 2012 15:43:13 +0100 Subject: [erlang-questions] Misultin 0.9 with "binary mode" enabled In-Reply-To: References: <562209BD-5E83-4E2E-AEAD-471AF0205C26@gmail.com> Message-ID: <2321D902-3A21-43C0-9BE4-479C67F158CB@gmail.com> Thanks, but I know that it was discontinued. This didn't not answer my question ;-) ? Regards, Zabrane On Mar 19, 2012, at 3:36 PM, Motiejus Jak?tys wrote: > On Mon, Mar 19, 2012 at 14:14, Zabrane Mickael wrote: >> Hi Roberto, >> >> I'd like to you Misutlin 0.9 (from github) with "binary mode" enabled >> (http://erlang.org/pipermail/erlang-questions/2011-May/059048.html). >> How should I do? Any option for that? > > Misultin has been discontinued. More info here: > http://erlang.org/pipermail/erlang-questions/2012-February/064389.html > > Motiejus From wmknapik@REDACTED Mon Mar 19 17:19:08 2012 From: wmknapik@REDACTED (Wojciech Knapik) Date: Mon, 19 Mar 2012 17:19:08 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang Message-ID: Hello everyone I have a question regarding mnesia that the books I have and the tutorials I found do not answer. I tried IRC, but was refered here. I tried the Manning forum, but was refered here. And here I was completely ignored. Not even a "go away", nothing. I had to check the list archives to make sure my emails reached their destination. I know this isn't some paid support list and you have no obligation to help anyone, but with a community of this size, you decide about the language's popularity by answering (or not answering) people's questions. With any non-trivial project, there are situations where people have to depend on outside help - where the documentation available simply doesn't allow them to answer a question and they need the opinion of someone experienced with the language. And this is not C++, or Java - you can't just ask any coworker for help. If you don't get an answer on this list, you're likely not getting it anywhere. I'm now part of a group of 9 developers undertaking a rather large project and Erlang is, in my opinion, the perfect fit for us. Most other members of the group are beginning to see this too. The only problem is that none of us know this language. Sure, we're reading the books (and they're excellent btw), but you know that books only go so far... I'd like nothing more than to stick to Erlang for this project, I've been a fan of functional languages for years, but we've only written a page of code and we've already run into a problem that noone seems to want, or be able to help us with. We all know a number of languages and make a living developing code in them, so the temptation to switch to something we're familiar with, where support will not be a problem, is pretty big. I'm guessing this is a very popular scenario when Erlang is being considered for a new project (once you get past the fact that everyone will have to learn a new language and, in most cases, a new programming paradigm). I could go on, but you get the idea. Perhaps these are the things to discuss in the next thread about Erlang's popularity, instead of ejabberd, Yaws and CouchDB... br, WK -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre@REDACTED Mon Mar 19 17:27:36 2012 From: andre@REDACTED (=?ISO-8859-1?Q?Andr=E9_Graf?=) Date: Mon, 19 Mar 2012 17:27:36 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: I guess you are referring to http://erlang.org/pipermail/erlang-questions/2012-March/065163.html ? On 19 March 2012 17:19, Wojciech Knapik wrote: > Hello everyone > > I have a question regarding mnesia that the books I have and the tutorials I > found do not answer. > > I tried IRC, but was refered here. I tried the Manning forum, but was > refered here. And here I was completely ignored. Not even a "go away", > nothing. I had to check the list archives to make sure my emails reached > their destination. > > I know this isn't some paid support list and you have no obligation to help > anyone, but with a community of this size, you decide about the language's > popularity by answering (or not answering) people's questions. > > With any non-trivial project, there are situations where people have to > depend on outside help - where the documentation available simply doesn't > allow them to answer a question and they need the opinion of someone > experienced with the language. And this is not C++, or Java - you can't just > ask any coworker for help. If you don't get an answer on this list, you're > likely not getting it anywhere. > > I'm now part of a group of 9 developers undertaking a rather large project > and Erlang is, in my opinion, the perfect fit for us. Most other members of > the group are beginning to see this too. The only problem is that none of us > know this language. Sure, we're reading the books (and they're excellent > btw), but you know that books only go so far... > > I'd like nothing more than to stick to Erlang for this project, I've been a > fan of functional languages for years, but we've only written a page of code > and we've already run into a problem that noone seems to want, or be able to > help us with. > > We all know a number of languages and make a living developing code in them, > so the temptation to switch to something we're familiar with, where support > will not be a problem, is pretty big. I'm guessing this is a very popular > scenario when Erlang is being considered for a new project (once you get > past the fact that everyone will have to learn a new language and, in most > cases, a new programming paradigm). > > I could go on, but you get the idea. Perhaps these are the things to discuss > in the next thread about Erlang's popularity, instead of ejabberd, Yaws and > CouchDB... > > br, > WK > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf@REDACTED Mon Mar 19 17:38:37 2012 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 19 Mar 2012 17:38:37 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: I, for one, don't recall what your question was. I can assure you that I didn't actively choose to ignore it. If you are part of a group doing commercial work in Erlang, you might want to contact Erlang Solutions. There, you can actually pay for the privilige of a guaranteed response, something you cannot do here. They also sell training and commercial support for Erlang - even have an office in Krakow. http://www.erlang-solutions.com/section/13/support As far as your question goes, a google search indicates you posted it once, 4 days ago. I'd say 4 days is a bit short before lecturing the community on their lack of responsiveness. It might be a suitable length of time before reposting the question, though. ;-) I much prefer to have a slightly more strict setup, where you e.g. designate one or two "master" machines that keep a persistent copy of the database. The other machines can start up with the env variable: extra_db_nodes : MasterNodes, and access the database without even having their own copy. Why does the data need to be fully replicated? Nodes starting with extra_db_nodes as above enjoy full distribution transparency. If very rapid read response is required, you can add a ram-based replica on the fly with the function mnesia:add_table_copy_type(Table, Node, CopyType). Update cost will increase noticeably with every replica you add, so in many cases, it may be counter-productive to use full replication. For fault tolerance, having two persistent copies of the data goes a long way. Nodes can also start with extra_db_nodes, receive a ram copy of the schema, and then change their schema to a persistent copy using mnesia:change_table_copy_type(schema, node(), disc_copies). BR, Ulf W On 19 Mar 2012, at 17:19, Wojciech Knapik wrote: > Hello everyone > > I have a question regarding mnesia that the books I have and the tutorials I found do not answer. > > I tried IRC, but was refered here. I tried the Manning forum, but was refered here. And here I was completely ignored. Not even a "go away", nothing. I had to check the list archives to make sure my emails reached their destination. > > I know this isn't some paid support list and you have no obligation to help anyone, but with a community of this size, you decide about the language's popularity by answering (or not answering) people's questions. > > With any non-trivial project, there are situations where people have to depend on outside help - where the documentation available simply doesn't allow them to answer a question and they need the opinion of someone experienced with the language. And this is not C++, or Java - you can't just ask any coworker for help. If you don't get an answer on this list, you're likely not getting it anywhere. > > I'm now part of a group of 9 developers undertaking a rather large project and Erlang is, in my opinion, the perfect fit for us. Most other members of the group are beginning to see this too. The only problem is that none of us know this language. Sure, we're reading the books (and they're excellent btw), but you know that books only go so far... > > I'd like nothing more than to stick to Erlang for this project, I've been a fan of functional languages for years, but we've only written a page of code and we've already run into a problem that noone seems to want, or be able to help us with. > > We all know a number of languages and make a living developing code in them, so the temptation to switch to something we're familiar with, where support will not be a problem, is pretty big. I'm guessing this is a very popular scenario when Erlang is being considered for a new project (once you get past the fact that everyone will have to learn a new language and, in most cases, a new programming paradigm). > > I could go on, but you get the idea. Perhaps these are the things to discuss in the next thread about Erlang's popularity, instead of ejabberd, Yaws and CouchDB... > > br, > WK > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From wmknapik@REDACTED Mon Mar 19 17:41:36 2012 From: wmknapik@REDACTED (Wojciech Knapik) Date: Mon, 19 Mar 2012 17:41:36 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: On Mon, Mar 19, 2012 at 5:27 PM, Andr? Graf wrote: > I guess you are referring to > http://erlang.org/pipermail/erlang-questions/2012-March/065163.html ? > Yes I am, though the code has changed since then. Through trial and error we got to that point: http://pastie.org/3627990 and it seems to work. But I don't like the look of that code. I'm guessing we're doing a lot if things wrong there. I'd hate to base our future work on something that's badly implemented... WK -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Mon Mar 19 17:53:28 2012 From: gumm@REDACTED (Jesse Gumm) Date: Mon, 19 Mar 2012 11:53:28 -0500 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: I think all of us have at some point or another posted something that had no responses. I know I have. I find, especially in the mailing list context, that for members for whom their best answer is "I don't know" you're less likely to get a response than if it were a more traditional forum. Further, it really helps to pare down the question to its simplest form, requiring the least amount of work from the members of the list. In my case, I tend to make too-long narratives that require too much reading on the part of the contributors who undoubtedly just mentally skip over it in "tl;dr" fashion. When that happens, it's my fault for not asking the right question. So my general recommendation is to make an effort to ask concise questions, and include code snippets directly in the post rather than posting links to pastebins, etc. (after all, link clicking is additional effort). In any case, I don't know the answer to the question you're referencing. -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 www.sigma-star.com @jessegumm On Mar 19, 2012 11:19 AM, "Wojciech Knapik" wrote: > Hello everyone > > I have a question regarding mnesia that the books I have and the tutorials > I found do not answer. > > I tried IRC, but was refered here. I tried the Manning forum, but was > refered here. And here I was completely ignored. Not even a "go away", > nothing. I had to check the list archives to make sure my emails reached > their destination. > > I know this isn't some paid support list and you have no obligation to > help anyone, but with a community of this size, you decide about the > language's popularity by answering (or not answering) people's questions. > > With any non-trivial project, there are situations where people have to > depend on outside help - where the documentation available simply doesn't > allow them to answer a question and they need the opinion of someone > experienced with the language. And this is not C++, or Java - you can't > just ask any coworker for help. If you don't get an answer on this list, > you're likely not getting it anywhere. > > I'm now part of a group of 9 developers undertaking a rather large project > and Erlang is, in my opinion, the perfect fit for us. Most other members of > the group are beginning to see this too. The only problem is that none of > us know this language. Sure, we're reading the books (and they're excellent > btw), but you know that books only go so far... > > I'd like nothing more than to stick to Erlang for this project, I've been > a fan of functional languages for years, but we've only written a page of > code and we've already run into a problem that noone seems to want, or be > able to help us with. > > We all know a number of languages and make a living developing code in > them, so the temptation to switch to something we're familiar with, where > support will not be a problem, is pretty big. I'm guessing this is a very > popular scenario when Erlang is being considered for a new project (once > you get past the fact that everyone will have to learn a new language and, > in most cases, a new programming paradigm). > > I could go on, but you get the idea. Perhaps these are the things to > discuss in the next thread about Erlang's popularity, instead of ejabberd, > Yaws and CouchDB... > > br, > WK > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wmknapik@REDACTED Mon Mar 19 17:59:51 2012 From: wmknapik@REDACTED (Wojciech Knapik) Date: Mon, 19 Mar 2012 17:59:51 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: On Mon, Mar 19, 2012 at 5:38 PM, Ulf Wiger wrote: > > I, for one, don't recall what your question was. I can assure you that I > didn't actively choose to ignore it. > > If you are part of a group doing commercial work in Erlang, you might want > to contact Erlang Solutions. There, you can actually pay for the privilige > of a guaranteed response, something you cannot do here. They also sell > training and commercial support for Erlang - even have an office in Krakow. > > http://www.erlang-solutions.com/section/13/support > Thanks for the hint. The project will be open source (BSD/MIT), but we're considering a separate license for commercial use. > As far as your question goes, a google search indicates you posted it > once, 4 days ago. I'd say 4 days is a bit short before lecturing the > community on their lack of responsiveness. It might be a suitable length of > time before reposting the question, though. ;-) > I wasn't expecting a reply after that time, considering the pretty healthy activity in other threads, but perhaps you're right ;] > I much prefer to have a slightly more strict setup, where you e.g. > designate one or two "master" machines that keep a persistent copy of the > database. The other machines can start up with the env variable: > extra_db_nodes : MasterNodes, and access the database without even having > their own copy. > > Why does the data need to be fully replicated? Nodes starting with > extra_db_nodes as above enjoy full distribution transparency. If very rapid > read response is required, you can add a ram-based replica on the fly with > the function mnesia:add_table_copy_type(Table, Node, CopyType). > > Update cost will increase noticeably with every replica you add, so in > many cases, it may be counter-productive to use full replication. For fault > tolerance, having two persistent copies of the data goes a long way. > > Nodes can also start with extra_db_nodes, receive a ram copy of the > schema, and then change their schema to a persistent copy using > mnesia:change_table_copy_type(schema, node(), disc_copies). > Ok, that's a lot to process for someone new to mnesia, so let me answer this fully, once I'm back home from work. For now a quick answer - we're aiming for a fully distributed setup, where every node is equal. We'd like the system to be able to function as long as at least a single node is up. We're just starting to work on the code, so we might change the approach to achieving this, but the goal will likely remain in place. There won't be much data stored in mnesia. For the larger sets of data, there will be a regular SQL database. Like I said - we're all new to Erlang, so our implementations might be far from optimal - that's why I was so looking forward to hearing your input. Thanks for the answers, I'll write more in a few hours. br, WK -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Mon Mar 19 18:07:16 2012 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 19 Mar 2012 18:07:16 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: <1339302F-97CB-45F4-B002-E2D0E635EDF0@feuerlabs.com> On 19 Mar 2012, at 17:59, Wojciech Knapik wrote: > For now a quick answer - we're aiming for a fully distributed setup, where every node is equal. We'd like the system to be able to function as long as at least a single node is up. We're just starting to work on the code, so we might change the approach to achieving this, but the goal will likely remain in place. There won't be much data stored in mnesia. For the larger sets of data, there will be a regular SQL database. There are definitely advantages to having each node being identical to every other node, but when it comes to a replicated database, there are some issues to consider. - Mnesia uses a simple broadcast algorithm for locking, and 2-phase commit protocols, so running transactions across a large number of replicas will become increasingly expensive. Ironically, I've often heard people become frustrated with mnesia's performance when running fully replicated, and instead switch to PostgreSQL, which until very recently, couldn't replicate at all. This suggest that replication wasn't a requirement in the first place (for them, I don't know enough about your requirements to know if it applies in this case). - You should not use dirty writes on replicated tables, _especially_ not if you have a large number of copies. Mnesia doesn't ensure consistency with dirty writes. You have to use transactions for that. I assume the SQL database is not running in fully replicated mode? If so, perhaps you have a layered architecture after all? BR, Ulf W -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Mon Mar 19 18:08:07 2012 From: gumm@REDACTED (Jesse Gumm) Date: Mon, 19 Mar 2012 12:08:07 -0500 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: Have you looked into riak for distributed, master-less data replication? -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 www.sigma-star.com @jessegumm On Mar 19, 2012 11:59 AM, "Wojciech Knapik" wrote: > > On Mon, Mar 19, 2012 at 5:38 PM, Ulf Wiger wrote: > >> >> I, for one, don't recall what your question was. I can assure you that I >> didn't actively choose to ignore it. >> >> If you are part of a group doing commercial work in Erlang, you might >> want to contact Erlang Solutions. There, you can actually pay for the >> privilige of a guaranteed response, something you cannot do here. They also >> sell training and commercial support for Erlang - even have an office in >> Krakow. >> >> http://www.erlang-solutions.com/section/13/support >> > > Thanks for the hint. The project will be open source (BSD/MIT), but we're > considering a separate license for commercial use. > > >> As far as your question goes, a google search indicates you posted it >> once, 4 days ago. I'd say 4 days is a bit short before lecturing the >> community on their lack of responsiveness. It might be a suitable length of >> time before reposting the question, though. ;-) >> > > I wasn't expecting a reply after that time, considering the pretty healthy > activity in other threads, but perhaps you're right ;] > > >> I much prefer to have a slightly more strict setup, where you e.g. >> designate one or two "master" machines that keep a persistent copy of the >> database. The other machines can start up with the env variable: >> extra_db_nodes : MasterNodes, and access the database without even having >> their own copy. >> >> Why does the data need to be fully replicated? Nodes starting with >> extra_db_nodes as above enjoy full distribution transparency. If very rapid >> read response is required, you can add a ram-based replica on the fly with >> the function mnesia:add_table_copy_type(Table, Node, CopyType). >> >> Update cost will increase noticeably with every replica you add, so in >> many cases, it may be counter-productive to use full replication. For fault >> tolerance, having two persistent copies of the data goes a long way. >> >> Nodes can also start with extra_db_nodes, receive a ram copy of the >> schema, and then change their schema to a persistent copy using >> mnesia:change_table_copy_type(schema, node(), disc_copies). >> > > Ok, that's a lot to process for someone new to mnesia, so let me answer > this fully, once I'm back home from work. > > For now a quick answer - we're aiming for a fully distributed setup, where > every node is equal. We'd like the system to be able to function as long as > at least a single node is up. We're just starting to work on the code, so > we might change the approach to achieving this, but the goal will likely > remain in place. There won't be much data stored in mnesia. For the larger > sets of data, there will be a regular SQL database. > Like I said - we're all new to Erlang, so our implementations might be far > from optimal - that's why I was so looking forward to hearing your input. > > Thanks for the answers, I'll write more in a few hours. > > br, > WK > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Mon Mar 19 18:11:18 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 19 Mar 2012 13:11:18 -0400 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: On Mon, Mar 19, 2012 at 12:19 PM, Wojciech Knapik wrote: > Hello everyone > > I have a question regarding mnesia that the books I have and the tutorials > I found do not answer. > > I tried IRC, but was refered here. I tried the Manning forum, but was > refered here. And here I was completely ignored. Not even a "go away", > nothing. I had to check the list archives to make sure my emails reached > their destination. > I was one of the people who tried to help you on IRC; I'm generally busy with my own things and not *that* knowledgeable about Mnesia, sorry I couldn't do more. It is better, in my opinion, to receive no reply than a 'go away'; the former means your question is either complex enough that people don't know the answer or that writing a useful answer would be too long for them, for whatever reason they have to be busy. A 'go away' is a direct sign of being closed, with somebody telling you they want you out -- which didn't happen. The referral from (IRC | Some site) to the Mailing list is usually something done because people wherever you are do not know the answer or do not have the resources to help you. The mailing list is where a lot of veteran Erlang programmers take their time to reply (whenever they have time), including some people from the OTP team itself, working at implementing the language. It's an escalation of tiers for help, but nothing comes really fast and yet again, answers may not come at all. > > I know this isn't some paid support list and you have no obligation to > help anyone, but with a community of this size, you decide about the > language's popularity by answering (or not answering) people's questions. > Many of us do our fair share. People here work on some of the 'killer apps' that more to Erlang's name than it had at Ericsson (although Ericsson people are on here too!). You've got the contributors of all major Erlang servers, many databases written in Erlang, message queues, build systems, regulars from many corporations (Klarna, etc.). I myself decided to write Learn You Some Erlang, and you've got other people on here who do professional training, contribute to a lot of open source projects, work on the core Erlang repositories themselves, etc. The community has a noticeable size, but it also has a *lot* of work to do and being done constantly. The same community that does a lot of this work also tries to write their own doc and come in here and do support (and ask for it when they need it). I understand this isn't exactly relevant to your problems, but I would say the community is not yet large enough to have too many resources and plenty of leeway for mailing list support. If you manage to solve your problem, I do invite you to stay on the list and help people who need help in the future. > > With any non-trivial project, there are situations where people have to > depend on outside help - where the documentation available simply doesn't > allow them to answer a question and they need the opinion of someone > experienced with the language. And this is not C++, or Java - you can't > just ask any coworker for help. If you don't get an answer on this list, > you're likely not getting it anywhere. > Yes, that is certainly a problem. I do not have a solution that doesn't include 'brute force your way through the source' (I hate this), 'wait for someone who can help' or 'hire help'. I don't think we can mandate people to give more support on this list than what they can do. Especially since once you break the point where people do this by passion, it becomes a job to them. > > I'm now part of a group of 9 developers undertaking a rather large project > and Erlang is, in my opinion, the perfect fit for us. Most other members of > the group are beginning to see this too. The only problem is that none of > us know this language. Sure, we're reading the books (and they're excellent > btw), but you know that books only go so far... > > I'd like nothing more than to stick to Erlang for this project, I've been > a fan of functional languages for years, but we've only written a page of > code and we've already run into a problem that noone seems to want, or be > able to help us with. > > We all know a number of languages and make a living developing code in > them, so the temptation to switch to something we're familiar with, where > support will not be a problem, is pretty big. I'm guessing this is a very > popular scenario when Erlang is being considered for a new project (once > you get past the fact that everyone will have to learn a new language and, > in most cases, a new programming paradigm). > Switching to a language you know might be a valid approach if you can't get what you need (and the help currently given in this thread isn't enough). Getting your products out is the most important thing, I figure. I just hope this wouldn't turn into a blog post about 'how the Erlang community is unfit for newcomers' or something like that :) > > I could go on, but you get the idea. Perhaps these are the things to > discuss in the next thread about Erlang's popularity, instead of ejabberd, > Yaws and CouchDB... > Eh, people talk about what they feel like talking about, and things they use more. You brought a topic to the table and it was discussed. Hopefully it's not too bad. Disclaimer: this e-mail represents my views and perceptions of the community, the mailing lists, etc. and I in no way speak for the rest of the people in here. Sorry if it may look like I'm speaking as if I knew what they think, this isn't my intention. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yoursurrogategod@REDACTED Mon Mar 19 18:17:08 2012 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Mon, 19 Mar 2012 13:17:08 -0400 Subject: [erlang-questions] math:pow(X, Y). Message-ID: Hi all, That's the standard method that I use to raise a value to a power. I compared this to the way Python does raising to a power and this is the result that I get: Erlang: 11> math:pow(42, 909). ** exception error: bad argument in an arithmetic expression in function math:pow/2 called as math:pow(42,909) Python: http://bin.cakephp.org/view/1006418268 Now. Why is it in Erlang's case I get a float as a return value? Is there a method that only returns a long? Or do most erlang coders have a custom power function if they want ridiculously large numbers? -------------- next part -------------- An HTML attachment was scrubbed... URL: From wmknapik@REDACTED Mon Mar 19 18:17:32 2012 From: wmknapik@REDACTED (Wojciech Knapik) Date: Mon, 19 Mar 2012 18:17:32 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: On Mon, Mar 19, 2012 at 5:53 PM, Jesse Gumm wrote: > I think all of us have at some point or another posted something that had > no responses. I know I have. > > I find, especially in the mailing list context, that for members for whom > their best answer is "I don't know" you're less likely to get a response > than if it were a more traditional forum. > > Further, it really helps to pare down the question to its simplest form, > requiring the least amount of work from the members of the list. In my > case, I tend to make too-long narratives that require too much reading on > the part of the contributors who undoubtedly just mentally skip over it in > "tl;dr" fashion. > > When that happens, it's my fault for not asking the right question. > > So my general recommendation is to make an effort to ask concise > questions, and include code snippets directly in the post rather than > posting links to pastebins, etc. (after all, link clicking is additional > effort). > Thanks, I'll keep that in mind. Concise questions and inline code from now on ;] WK -------------- next part -------------- An HTML attachment was scrubbed... URL: From jared.nance@REDACTED Mon Mar 19 18:26:02 2012 From: jared.nance@REDACTED (Jared Kofron) Date: Mon, 19 Mar 2012 10:26:02 -0700 Subject: [erlang-questions] math:pow(X, Y). In-Reply-To: References: Message-ID: Just a guess, but math:pow/2 is implemented as a NIF and C may not enjoy the size of the number you are trying to calculate. Here's GHCI's answer: Prelude> 42**909 Infinity On Mar 19, 2012, at 10:17 AM, Yves S. Garret wrote: > Hi all, > > That's the standard method that I use to raise a value to a power. I compared this to the way Python does raising to a power and this is the result that I get: > > Erlang: > 11> math:pow(42, 909). > ** exception error: bad argument in an arithmetic expression > in function math:pow/2 > called as math:pow(42,909) > > Python: > http://bin.cakephp.org/view/1006418268 > > Now. Why is it in Erlang's case I get a float as a return value? Is there a method that only returns a long? Or do most erlang coders have a custom power function if they want ridiculously large numbers? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Mon Mar 19 18:29:21 2012 From: gumm@REDACTED (Jesse Gumm) Date: Mon, 19 Mar 2012 12:29:21 -0500 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: The obvious exception to inline code of course being long modules. The benefit to short inline snippets is that it might gain attention of someone more quickly, and from there you have him/her hooked. Then you can start posting pasties and gists accompanied by long descriptions. You have to catch the fish before you can filet it, so make the bait easy to bite. -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 www.sigma-star.com @jessegumm On Mar 19, 2012 12:17 PM, "Wojciech Knapik" wrote: > > > On Mon, Mar 19, 2012 at 5:53 PM, Jesse Gumm wrote: > >> I think all of us have at some point or another posted something that had >> no responses. I know I have. >> >> I find, especially in the mailing list context, that for members for whom >> their best answer is "I don't know" you're less likely to get a response >> than if it were a more traditional forum. >> >> Further, it really helps to pare down the question to its simplest form, >> requiring the least amount of work from the members of the list. In my >> case, I tend to make too-long narratives that require too much reading on >> the part of the contributors who undoubtedly just mentally skip over it in >> "tl;dr" fashion. >> >> When that happens, it's my fault for not asking the right question. >> >> So my general recommendation is to make an effort to ask concise >> questions, and include code snippets directly in the post rather than >> posting links to pastebins, etc. (after all, link clicking is additional >> effort). >> > Thanks, I'll keep that in mind. Concise questions and inline code from now > on ;] > > WK > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Mon Mar 19 18:29:49 2012 From: bob@REDACTED (Bob Ippolito) Date: Mon, 19 Mar 2012 10:29:49 -0700 Subject: [erlang-questions] math:pow(X, Y). In-Reply-To: References: Message-ID: In general the math library is for float arithmetic, just like in C. My guess is that the time when this was written, there was no bignum support. There's an implementation of an int_pow/2 function in mochinum (currently part of mochiweb): https://github.com/mochi/mochiweb/blob/master/src/mochinum.erl#L50 Not the fastest algorithm on the planet, but much better than the naive O(N) implementation. There might be better libraries out there with faster algorithms or maybe a GMP library, but my needs were pretty minimal so I wrote the few functions I needed. 1> mochinum:int_pow(42, 909). 34166852412481[?] -bob On Mon, Mar 19, 2012 at 10:17 AM, Yves S. Garret wrote: > Hi all, > > That's the standard method that I use to raise a value to a power. I > compared this to the way Python does raising to a power and this is the > result that I get: > > Erlang: > 11> math:pow(42, 909). > ** exception error: bad argument in an arithmetic expression > in function math:pow/2 > called as math:pow(42,909) > > Python: > http://bin.cakephp.org/view/1006418268 > > Now. Why is it in Erlang's case I get a float as a return value? Is > there a method that only returns a long? Or do most erlang coders have a > custom power function if they want ridiculously large numbers? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wmknapik@REDACTED Mon Mar 19 18:31:55 2012 From: wmknapik@REDACTED (Wojciech Knapik) Date: Mon, 19 Mar 2012 18:31:55 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: On Mon, Mar 19, 2012 at 6:11 PM, Fred Hebert wrote: > > > On Mon, Mar 19, 2012 at 12:19 PM, Wojciech Knapik wrote: > >> Hello everyone >> >> I have a question regarding mnesia that the books I have and the >> tutorials I found do not answer. >> >> I tried IRC, but was refered here. I tried the Manning forum, but was >> refered here. And here I was completely ignored. Not even a "go away", >> nothing. I had to check the list archives to make sure my emails reached >> their destination. >> > > I was one of the people who tried to help you on IRC; I'm generally busy > with my own things and not *that* knowledgeable about Mnesia, sorry I > couldn't do more. > I'm sensing this will be a common theme with mnesia. I guess not a lot of people feel their knowledge of mnesia is very strong. It's a little surprising, since it's such an impressive feature of Erlang. > The referral from (IRC | Some site) to the Mailing list is usually > something done because people wherever you are do not know the answer or do > not have the resources to help you. The mailing list is where a lot of > veteran Erlang programmers take their time to reply (whenever they have > time), including some people from the OTP team itself, working at > implementing the language. It's an escalation of tiers for help, but > nothing comes really fast and yet again, answers may not come at all. > The presence of so many people involved in the creation of the language, OTP and the books about both on this list is a great thing. It only furthers people's expectations though ;] > > >> >> I know this isn't some paid support list and you have no obligation to >> help anyone, but with a community of this size, you decide about the >> language's popularity by answering (or not answering) people's questions. >> > > Many of us do our fair share. People here work on some of the 'killer > apps' that more to Erlang's name than it had at Ericsson (although Ericsson > people are on here too!). You've got the contributors of all major Erlang > servers, many databases written in Erlang, message queues, build systems, > regulars from many corporations (Klarna, etc.). I myself decided to write > Learn You Some Erlang, and you've got other people on here who do > professional training, contribute to a lot of open source projects, work on > the core Erlang repositories themselves, etc. > > The community has a noticeable size, but it also has a *lot* of work to > do and being done constantly. The same community that does a lot of this > work also tries to write their own doc and come in here and do support (and > ask for it when they need it). > > I understand this isn't exactly relevant to your problems, but I would say > the community is not yet large enough to have too many resources and plenty > of leeway for mailing list support. > Of course. I get it. I must admit, I needed to draw your attention a little bit, hence the slightly provocative post ;] > > If you manage to solve your problem, I do invite you to stay on the list > and help people who need help in the future. > > Oh, I definitely will! > > >> >> I'm now part of a group of 9 developers undertaking a rather large >> project and Erlang is, in my opinion, the perfect fit for us. Most other >> members of the group are beginning to see this too. The only problem is >> that none of us know this language. Sure, we're reading the books (and >> they're excellent btw), but you know that books only go so far... >> >> I'd like nothing more than to stick to Erlang for this project, I've been >> a fan of functional languages for years, but we've only written a page of >> code and we've already run into a problem that noone seems to want, or be >> able to help us with. >> >> We all know a number of languages and make a living developing code in >> them, so the temptation to switch to something we're familiar with, where >> support will not be a problem, is pretty big. I'm guessing this is a very >> popular scenario when Erlang is being considered for a new project (once >> you get past the fact that everyone will have to learn a new language and, >> in most cases, a new programming paradigm). >> > > Switching to a language you know might be a valid approach if you can't > get what you need (and the help currently given in this thread isn't > enough). Getting your products out is the most important thing, I figure. > Well, one of MY major goals with this project is to have fun learning Erlang, I'm not really interested in writing this in C++, so I need to make sure we can work without worrying about being left completely alone with our problems a few months down the road. > > I just hope this wouldn't turn into a blog post about 'how the Erlang > community is unfit for newcomers' or something like that :) > Hehe, no, certainly not. WK -------------- next part -------------- An HTML attachment was scrubbed... URL: From wmknapik@REDACTED Mon Mar 19 18:42:38 2012 From: wmknapik@REDACTED (Wojciech Knapik) Date: Mon, 19 Mar 2012 18:42:38 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: On Mon, Mar 19, 2012 at 6:08 PM, Jesse Gumm wrote: > Have you looked into riak for distributed, master-less data replication? > >From what I've read on github, I can't tell what the benefits would be and I was hoping to familiarize myself with mnesia. Would riak be a better fit ? WK -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Mon Mar 19 18:45:09 2012 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 19 Mar 2012 18:45:09 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: <61A30020-4D60-4825-92CC-917A0DDED075@feuerlabs.com> Not sure if you've seen the (admittedly short) tutorials on TrapExit: http://www.trapexit.org/Distributing_a_Mnesia_schema A minor comment: you can do this instead: mnesia:start([{extra_db_nodes, Nodes}]) If you want the nodes to start from a boot script, you can either roll your own the old-fashioned way: http://www.trapexit.org/OTP_Release_Handling_Tutorial or use rebar and Reltool: http://www.metabrew.com/article/erlang-rebar-tutorial-generating-releases-upgrades In this case, it's better to put the extra_db_nodes setting in the sys.config file: [?, {mnesia, [ {extra_db_nodes, [node@REDACTED, node@REDACTED]} ]} Then, calling mnesia:start() will suffice, given that you've started the node in the right way. BR, Ulf W On 19 Mar 2012, at 17:59, Wojciech Knapik wrote: > I much prefer to have a slightly more strict setup, where you e.g. designate one or two "master" machines that keep a persistent copy of the database. The other machines can start up with the env variable: extra_db_nodes : MasterNodes, and access the database without even having their own copy. > > Why does the data need to be fully replicated? Nodes starting with extra_db_nodes as above enjoy full distribution transparency. If very rapid read response is required, you can add a ram-based replica on the fly with the function mnesia:add_table_copy_type(Table, Node, CopyType). > > Update cost will increase noticeably with every replica you add, so in many cases, it may be counter-productive to use full replication. For fault tolerance, having two persistent copies of the data goes a long way. > > Nodes can also start with extra_db_nodes, receive a ram copy of the schema, and then change their schema to a persistent copy using mnesia:change_table_copy_type(schema, node(), disc_copies). > > Ok, that's a lot to process for someone new to mnesia, so let me answer this fully, once I'm back home from work. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Mon Mar 19 18:54:35 2012 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 19 Mar 2012 18:54:35 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: <9FDE3C40-2713-471F-A7DE-9E9C775034E0@feuerlabs.com> On 19 Mar 2012, at 18:42, Wojciech Knapik wrote: > > > On Mon, Mar 19, 2012 at 6:08 PM, Jesse Gumm wrote: > Have you looked into riak for distributed, master-less data replication? > > > From what I've read on github, I can't tell what the benefits would be and I was hoping to familiarize myself with mnesia. > > Would riak be a better fit ? It depends much on your access patterns. Mnesia offers transaction consistency, meaning that you can run fairly complex update transactions and let mnesia worry about consistency across your data set. This is a wonderful feature, but it tends to become troublesome in very large clusters, since the increasing cost of ensuring consistency puts a limit on scalability, and netsplits become increasingly likely as your clusters go. Netsplits and transaction consistency make poor bedfellows. Riak is a key-value database with no transaction concept whatsoever (although individual writes are of course atomic). Thus, if you have concurrent read-write patterns against the same data, you will find the results ?arm, interesting for any but the simplest of update patterns. (See e.g. http://www.infoq.com/presentations/Eventually-Consistent-HTTP-with-Statebox-and-Riak for an example of roughly how far you can push concurrent updates in riak. It should also give you an idea of how riak works.) OTOH, if all you do is simple get/put, and you need high throughput and excellent fault tolerance, riak will let you scale to levels way beyond what any sane person would attempt with mnesia. BR, Ulf W -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias@REDACTED Mon Mar 19 19:23:33 2012 From: matthias@REDACTED (Matthias Lang) Date: Mon, 19 Mar 2012 19:23:33 +0100 Subject: [erlang-questions] math:pow(X, Y). In-Reply-To: References: Message-ID: <20120319182333.GA13317@corelatus.se> On Monday, March 19, Jared Kofron wrote: > Just a guess, but math:pow/2 is implemented as a NIF and C may not > enjoy the size of the number you are trying to calculate. Exactly. The bottom of the 'math' manpage sort of says so, under 'bugs': "As these are the C library, the bugs are the same." Matt From torben.lehoff@REDACTED Mon Mar 19 20:46:12 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Mon, 19 Mar 2012 20:46:12 +0100 Subject: [erlang-questions] [OT] Re: GPL vs. whatever [was: Erlang UUID] In-Reply-To: <06A71466-5648-484B-B449-18DAF4E0B6DB@kent.ac.uk> References: <4F4A9BDE.1080408@gmail.com> <4F4F1827.70800@gmail.com> <4F4FA9A4.9050909@gmail.com> <4F5C251C.7090803@mansionfamily.plus.com> <4F5C5A92.6020505@gmail.com> <4F5D18A6.3030106@mansionfamily.plus.com> <4F5D34D9.40009@gmail.com> <4F5D6E65.90800@gmail.com> <4F5D736A.8070901@meetinghouse.net> <4F64DEFA.6040104@cs.ntua.gr> <4F661E22.3020502@meetinghouse.net> <06A71466-5648-484B-B449-18DAF4E0B6DB@kent.ac.uk> Message-ID: <4F678D04.8000702@gmail.com> I think that many big companies avoid GPLv3-code like the plague. I worked for Motorola and it was virtually impossible to get approval to use a GPLv3 licensed piece of software - this was due to an incident where Motorola lost some patents due to incorrect usage of a GPLv3 library. This viral aspect of GPLv3 is keeping big companies from using products with that license. You may argue that "why don't they just get a non-GPL license agreement with the author", but unless you are dealing with a single contributor it becomes very difficult to do and even in the case of a single contributor it is not easy to get set up correctly. That is too much risk and stops the usage of GPLv3 software in many companies. Just a different angle on the discussion... Torben On Mon Mar 19 14:46:58 2012, Simon Thompson wrote: > Isn't the biggest difference likely to be that many companies won't contribute patches to GPL code, because they're not using it? At least, not as a component of their product. OK, some cowboys may be using it in violation of the licence... but then they're hardly going to advertise the fact by contributing patches! > > John > > Sent from my iPhone > > On 18 Mar 2012, at 18:40, Miles Fidelman wrote: > >> Daniel Dormont wrote: >>> In the simple case, it's pretty straightforward: >>> >>> BSD - I can make fixes in your code and send them to you, and it's cool because both of us could release our own proprietary version based on those if we want >>> >>> GPL - I can make fixes in your code and send them to you, and it's cool because neither of us can make our own proprietary version >>> >>> Except that, as mentioned above, some companies use the "dual-license" approach so that they and they and they alone can produce a proprietary version of their own code. I assume RMS must find this terribly ironic, but it is only the GPL and not BSD that enables this. But in this case, upstream contributions are a problem unless the contributor is required to give permission to the original author to dual-license the code. But if you were a potential contributor to such a project, wouldn't you think twice about it? >> >> Not necessarily. If the original author is both doing most of the development, and being fairly responsible about releasing code as open source (i.e., the commercial version has serious value added capabilities, rather than the "community version" being a substantially crippled version of the commercial product), I certainly don't begrudge the developers the ability to generate revenue. Rather, it benefits me for them to be able to support their activities, and for contributions and fixes to make their way into the open source version. On other hand, contributing a fix to a crippled version of code, that enhances the commercial version - that I'd think twice about. >> >> Miles Fidelman >> >> >> -- >> In theory, there is no difference between theory and practice. >> In practice, there is. .... Yogi Berra >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- -- http://www.linkedin.com/in/torbenhoffmann From clist@REDACTED Mon Mar 19 21:08:15 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Mon, 19 Mar 2012 21:08:15 +0100 Subject: [erlang-questions] strange rebar error Message-ID: <201203192108.15432.clist@uah.es> Hi Im using rebar for several projects and i used to have it installed as a RPM from a openSUSE repo and seems to run fine. DEBUG: Rebar location: "/usr/bin/rebar" DEBUG: Entering /home/sinosuke/Documentos/Trabajo/2012/Investigaci?n/ResMol/Erlang_NoseHoover_16_12_2012/nosehoover DEBUG: Available deps: [] DEBUG: Missing deps : [] DEBUG: Predirs: [] ==> nosehoover (clean) DEBUG: Postdirs: [] DEBUG: Entering /home/sinosuke/Documentos/Trabajo/2012/Investigaci?n/ResMol/Erlang_NoseHoover_16_12_2012/nosehoover DEBUG: Available deps: [] DEBUG: Missing deps : [] DEBUG: Predirs: [] ==> nosehoover (compile) DEBUG: Matched required ERTS version: 5.9 -> .* DEBUG: Matched required OTP release: R15B -> .* DEBUG: erl_opts [native,inline,{i,"include"}] DEBUG: Starting 3 compile worker(s) src/nosehoover.erl:48: Warning: variable 'OtrosArgs' is unused Compiled src/nosehoover.erl Compiled src/nosehoover_loader_gaussian.erl Compiled src/getopt.erl DEBUG: Worker exited cleanly src/nosehoover_core.erl:21: Warning: variable 'Other' is unused Compiled src/nosehoover_core.erl DEBUG: Worker exited cleanly Compiled src/nosehoover_tools.erl DEBUG: Worker exited cleanly DEBUG: Postdirs: [] DEBUG: Entering /home/sinosuke/Documentos/Trabajo/2012/Investigaci?n/ResMol/Erlang_NoseHoover_16_12_2012/nosehoover DEBUG: Available deps: [] DEBUG: Missing deps : [] DEBUG: Predirs: [] ==> nosehoover (escriptize) DEBUG: Postdirs: [] sinosuke@REDACTED:~/Documentos/Trabajo/2012/Investigaci?n/ResMol/Erlang_NoseHoover_16_12_2012/nosehoover> ./nosehoover -D ./data/METANO metano Reading geom: # Metano, formato etiquetado "atomo" "X" "Y" "Z" Reading geom: # el formato est? defomardo para ejercitar el parser {system,[{atom,"C1",{cartesian,0.0,0.0,1.089}}, {atom,"H1",{cartesian,1.02672,0.0,-0.362996}}, {atom,"H2",{cartesian,-0.51336,-0.889165,-0.363}}, {atom,"H3",{cartesian,-0.51336,0.889165,-0.363}}], {cartesian,[0.0,0.0,0.0,0.0,0.0,-0.970109808,-0.685963027,0.0, 1.94021962,2.05791176,-0.685970586,1.68027834, -0.970109808,-0.685970586,-1.68027834]}, {gradient,[-1.28369537e-16,3.49033183e-7,-5.12583544e-7, 2.74086309e-16,5.68362124e-7,-0.00249459546,-0.00176301405, -5.2909066e-17,0.00498827352,0.00529098265,-0.00176372801, 0.00432037169,-0.00249459546,-0.00176372801, -0.00432037169]}, {hessian,[-3.6869881e-16,2.60106291e-7,0.824862682,1.3382681e-16, 0.824865666,0.0696220165,-1.87359717e-8,-5.90803338e-17, -0.0798212583,0.824862747,0.0696218852,-3.25950592e-19, -9.21169816e-17,-0.0798212533,-6.03652537e-17,..... .. more chem related data. Today ive downloaded lastest rebar from git and tested the same way... sinosuke@REDACTED:~/Documentos/Trabajo/2012/Investigaci?n/ResMol/Erlang_NoseHoover_16_12_2012/nosehoover> ./rebar -v clean compile escriptize ==> nosehoover (clean) ==> nosehoover (compile) src/nosehoover.erl:48: Warning: variable 'OtrosArgs' is unused Compiled src/nosehoover.erl Compiled src/nosehoover_loader_gaussian.erl src/nosehoover_core.erl:21: Warning: variable 'Other' is unused Compiled src/nosehoover_core.erl Compiled src/getopt.erl Compiled src/nosehoover_tools.erl ==> nosehoover (escriptize) sinosuke@REDACTED:~/Documentos/Trabajo/2012/Investigaci?n/ResMol/Erlang_NoseHoover_16_12_2012/nosehoover> ./nosehoover -D ./data/METANO metano ./nosehoover:5: undefined macro '?' ./nosehoover:18: illegal atom ./nosehoover:25: illegal atom ./nosehoover:25: syntax error before: 3 ./nosehoover:28: illegal atom ./nosehoover:28: illegal atom ./nosehoover:30: illegal atom ./nosehoover:34: syntax error before: M ./nosehoover:34: undefined macro '??P???XK???n???QF' ./nosehoover:48: illegal atom ./nosehoover:50: illegal atom ./nosehoover:56: illegal atom ./nosehoover:66: illegal atom ./nosehoover:75: undefined macro 'v' ./nosehoover:88: illegal macro call '?'?'' ./nosehoover:101: undefined macro '?x' ./nosehoover:104: illegal atom ./nosehoover:109: syntax error before: A? ./nosehoover:115: illegal atom ./nosehoover:119: illegal atom ./nosehoover:143: illegal atom ./nosehoover:146: illegal atom ./nosehoover:149: syntax error before: '?' ./nosehoover:154: undefined macro 'J?u' ./nosehoover:160: illegal integer ./nosehoover:161: undefined macro 'C' ./nosehoover:177: illegal atom ./nosehoover:181: undefined macro '?' ./nosehoover:211: illegal atom ./nosehoover:212: syntax error before: '#' ./nosehoover:217: illegal atom ./nosehoover:218: syntax error before: _ ./nosehoover:222: undefined macro '?' ./nosehoover:246: illegal atom ./nosehoover:250: illegal macro call '?'?'' ./nosehoover:264: illegal macro call '?'+'' ./nosehoover:267: illegal atom ./nosehoover:271: syntax error before: '?' ./nosehoover:272: syntax error before: J?B ./nosehoover:277: undefined macro '?' ./nosehoover:281: illegal atom ./nosehoover:291: illegal atom ./nosehoover:291: syntax error before: '?' ./nosehoover:292: illegal atom ./nosehoover:298: illegal atom ./nosehoover:305: illegal atom ./nosehoover:328: illegal macro call '?3' ./nosehoover:328: undefined macro '?' escript: There were compilation errors. the generated script crashes badly and i dont know why. We are feared of lossing our good rebar on a RPM update and face this problem with newer rebar versions... ?Can anyone give a clue on what's wrong? Thanks! From clist@REDACTED Mon Mar 19 22:53:41 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Mon, 19 Mar 2012 22:53:41 +0100 Subject: [erlang-questions] [madrid-erlang-users] strange rebar error **SOLVED** In-Reply-To: <201203192206.54568.clist@uah.es> References: <201203192117.08878.clist@uah.es> <2999163138293542984@unknownmsgid> <201203192206.54568.clist@uah.es> Message-ID: <201203192253.41876.clist@uah.es> rebar.config was causing trouble.... deleteing this file solves the problem but you have to write a new one... Some options have changed hopefully.... By the way, the offending terms were: {escript_emu_args,["+sbtu","+K true","+A 2" ]}. What are the current way of passing those args to the VM? Thanks all for the answers!! regards, Angel On Lunes, 19 de Marzo de 2012 22:06:54 Angel J. Alvarez Miguel escribi?: > it is nor elrang just usiong old rebar with same erlang goes fine.. > > On Lunes, 19 de Marzo de 2012 21:42:49 Manuel Dur?n Aguete escribi?: > > Hello, > > What's your erlang version? > > > > Seems it doesn't support scripts with zip archives attached. > > > > Regards > > > > Enviado desde mi iPhone > > > > El 19/03/2012, a las 21:17, "Angel J. Alvarez Miguel" > > escribi?: > > > Hi > > > > > > Im using rebar for several projects and i used to have it installed as > > > a RPM from a openSUSE repo and seems to run fine. > > > > > > DEBUG: Rebar location: "/usr/bin/rebar" > > > DEBUG: Entering > > > /home/sinosuke/Documentos/Trabajo/2012/Investigaci?n/ResMol/Erlang_Nose > > > Ho over_16_12_2012/nosehoover DEBUG: Available deps: [] > > > DEBUG: Missing deps : [] > > > DEBUG: Predirs: [] > > > ==> nosehoover (clean) > > > DEBUG: Postdirs: [] > > > DEBUG: Entering > > > /home/sinosuke/Documentos/Trabajo/2012/Investigaci?n/ResMol/Erlang_Nose > > > Ho over_16_12_2012/nosehoover DEBUG: Available deps: [] > > > DEBUG: Missing deps : [] > > > DEBUG: Predirs: [] > > > ==> nosehoover (compile) > > > DEBUG: Matched required ERTS version: 5.9 -> .* > > > DEBUG: Matched required OTP release: R15B -> .* > > > DEBUG: erl_opts [native,inline,{i,"include"}] > > > DEBUG: Starting 3 compile worker(s) > > > src/nosehoover.erl:48: Warning: variable 'OtrosArgs' is unused > > > Compiled src/nosehoover.erl > > > Compiled src/nosehoover_loader_gaussian.erl > > > Compiled src/getopt.erl > > > DEBUG: Worker exited cleanly > > > src/nosehoover_core.erl:21: Warning: variable 'Other' is unused > > > Compiled src/nosehoover_core.erl > > > DEBUG: Worker exited cleanly > > > Compiled src/nosehoover_tools.erl > > > DEBUG: Worker exited cleanly > > > DEBUG: Postdirs: [] > > > DEBUG: Entering > > > /home/sinosuke/Documentos/Trabajo/2012/Investigaci?n/ResMol/Erlang_Nose > > > Ho over_16_12_2012/nosehoover DEBUG: Available deps: [] > > > DEBUG: Missing deps : [] > > > DEBUG: Predirs: [] > > > ==> nosehoover (escriptize) > > > DEBUG: Postdirs: [] > > > > > > > > > sinosuke@REDACTED:~/Documentos/Trabajo/2012/Investigaci?n/ResMol/Erlan > > > g_ NoseHoover_16_12_2012/nosehoover> ./nosehoover -D ./data/METANO > > > metano Reading geom: # Metano, formato etiquetado "atomo" "X" "Y" "Z" > > > Reading geom: # el formato est? defomardo para ejercitar el parser > > > {system,[{atom,"C1",{cartesian,0.0,0.0,1.089}}, > > > > > > {atom,"H1",{cartesian,1.02672,0.0,-0.362996}}, > > > {atom,"H2",{cartesian,-0.51336,-0.889165,-0.363}}, > > > {atom,"H3",{cartesian,-0.51336,0.889165,-0.363}}], > > > > > > {cartesian,[0.0,0.0,0.0,0.0,0.0,-0.970109808,-0.685963027,0.0, > > > > > > 1.94021962,2.05791176,-0.685970586,1.68027834, > > > -0.970109808,-0.685970586,-1.68027834]}, > > > > > > {gradient,[-1.28369537e-16,3.49033183e-7,-5.12583544e-7, > > > > > > 2.74086309e-16,5.68362124e-7,-0.00249459546,-0.0017630 > > > 14 05, > > > -5.2909066e-17,0.00498827352,0.00529098265,-0.00176372 > > > 8 01, 0.00432037169,-0.00249459546,-0.00176372801, > > > -0.00432037169]}, > > > > > > {hessian,[-3.6869881e-16,2.60106291e-7,0.824862682,1.3382681e-16, > > > > > > 0.824865666,0.0696220165,-1.87359717e-8,-5.90803338e-17 > > > , > > > -0.0798212583,0.824862747,0.0696218852,-3.25950592e-19 > > > , -9.21169816e-17,-0.0798212533,-6.03652537e-17,..... > > > > > > .. more chem related data. > > > > > > > > > > > > > > > Today ive downloaded lastest rebar from git and tested the same way... > > > > > > sinosuke@REDACTED:~/Documentos/Trabajo/2012/Investigaci?n/ResMol/Erlan > > > g_ NoseHoover_16_12_2012/nosehoover> ./rebar -v clean compile > > > escriptize ==> nosehoover (clean) > > > ==> nosehoover (compile) > > > src/nosehoover.erl:48: Warning: variable 'OtrosArgs' is unused > > > Compiled src/nosehoover.erl > > > Compiled src/nosehoover_loader_gaussian.erl > > > src/nosehoover_core.erl:21: Warning: variable 'Other' is unused > > > Compiled src/nosehoover_core.erl > > > Compiled src/getopt.erl > > > Compiled src/nosehoover_tools.erl > > > ==> nosehoover (escriptize) > > > sinosuke@REDACTED:~/Documentos/Trabajo/2012/Investigaci?n/ResMol/Erlan > > > g_ NoseHoover_16_12_2012/nosehoover> ./nosehoover -D ./data/METANO > > > metano > > > > > > ./nosehoover:5: undefined macro '?' > > > ./nosehoover:18: illegal atom > > > ./nosehoover:25: illegal atom > > > ./nosehoover:25: syntax error before: 3 > > > ./nosehoover:28: illegal atom > > > ./nosehoover:28: illegal atom > > > ./nosehoover:30: illegal atom > > > ./nosehoover:34: syntax error before: M > > > ./nosehoover:34: undefined macro '??P???XK???n???QF' > > > ./nosehoover:48: illegal atom > > > ./nosehoover:50: illegal atom > > > ./nosehoover:56: illegal atom > > > ./nosehoover:66: illegal atom > > > ./nosehoover:75: undefined macro 'v' > > > ./nosehoover:88: illegal macro call '?'?'' > > > ./nosehoover:101: undefined macro '?x' > > > ./nosehoover:104: illegal atom > > > ./nosehoover:109: syntax error before: A? > > > ./nosehoover:115: illegal atom > > > ./nosehoover:119: illegal atom > > > ./nosehoover:143: illegal atom > > > ./nosehoover:146: illegal atom > > > ./nosehoover:149: syntax error before: '?' > > > ./nosehoover:154: undefined macro 'J?u' > > > ./nosehoover:160: illegal integer > > > ./nosehoover:161: undefined macro 'C' > > > ./nosehoover:177: illegal atom > > > ./nosehoover:181: undefined macro '?' > > > ./nosehoover:211: illegal atom > > > ./nosehoover:212: syntax error before: '#' > > > ./nosehoover:217: illegal atom > > > ./nosehoover:218: syntax error before: _ > > > ./nosehoover:222: undefined macro '?' > > > ./nosehoover:246: illegal atom > > > ./nosehoover:250: illegal macro call '?'?'' > > > ./nosehoover:264: illegal macro call '?'+'' > > > ./nosehoover:267: illegal atom > > > ./nosehoover:271: syntax error before: '?' > > > ./nosehoover:272: syntax error before: J?B > > > ./nosehoover:277: undefined macro '?' > > > ./nosehoover:281: illegal atom > > > ./nosehoover:291: illegal atom > > > ./nosehoover:291: syntax error before: '?' > > > ./nosehoover:292: illegal atom > > > ./nosehoover:298: illegal atom > > > ./nosehoover:305: illegal atom > > > ./nosehoover:328: illegal macro call '?3' > > > ./nosehoover:328: undefined macro '?' > > > escript: There were compilation errors. > > > > > > the generated script crashes badly and i dont know why. We are feared > > > of lossing our good rebar on a RPM update and face this problem with > > > newer rebar versions... > > > > > > ?Can anyone give a clue on what's wrong? > > > > > > Thanks! > > > > > > Angel > > > > > > > > > > > > > > > _______________________________________________ > > > madrid-erlang-users mailing list > > > madrid-erlang-users@REDACTED > > > https://babel.ls.fi.upm.es/cgi-bin/mailman/listinfo/madrid-erlang-users > > _______________________________________________ > madrid-erlang-users mailing list > madrid-erlang-users@REDACTED > https://babel.ls.fi.upm.es/cgi-bin/mailman/listinfo/madrid-erlang-users -- "but we no longer provide any UI to configure LVM or MD RAID itself - you are expected to use the command-line tools to do so yourself" , 2012 - Gnome journey back to the 90's -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Tue Mar 20 00:03:45 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 20 Mar 2012 00:03:45 +0100 Subject: [erlang-questions] [madrid-erlang-users] strange rebar error **SOLVED** In-Reply-To: <201203192253.41876.clist@uah.es> References: <201203192117.08878.clist@uah.es> <2999163138293542984@unknownmsgid> <201203192206.54568.clist@uah.es> <201203192253.41876.clist@uah.es> Message-ID: On Mon, Mar 19, 2012 at 10:53 PM, Angel J. Alvarez Miguel wrote: > rebar.config was causing trouble.... > > deleteing this file solves the problem but you have to write a new > one... > > Some options have changed hopefully.... > > By the way, the offending terms were: > > {escript_emu_args,["+sbtu","+K true","+A 2" ]}. You should use {escript_emu_args, "%%!+sbtu +Ktrue +A2\n"}. instead. rebar_escripter embeds the string as is, so don't forget the terminating \n (thanks Angel for the correction). > What are the current way of passing those args to the VM? escript_emu_args works and allows you to embed "permanent" args. Alternatively you can use the ERL_AFLAGS and ERL_FLAGS/ERL_ZFLAGS environment variables. From andy.richards.iit@REDACTED Tue Mar 20 00:06:53 2012 From: andy.richards.iit@REDACTED (Andy Richards) Date: Mon, 19 Mar 2012 23:06:53 +0000 Subject: [erlang-questions] Node deployment best practice Message-ID: Hi what I the best way to deploy a release to all nodes in a cluster? Do I have to do his manually or can I do it centrally from one node? Also once I hot deploy an upgrade to a node can I push this hot deployment out to all nodes? It looks as though if I use the release_handler I have to manually copy my release to all nodes to do a hot upgrade/release? Does anyone have any suggestions on best practices? Many thanks, Andy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Mar 20 02:35:02 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 20 Mar 2012 14:35:02 +1300 Subject: [erlang-questions] math:pow(X, Y). In-Reply-To: References: Message-ID: <37C435A0-E191-4583-852F-24E495F4BB56@cs.otago.ac.nz> On 20/03/2012, at 6:17 AM, Yves S. Garret wrote: > Hi all, > > That's the standard method that I use to raise a value to a power. I compared this to the way Python does raising to a power and this is the result that I get: > > Erlang: > 11> math:pow(42, 909). > ** exception error: bad argument in an arithmetic expression > in function math:pow/2 > called as math:pow(42,909) > > Python: > http://bin.cakephp.org/view/1006418268 > > Now. Why is it in Erlang's case I get a float as a return value? Because you explicitly called something that is *defined* to return floating point values. Basically, the math: module consists of wrappers around the C library. math:pow(1, 1) => 1.0 because math:pow/2 is something you call when you *want* a floating point result. > Is there a method that only returns a long? What's a "long"? -module(rtp). -export([rtp/2]). %% rtp(Base, Power) returns Base ** Power. %% The answer has the same type as Base. %% If Base is an integer, Power must be non-negative. -spec rtp(integer(), integer()) -> integer() ; (float(), integer()) -> float(). rtp(Base, Power) when is_integer(Base), is_integer(Power), Power >= 0 -> rtpi(Base, Power, 1); rtp(Base, Power) when is_float(Base), is_integer(Power), Power >= 0 -> rtpf(Base, Power, 1.0); rtp(Base, Power) when is_float(Base), is_integer(Power), Power < 0 -> 1.0 / rtpf(Base, -Power, 1.0). -spec rtpi(integer(), integer(), integer()) -> integer(). rtpi(Base, Power, R) when Power > 2 -> rtpi(Base * Base, Power bsr 1, case Power band 1 of 0 -> R ; 1 -> R*Base end); rtpi(Base, 2, R) -> R * (Base * Base); rtpi(Base, 1, R) -> R * Base; rtpi( _, 0, R) -> R. -spec rtpf(float(), integer(), float()) -> float(). rtpf(Base, Power, R) when Power > 2 -> rtpf(Base * Base, Power bsr 1, case Power band 1 of 0 -> R ; 1 -> R*Base end); rtpf(Base, 2, R) -> R * (Base * Base); rtpf(Base, 1, R) -> R * Base; rtpf( _, 0, R) -> R. From ok@REDACTED Tue Mar 20 02:38:40 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 20 Mar 2012 14:38:40 +1300 Subject: [erlang-questions] math:pow(X, Y). In-Reply-To: References: Message-ID: <9719F7B8-F5CF-402F-BD9A-205BEA1DE963@cs.otago.ac.nz> On 20/03/2012, at 6:29 AM, Bob Ippolito wrote: > In general the math library is for float arithmetic, just like in C. My guess is that the time when this was written, there was no bignum support. No, it's just that it is *supposed* to be a wrapper for C's pow(), which always answers doubles. It would be no bad thing to have the raise-to-an-integer-power function as a standard thing in Erlang/OTP, but it would be quite a bad thing to put it in the math: module. Is there any reason not to have a '**' operator in the language? I'd write an EEP for this, but now that you have to use Markdown, I can't write EEPs any more. From roberto@REDACTED Tue Mar 20 02:53:54 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 19 Mar 2012 18:53:54 -0700 Subject: [erlang-questions] Misultin 0.9 with "binary mode" enabled In-Reply-To: <2321D902-3A21-43C0-9BE4-479C67F158CB@gmail.com> References: <562209BD-5E83-4E2E-AEAD-471AF0205C26@gmail.com> <2321D902-3A21-43C0-9BE4-479C67F158CB@gmail.com> Message-ID: well, what is your question then? there'sn no such thing as 'binary mode' in misultin. perhaps you are referring to using the http_bin option as packet type? if so, why would you want to do that? if you are looking to optimize memory usage, this is not the main place to look to do so. instead, the fact that two processes are being used in misultin for every single connection, with headers being copied over from one another is where i'd probably start if memory is your main concern. cheers, r. On Mon, Mar 19, 2012 at 7:43 AM, Zabrane Mickael wrote: > Thanks, but I know that it was discontinued. > This didn't not answer my question ;-) ? > > Regards, > Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From jared.kofron@REDACTED Tue Mar 20 03:23:08 2012 From: jared.kofron@REDACTED (Jared Kofron) Date: Mon, 19 Mar 2012 19:23:08 -0700 Subject: [erlang-questions] rebar error Message-ID: <27261A65-B588-4B53-A711-29DBA3103A7B@gmail.com> Hi List, I just upgraded to R15B and now I am getting the following error when I try to generate a release with rebar: ==> release (generate) ERROR: Unexpected error: {'EXIT', {{badmatch, {error, {1, "cp: /Users/kofron/Code/Erlang/dripline/release/files/erl: No such file or directory\n"}}}, [{rebar_file_utils,cp_r,2,[]}, {rebar_reltool,execute_overlay,4,[]}, {rebar_reltool,generate,2,[]}, {rebar_core,run_modules,4, [{file,"src/rebar_core.erl"},{line,402}]}, {rebar_core,execute,4, [{file,"src/rebar_core.erl"},{line,336}]}, {rebar_core,process_dir0,6, [{file,"src/rebar_core.erl"},{line,201}]}, {rebar_core,process_each,5, [{file,"src/rebar_core.erl"},{line,270}]}, {rebar_core,process_dir0,6, [{file,"src/rebar_core.erl"}, {line,181}]}]}} make: *** [release] Error 1 erl is definitely in my path. I agree that dripline/release/files/erl does not exist, but I'm not sure what the fix is here. making clean makes no difference. Any ideas? From arthurbeall@REDACTED Tue Mar 20 03:54:44 2012 From: arthurbeall@REDACTED (Art Beall) Date: Mon, 19 Mar 2012 19:54:44 -0700 (PDT) Subject: [erlang-questions] Getting started with Cowboy Message-ID: <1332212084.89885.YahooMailNeo@web39301.mail.mud.yahoo.com> Hello, ? First post here, thanks for all the information provided through this digest! ? At my company we're looking at implementing an api using?a RESTfull web service. Erlang has been suggested. I've been looking for some documentation, examples or tutorials on using Cowboy. My background is in Windows/C++. I've taught myself Erlang from the Programming Erlang book. But the unix tools are mostly new to me. ? Hoping I could get some help or suggestions for getting started. ? Thanks for any help! Art -------------- next part -------------- An HTML attachment was scrubbed... URL: From jared.kofron@REDACTED Tue Mar 20 03:59:46 2012 From: jared.kofron@REDACTED (Jared Kofron) Date: Mon, 19 Mar 2012 19:59:46 -0700 Subject: [erlang-questions] Getting started with Cowboy In-Reply-To: <1332212084.89885.YahooMailNeo@web39301.mail.mud.yahoo.com> References: <1332212084.89885.YahooMailNeo@web39301.mail.mud.yahoo.com> Message-ID: Hi Art! I have no experience with Cowboy, but I can say from my RESTful API experience with Erlang, you might also look at Webmachine - mostly because I found their examples and diagrams most helpful when I was getting started. Best, JK On Mar 19, 2012, at 7:54 PM, Art Beall wrote: > Hello, > > First post here, thanks for all the information provided through this digest! > > At my company we're looking at implementing an api using a RESTfull web service. Erlang has been suggested. I've been looking for some documentation, examples or tutorials on using Cowboy. My background is in Windows/C++. I've taught myself Erlang from the Programming Erlang book. But the unix tools are mostly new to me. > > Hoping I could get some help or suggestions for getting started. > > Thanks for any help! > > Art > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Tue Mar 20 06:43:27 2012 From: bourinov@REDACTED (Max Bourinov) Date: Tue, 20 Mar 2012 09:43:27 +0400 Subject: [erlang-questions] Getting started with Cowboy In-Reply-To: References: <1332212084.89885.YahooMailNeo@web39301.mail.mud.yahoo.com> Message-ID: <3128580896263986815@unknownmsgid> Webmachines could do the job for you. Unix is not that hard to start. Once you know a bit you will see that it is very comfortable system. Sent from my iPhone On 20.03.2012, at 7:00, Jared Kofron wrote: Hi Art! I have no experience with Cowboy, but I can say from my RESTful API experience with Erlang, you might also look at Webmachine - mostly because I found their examples and diagrams most helpful when I was getting started. Best, JK On Mar 19, 2012, at 7:54 PM, Art Beall wrote: Hello, First post here, thanks for all the information provided through this digest! At my company we're looking at implementing an api using a RESTfull web service. Erlang has been suggested. I've been looking for some documentation, examples or tutorials on using Cowboy. My background is in Windows/C++. I've taught myself Erlang from the Programming Erlang book. But the unix tools are mostly new to me. Hoping I could get some help or suggestions for getting started. Thanks for any help! Art _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Tue Mar 20 08:06:11 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Tue, 20 Mar 2012 08:06:11 +0100 Subject: [erlang-questions] math:pow(X, Y). In-Reply-To: <9719F7B8-F5CF-402F-BD9A-205BEA1DE963@cs.otago.ac.nz> References: <9719F7B8-F5CF-402F-BD9A-205BEA1DE963@cs.otago.ac.nz> Message-ID: On Tue, Mar 20, 2012 at 02:38, Richard O'Keefe wrote: > I'd write an EEP for this, but now that you have to use Markdown, > I can't write EEPs any more. Richard, feel free to write it in any markup language you want. I will, and probably others will help too, gladly reformat it in Markdown for EEP process :) From daniel.eliasson@REDACTED Tue Mar 20 09:11:28 2012 From: daniel.eliasson@REDACTED (Daniel Eliasson) Date: Tue, 20 Mar 2012 09:11:28 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: <9FDE3C40-2713-471F-A7DE-9E9C775034E0@feuerlabs.com> References: <9FDE3C40-2713-471F-A7DE-9E9C775034E0@feuerlabs.com> Message-ID: Hi Wojciech, I think you should consider what kind of data you're storing and how you want to use it, and decide on your database from that. Mnesia is good for a "working database", but wouldn't be my choice for persistent storage of large amounts of data. You'll hog a lot of RAM and run into table size limits. Riak seems to be a better choice for large distributed key-value storage, assuming you can live without transactions. But ask yourself: will I need to use this data outside my application? Do you need to perform analysis, run arbitrary queries on it? If the answer is yes, I would use PostgreSQL or some other competent relational database. Another advantage (a really big one) of having a database with proper schemas is that you get your datatypes defined in at least one place. A dynamic language like Erlang combined with a database like Mnesia that happily stores whatever terms for you easily leads to very ill defined data structures. That will lead to a lot of pain over time. Best, Daniel Eliasson On 19 March 2012 18:54, Ulf Wiger wrote: > > On 19 Mar 2012, at 18:42, Wojciech Knapik wrote: > > > > On Mon, Mar 19, 2012 at 6:08 PM, Jesse Gumm wrote: >> >> Have you looked into riak for distributed, master-less data replication? > > > From what I've read on github, I can't tell what the benefits would be and I > was hoping to familiarize myself with mnesia. > > Would riak be a better fit ? > > > > It depends much on your access patterns. > > Mnesia offers transaction consistency, meaning that you can run fairly > complex update transactions and let mnesia worry about consistency across > your data set. This is a wonderful feature, but it tends to become > troublesome in very large clusters, since the increasing cost of ensuring > consistency puts a limit on scalability, and netsplits become increasingly > likely as your clusters go. Netsplits and transaction consistency make poor > bedfellows. > > Riak is a key-value database with no transaction concept whatsoever > (although individual writes are of course atomic). Thus, if you have > concurrent read-write patterns against the same data, you will find the > results ?arm, interesting for any but the simplest of update patterns. > > (See > e.g.?http://www.infoq.com/presentations/Eventually-Consistent-HTTP-with-Statebox-and-Riak?for > an example of roughly how far you can push concurrent updates in riak. It > should also give you an idea of how riak works.) > > OTOH, if all you do is simple get/put, and you need high throughput and > excellent fault tolerance, riak will let you scale to levels way beyond what > any sane person would attempt with mnesia. > > BR, > Ulf W > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From andy.richards.iit@REDACTED Tue Mar 20 10:03:48 2012 From: andy.richards.iit@REDACTED (Andy Richards) Date: Tue, 20 Mar 2012 09:03:48 +0000 Subject: [erlang-questions] rebar error In-Reply-To: <27261A65-B588-4B53-A711-29DBA3103A7B@gmail.com> References: <27261A65-B588-4B53-A711-29DBA3103A7B@gmail.com> Message-ID: Hi Jared, Just a thought have you run ../rebar create-node nodeid=dripline from your releases dir as I thought this command generates these files looking at the rebar wiki. https://github.com/basho/rebar/wiki/Release-handling Cheers, Andy. On Tuesday, 20 March 2012, Jared Kofron wrote: > Hi List, > I just upgraded to R15B and now I am getting the following error when I try to generate a release with rebar: > > ==> release (generate) > ERROR: Unexpected error: {'EXIT', > {{badmatch, > {error, > {1, > "cp: /Users/kofron/Code/Erlang/dripline/release/files/erl: No such file or directory\n"}}}, > [{rebar_file_utils,cp_r,2,[]}, > {rebar_reltool,execute_overlay,4,[]}, > {rebar_reltool,generate,2,[]}, > {rebar_core,run_modules,4, > [{file,"src/rebar_core.erl"},{line,402}]}, > {rebar_core,execute,4, > [{file,"src/rebar_core.erl"},{line,336}]}, > {rebar_core,process_dir0,6, > [{file,"src/rebar_core.erl"},{line,201}]}, > {rebar_core,process_each,5, > [{file,"src/rebar_core.erl"},{line,270}]}, > {rebar_core,process_dir0,6, > [{file,"src/rebar_core.erl"}, > {line,181}]}]}} > make: *** [release] Error 1 > > erl is definitely in my path. I agree that dripline/release/files/erl does not exist, but I'm not sure what the fix is here. > making clean makes no difference. Any ideas? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Tue Mar 20 11:10:51 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 20 Mar 2012 19:10:51 +0900 Subject: [erlang-questions] math:pow(X, Y). In-Reply-To: <20120319182333.GA13317@corelatus.se> References: <20120319182333.GA13317@corelatus.se> Message-ID: "As these are the C library, the bugs are the same." I'm not sure what the IEEE floating point spec says about cases where you've exceeded what can be represented in double precision floating point, in computing an exponential. "Bad argument" might not be the best way to express what went wrong, but something is definitely wrong with the computation itself: you can't express a number that big in IEEE double precision. Maybe the exception thrown in the standard is something like "bad argument." -michael turner On Tue, Mar 20, 2012 at 3:23 AM, Matthias Lang wrote: > On Monday, March 19, Jared Kofron wrote: > >> Just a guess, but math:pow/2 is implemented as a NIF and C may not >> enjoy the size of the number you are trying to calculate. > > Exactly. > > The bottom of the 'math' manpage sort of says so, under 'bugs': > > ?"As these are the C library, the bugs are the same." > > Matt > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Tue Mar 20 11:42:27 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 20 Mar 2012 11:42:27 +0100 Subject: [erlang-questions] Getting started with Cowboy In-Reply-To: <1332212084.89885.YahooMailNeo@web39301.mail.mud.yahoo.com> References: <1332212084.89885.YahooMailNeo@web39301.mail.mud.yahoo.com> Message-ID: <4F685F13.7050508@ninenines.eu> People using the Cowboy REST handlers are currently using the Webmachine documentation to do it. It's based on the same diagram and the resource callbacks have the same names and mostly the same semantics. Proper documentation is in preparation. On 03/20/2012 03:54 AM, Art Beall wrote: > Hello, > First post here, thanks for all the information provided through this > digest! > At my company we're looking at implementing an api using a RESTfull web > service. Erlang has been suggested. I've been looking for some > documentation, examples or tutorials on using Cowboy. My background is > in Windows/C++. I've taught myself Erlang from the Programming Erlang > book. But the unix tools are mostly new to me. > Hoping I could get some help or suggestions for getting started. > Thanks for any help! > Art > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Erlang Cowboy Nine Nines From andy.richards.iit@REDACTED Tue Mar 20 11:57:20 2012 From: andy.richards.iit@REDACTED (Andy Richards) Date: Tue, 20 Mar 2012 10:57:20 +0000 Subject: [erlang-questions] downgrade / rollback a hot deployment without restarting OTP In-Reply-To: References: Message-ID: Hi, thanks all! yes this resolved my issue. I did read the release handler docs but didn't figure that downgrading would just mean reinstalling the older version. I did have to work around a couple of gotchas, as I was using rebar to generate my release and my .boot file was called my app.boot not start.boot which the release handler was looking for. A sym link fixed this. Also I didn't realease that code_change was called in my upgrade release gem_server which allowed me to revert the State before downgrading to the original version. Thanks again for confirming release_handler:install_version was the way to go. Andy. On Friday, 16 March 2012, Samuel Elliott wrote: > On Fri, Mar 16, 2012 at 1:54 PM, Per Melin wrote: >> On Mar 16, 2012, at 11:13 , Andy Richards wrote: >> >>> I'm struggling to find documentation on how I perform a rollback of a >>> unsuccessful OTP hot deployment. I'm currently manually creating my .appup >>> file with upgrade and downgrade instructions and when I perform a >>> release_handler:install_release("someApp") my application hot deploys >>> successfully, excellent! What i cant figure out however is if there were a >>> problem following an upgrade how I downgrade / rollback to the previous >>> release? >> >> It's been a long time since I last did this by hand, but if I remember correctly you simply do release_handler:install_release("OldVer") to roll back. > > Seems so, reading Ferd's Guide: > http://learnyousomeerlang.com/relups#upgrading-the-release > > Sam > >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Samuel Elliott > sam@REDACTED > http://lenary.co.uk/ > +44 (0)7891 993 664 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias@REDACTED Tue Mar 20 13:47:47 2012 From: matthias@REDACTED (Matthias Lang) Date: Tue, 20 Mar 2012 13:47:47 +0100 Subject: [erlang-questions] math:pow(X, Y). In-Reply-To: References: <20120319182333.GA13317@corelatus.se> Message-ID: <20120320124747.GA2490@corelatus.se> On Tuesday, March 20, Michael Turner wrote: > "As these are the C library, the bugs are the same." > I'm not sure what the IEEE floating point spec says The 'math' module doesn't mention IEEE. There was a mention of it in a thread about NaN a few weeks ago: http://erlang.org/pipermail/erlang-questions/2012-February/064713.html But anyway, I'm interested in understanding, and, to do that, just follow the clues: Clue #1: The man page says it's "just the C library" Clue #2: The C library on your machine probably has documentation for what pow() does. On mine, it says: RETURN VALUE On success, these functions return the value of x to the power of y. If x is a finite value less than 0, and y is a finite noninteger, a domain error occurs, and a NaN is returned. If the result overflows, a range error occurs, and the functions return HUGE_VAL, HUGE_VALF, or HUGE_VALL, respectively, with the mathemati? cally correct sign. If result underflows, and is not representable, a range error occurs, and 0.0 is returned. Clue #3: Take a look at the Erlang source. It's in erl_math.c, which is just 233 lines long, so nothing hard. Looking at it, it funnels all functions through math_call_2() or math_call_1(), and that checks that the output is finite (in the macro ERTS_FP_ERROR_THOROUGH). Finished. If the underlying C function returns NaN or INF, then Erlang is going to throw a badarith, and that's true for all the functions in the math module. There's one dangling thing: I expect to see badarith, but the pretty printed error suggests a badarg: 8> math:pow(20,900). ** exception error: bad argument in an arithmetic expression in function math:pow/2 called as math:pow(20,900) The non-pretty one doesn't: 9> catch math:pow(20,900). {'EXIT',{badarith,[{math,pow,[20,900],[]}, The pretty error comes from stdlib/src/lib.erl. Maybe "error evaluating arithmetic expression" would cover things better. Matt From straightflush@REDACTED Tue Mar 20 14:09:03 2012 From: straightflush@REDACTED (AD) Date: Tue, 20 Mar 2012 09:09:03 -0400 Subject: [erlang-questions] Getting started with Cowboy In-Reply-To: <4F685F13.7050508@ninenines.eu> References: <1332212084.89885.YahooMailNeo@web39301.mail.mud.yahoo.com> <4F685F13.7050508@ninenines.eu> Message-ID: I am fairly new to cowboy/erlang as well but found it very easy to use. You simply need to write your own handler module and then in your handle(Req,State) callback you just do what you want with the request. you can make a simple case statement to match on paths (look at cowboy_http_req). For example to check for a header I think you can do something like case cowboy_http_req:header('Host',Req) of 'www.domain.com' -> {ok, Req2} = cowboy_http_req:reply(200, [{'Content-Type', <<"text/html">>}],<<"You went to www !">>,Req); 'images.domain.com' -> {ok, Req2} = cowboy_http_req:reply(200, [{'Content-Type', <<"text/html">>}],<<"You went to images">>,Req); end {ok, Req2, State}. Pretty straightforward. -AD On Tue, Mar 20, 2012 at 6:42 AM, Lo?c Hoguin wrote: > People using the Cowboy REST handlers are currently using the Webmachine > documentation to do it. It's based on the same diagram and the resource > callbacks have the same names and mostly the same semantics. > > Proper documentation is in preparation. > > > On 03/20/2012 03:54 AM, Art Beall wrote: > >> Hello, >> First post here, thanks for all the information provided through this >> digest! >> At my company we're looking at implementing an api using a RESTfull web >> service. Erlang has been suggested. I've been looking for some >> documentation, examples or tutorials on using Cowboy. My background is >> in Windows/C++. I've taught myself Erlang from the Programming Erlang >> book. But the unix tools are mostly new to me. >> Hoping I could get some help or suggestions for getting started. >> Thanks for any help! >> Art >> >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aleksandr.vin@REDACTED Tue Mar 20 16:07:18 2012 From: aleksandr.vin@REDACTED (Aleksandr Vinokurov) Date: Tue, 20 Mar 2012 19:07:18 +0400 Subject: [erlang-questions] math:pow(X, Y). In-Reply-To: <37C435A0-E191-4583-852F-24E495F4BB56@cs.otago.ac.nz> References: <37C435A0-E191-4583-852F-24E495F4BB56@cs.otago.ac.nz> Message-ID: <48BCC786-151C-4835-9EA2-DB519871601F@gmail.com> Richard, what the point of writing two equal funs? rtpi and rtpf? Really. 20.03.2012, ? 5:35, "Richard O'Keefe" ???????(?): > > On 20/03/2012, at 6:17 AM, Yves S. Garret wrote: > >> Hi all, >> >> That's the standard method that I use to raise a value to a power. I compared this to the way Python does raising to a power and this is the result that I get: >> >> Erlang: >> 11> math:pow(42, 909). >> ** exception error: bad argument in an arithmetic expression >> in function math:pow/2 >> called as math:pow(42,909) >> >> Python: >> http://bin.cakephp.org/view/1006418268 >> >> Now. Why is it in Erlang's case I get a float as a return value? > > Because you explicitly called something that is *defined* to return > floating point values. Basically, the math: module consists of > wrappers around the C library. math:pow(1, 1) => 1.0 > because math:pow/2 is something you call when you *want* a floating > point result. > >> Is there a method that only returns a long? > > What's a "long"? > > -module(rtp). > -export([rtp/2]). > > %% rtp(Base, Power) returns Base ** Power. > %% The answer has the same type as Base. > %% If Base is an integer, Power must be non-negative. > > -spec rtp(integer(), integer()) -> integer() > ; (float(), integer()) -> float(). > > rtp(Base, Power) > when is_integer(Base), is_integer(Power), Power >= 0 -> > rtpi(Base, Power, 1); > rtp(Base, Power) > when is_float(Base), is_integer(Power), Power >= 0 -> > rtpf(Base, Power, 1.0); > rtp(Base, Power) > when is_float(Base), is_integer(Power), Power < 0 -> > 1.0 / rtpf(Base, -Power, 1.0). > > -spec rtpi(integer(), integer(), integer()) -> integer(). > > rtpi(Base, Power, R) when Power > 2 -> > rtpi(Base * Base, Power bsr 1, > case Power band 1 > of 0 -> R > ; 1 -> R*Base > end); > rtpi(Base, 2, R) -> R * (Base * Base); > rtpi(Base, 1, R) -> R * Base; > rtpi( _, 0, R) -> R. > > -spec rtpf(float(), integer(), float()) -> float(). > > rtpf(Base, Power, R) when Power > 2 -> > rtpf(Base * Base, Power bsr 1, > case Power band 1 > of 0 -> R > ; 1 -> R*Base > end); > rtpf(Base, 2, R) -> R * (Base * Base); > rtpf(Base, 1, R) -> R * Base; > rtpf( _, 0, R) -> R. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dawid.figiel@REDACTED Tue Mar 20 17:17:09 2012 From: dawid.figiel@REDACTED (Dawid Figiel) Date: Tue, 20 Mar 2012 17:17:09 +0100 Subject: [erlang-questions] How to clean up before termination... Message-ID: Hi, There is a recommendation in Erlang doc, which is telling to set up the trap_exit to true in the gen_server:init, when want to do clean up before supervisor will terminate the gen_server. http://www.erlang.org/doc/design_principles/gen_server_concepts.html#id66567 My question is: What for trap_exit has to be set to true when the clean up is needed? The place for doing some clean ups is under terminate function. We want to have trape_exit set to true when we need to handle 'EXIT' signals coming from linked processes to (for example) not crash gen_server. The only explanation for me is that the supervisor may crash too and send 'Exit' to gen_server as well... but this doesn't convince even myself. Has anyone some explanation for this advice ? Best, Dawid From jared.kofron@REDACTED Tue Mar 20 17:20:44 2012 From: jared.kofron@REDACTED (Jared Kofron) Date: Tue, 20 Mar 2012 09:20:44 -0700 Subject: [erlang-questions] rebar error In-Reply-To: References: <27261A65-B588-4B53-A711-29DBA3103A7B@gmail.com> Message-ID: <02BBB8A1-EB45-4962-A32F-4D5C9A32F8EF@gmail.com> Hi All, I sorted it out. Somewhere deep in the bowels erts was carrying a version tag around from my old version of erlang. Starting from scratch was the way to fix this all up. Thanks for the suggestions - JK On Mar 20, 2012, at 2:03 AM, Andy Richards wrote: > Hi Jared, > > Just a thought have you run ../rebar create-node nodeid=dripline from your releases dir as I thought this command generates these files looking at the rebar wiki. > > https://github.com/basho/rebar/wiki/Release-handling > > Cheers, > > Andy. > > On Tuesday, 20 March 2012, Jared Kofron wrote: > > Hi List, > > I just upgraded to R15B and now I am getting the following error when I try to generate a release with rebar: > > > > ==> release (generate) > > ERROR: Unexpected error: {'EXIT', > > {{badmatch, > > {error, > > {1, > > "cp: /Users/kofron/Code/Erlang/dripline/release/files/erl: No such file or directory\n"}}}, > > [{rebar_file_utils,cp_r,2,[]}, > > {rebar_reltool,execute_overlay,4,[]}, > > {rebar_reltool,generate,2,[]}, > > {rebar_core,run_modules,4, > > [{file,"src/rebar_core.erl"},{line,402}]}, > > {rebar_core,execute,4, > > [{file,"src/rebar_core.erl"},{line,336}]}, > > {rebar_core,process_dir0,6, > > [{file,"src/rebar_core.erl"},{line,201}]}, > > {rebar_core,process_each,5, > > [{file,"src/rebar_core.erl"},{line,270}]}, > > {rebar_core,process_dir0,6, > > [{file,"src/rebar_core.erl"}, > > {line,181}]}]}} > > make: *** [release] Error 1 > > > > erl is definitely in my path. I agree that dripline/release/files/erl does not exist, but I'm not sure what the fix is here. > > making clean makes no difference. Any ideas? > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From hynek@REDACTED Tue Mar 20 17:22:37 2012 From: hynek@REDACTED (Hynek Vychodil) Date: Tue, 20 Mar 2012 17:22:37 +0100 Subject: [erlang-questions] math:pow(X, Y). In-Reply-To: <48BCC786-151C-4835-9EA2-DB519871601F@gmail.com> References: <37C435A0-E191-4583-852F-24E495F4BB56@cs.otago.ac.nz> <48BCC786-151C-4835-9EA2-DB519871601F@gmail.com> Message-ID: I bet it is because HiPE will optimize it in different way and result will be faster. 2012/3/20 Aleksandr Vinokurov : > Richard, what the point of writing two equal funs? rtpi and rtpf? Really. > > > > 20.03.2012, ? 5:35, "Richard O'Keefe" ???????(?): > >> >> On 20/03/2012, at 6:17 AM, Yves S. Garret wrote: >> >>> Hi all, >>> >>> ? That's the standard method that I use to raise a value to a power. ?I compared this to the way Python does raising to a power and this is the result that I get: >>> >>> Erlang: >>> 11> math:pow(42, 909). >>> ** exception error: bad argument in an arithmetic expression >>> ? ? in function ?math:pow/2 >>> ? ? ? ?called as math:pow(42,909) >>> >>> Python: >>> http://bin.cakephp.org/view/1006418268 >>> >>> Now. ?Why is it in Erlang's case I get a float as a return value? >> >> Because you explicitly called something that is *defined* to return >> floating point values. ?Basically, the math: module consists of >> wrappers around the C library. ?math:pow(1, 1) => 1.0 >> because math:pow/2 is something you call when you *want* a floating >> point result. >> >>> Is there a method that only returns a long? >> >> What's a "long"? >> >> -module(rtp). >> -export([rtp/2]). >> >> %% rtp(Base, Power) returns Base ** Power. >> %% The answer has the same type as Base. >> %% If Base is an integer, Power must be non-negative. >> >> -spec rtp(integer(), integer()) -> integer() >> ? ?; ? ?(float(), ? integer()) -> float(). >> >> rtp(Base, Power) >> ?when is_integer(Base), is_integer(Power), Power >= 0 -> >> ? ?rtpi(Base, Power, 1); >> rtp(Base, Power) >> ?when is_float(Base), is_integer(Power), Power >= 0 -> >> ? ?rtpf(Base, Power, 1.0); >> rtp(Base, Power) >> ?when is_float(Base), is_integer(Power), Power < 0 -> >> ? ?1.0 / rtpf(Base, -Power, 1.0). >> >> -spec rtpi(integer(), integer(), integer()) -> integer(). >> >> rtpi(Base, Power, R) when Power > 2 -> >> ? ?rtpi(Base * Base, Power bsr 1, >> ? ? ? ?case Power band 1 >> ? ? ? ? ?of 0 -> R >> ? ? ? ? ? ; 1 -> R*Base >> ? ? ? ?end); >> rtpi(Base, 2, R) -> R * (Base * Base); >> rtpi(Base, 1, R) -> R * Base; >> rtpi( ? _, 0, R) -> R. >> >> -spec rtpf(float(), integer(), float()) -> float(). >> >> rtpf(Base, Power, R) when Power > 2 -> >> ? ?rtpf(Base * Base, Power bsr 1, >> ? ? ? ?case Power band 1 >> ? ? ? ? ?of 0 -> R >> ? ? ? ? ? ; 1 -> R*Base >> ? ? ? ?end); >> rtpf(Base, 2, R) -> R * (Base * Base); >> rtpf(Base, 1, R) -> R * Base; >> rtpf( ? _, 0, R) -> R. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hynek Vychodil BI consultant GoodData n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole Office: ? +420 530 50 7704 E-mail: ?hynek@REDACTED Web: ? ? www.gooddata.com From ttom.kelly@REDACTED Tue Mar 20 17:29:21 2012 From: ttom.kelly@REDACTED (tom kelly) Date: Tue, 20 Mar 2012 16:29:21 +0000 Subject: [erlang-questions] How to clean up before termination... In-Reply-To: References: Message-ID: Hi Dawid, You may have mis-read the documentation slightly, or else I misunderstand your question. If trap_exit is set to true then the terminate function is called when the gen_server is shutting down, if it is false then the terminate function is never called. All cleanup should be done in terminate, nowhere else. Is this what you're asking? //TTom. On Tue, Mar 20, 2012 at 4:17 PM, Dawid Figiel wrote: > Hi, > > There is a recommendation in Erlang doc, which is telling to set up > the trap_exit to true in the gen_server:init, when want to do clean up > before supervisor will terminate the gen_server. > > > http://www.erlang.org/doc/design_principles/gen_server_concepts.html#id66567 > > My question is: > What for trap_exit has to be set to true when the clean up is needed? > > The place for doing some clean ups is under terminate function. > We want to have trape_exit set to true when we need to handle 'EXIT' > signals coming from linked processes to (for example) not crash > gen_server. > > The only explanation for me is that the supervisor may crash too and > send 'Exit' to gen_server as well... but this doesn't convince even > myself. > > Has anyone some explanation for this advice ? > > > Best, > > Dawid > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jimenezrick@REDACTED Tue Mar 20 17:35:02 2012 From: jimenezrick@REDACTED (Ricardo Catalinas =?iso-8859-1?Q?Jim=E9nez?=) Date: Tue, 20 Mar 2012 17:35:02 +0100 Subject: [erlang-questions] Comparing terms == vs. =:= Message-ID: <20120320163502.GA935@viper.local> I have read here http://erlang.org/pipermail/erlang-patches/2012-March/002721.html: "It is always better to use =:= instead of == when not comparing numbers" So, apart from prevent an implicit integer cast to float in the comparison: 1> 1 =:= 1.0. false What is the purpose of using =:= with all kind of terms? Better performance? Best regards -- Ricardo (http://r.untroubled.be/) From dmkolesnikov@REDACTED Tue Mar 20 17:43:42 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Tue, 20 Mar 2012 18:43:42 +0200 Subject: [erlang-questions] How to clean up before termination... In-Reply-To: References: Message-ID: <-3099311795155292345@unknownmsgid> Hi, In some cases a gen_server might spawn a linked child processes. You might want to flush some buffers on disk if one of child dies. But this is really an application specific. In most of cases you can ignore it. Simply let it fail! Best Regards, Dmitry >-|-|-*> On 20.3.2012, at 18.17, Dawid Figiel wrote: > Hi, > > There is a recommendation in Erlang doc, which is telling to set up > the trap_exit to true in the gen_server:init, when want to do clean up > before supervisor will terminate the gen_server. > > http://www.erlang.org/doc/design_principles/gen_server_concepts.html#id66567 > > My question is: > What for trap_exit has to be set to true when the clean up is needed? > > The place for doing some clean ups is under terminate function. > We want to have trape_exit set to true when we need to handle 'EXIT' > signals coming from linked processes to (for example) not crash > gen_server. > > The only explanation for me is that the supervisor may crash too and > send 'Exit' to gen_server as well... but this doesn't convince even > myself. > > Has anyone some explanation for this advice ? > > > Best, > > Dawid > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dawid.figiel@REDACTED Tue Mar 20 17:44:46 2012 From: dawid.figiel@REDACTED (Dawid Figiel) Date: Tue, 20 Mar 2012 17:44:46 +0100 Subject: [erlang-questions] How to clean up before termination... In-Reply-To: References: Message-ID: Heh.. I read it view times, then I call someone to read this for me... then that someone becomes confused too ;) (1) We jump to terminate/1 when supervisor wants to shut down the gen_server. (2) Trap_exit lets you handle 'EXIT' signals from linked processes. What is the relation between (1) and (2). In my opinion function terminate has nothing to do with trap_exit. Regards, Dawid On 3/20/12, tom kelly wrote: > Hi Dawid, > > You may have mis-read the documentation slightly, or else I misunderstand > your question. > > If trap_exit is set to true then the terminate function is called when the > gen_server is shutting down, if it is false then the terminate function is > never called. All cleanup should be done in terminate, nowhere else. > > Is this what you're asking? > > //TTom. > > > > On Tue, Mar 20, 2012 at 4:17 PM, Dawid Figiel wrote: > >> Hi, >> >> There is a recommendation in Erlang doc, which is telling to set up >> the trap_exit to true in the gen_server:init, when want to do clean up >> before supervisor will terminate the gen_server. >> >> >> http://www.erlang.org/doc/design_principles/gen_server_concepts.html#id66567 >> >> My question is: >> What for trap_exit has to be set to true when the clean up is needed? >> >> The place for doing some clean ups is under terminate function. >> We want to have trape_exit set to true when we need to handle 'EXIT' >> signals coming from linked processes to (for example) not crash >> gen_server. >> >> The only explanation for me is that the supervisor may crash too and >> send 'Exit' to gen_server as well... but this doesn't convince even >> myself. >> >> Has anyone some explanation for this advice ? >> >> >> Best, >> >> Dawid >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From ttom.kelly@REDACTED Tue Mar 20 17:52:17 2012 From: ttom.kelly@REDACTED (tom kelly) Date: Tue, 20 Mar 2012 16:52:17 +0000 Subject: [erlang-questions] How to clean up before termination... In-Reply-To: References: Message-ID: I see what you mean now, I should have read the document you linked properly before posting ;-) It seems either gen_server or supervisor makes a decision on whether or not to execute terminate/1 based on trap_exit being set. It's not immediately obvious to me why that should be. Time for a browse through the source... On Tue, Mar 20, 2012 at 4:44 PM, Dawid Figiel wrote: > Heh.. > > I read it view times, then I call someone to read this for me... then > that someone becomes confused too ;) > > (1) We jump to terminate/1 when supervisor wants to shut down the > gen_server. > (2) Trap_exit lets you handle 'EXIT' signals from linked processes. > > What is the relation between (1) and (2). > In my opinion function terminate has nothing to do with trap_exit. > > Regards, > > Dawid > > > On 3/20/12, tom kelly wrote: > > Hi Dawid, > > > > You may have mis-read the documentation slightly, or else I misunderstand > > your question. > > > > If trap_exit is set to true then the terminate function is called when > the > > gen_server is shutting down, if it is false then the terminate function > is > > never called. All cleanup should be done in terminate, nowhere else. > > > > Is this what you're asking? > > > > //TTom. > > > > > > > > On Tue, Mar 20, 2012 at 4:17 PM, Dawid Figiel >wrote: > > > >> Hi, > >> > >> There is a recommendation in Erlang doc, which is telling to set up > >> the trap_exit to true in the gen_server:init, when want to do clean up > >> before supervisor will terminate the gen_server. > >> > >> > >> > http://www.erlang.org/doc/design_principles/gen_server_concepts.html#id66567 > >> > >> My question is: > >> What for trap_exit has to be set to true when the clean up is needed? > >> > >> The place for doing some clean ups is under terminate function. > >> We want to have trape_exit set to true when we need to handle 'EXIT' > >> signals coming from linked processes to (for example) not crash > >> gen_server. > >> > >> The only explanation for me is that the supervisor may crash too and > >> send 'Exit' to gen_server as well... but this doesn't convince even > >> myself. > >> > >> Has anyone some explanation for this advice ? > >> > >> > >> Best, > >> > >> Dawid > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Tue Mar 20 17:57:50 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 20 Mar 2012 17:57:50 +0100 Subject: [erlang-questions] How to clean up before termination... In-Reply-To: References: Message-ID: Hi Dawid, The gen_server loop will always initiate a shutdown if it receives an {'EXIT', Parent, shutdown}. As part of this process, it calls Mod:terminate(shutdown, ModState). If it doesn't trap exits, the Erlang VM will simply terminate the process, and the EXIT signal will never reach the erlang code level. Thus, without the trap_exit flag set, there is no way the terminate/2 function can be called. To further clarify, when the supervisor wants to terminate a child, it doesn't ask nicely; it simply calls exit(Child, shutdown). BR, Ulf W On 20 Mar 2012, at 17:44, Dawid Figiel wrote: > Heh.. > > I read it view times, then I call someone to read this for me... then > that someone becomes confused too ;) > > (1) We jump to terminate/1 when supervisor wants to shut down the gen_server. > (2) Trap_exit lets you handle 'EXIT' signals from linked processes. > > What is the relation between (1) and (2). > In my opinion function terminate has nothing to do with trap_exit. > > Regards, > > Dawid > > > On 3/20/12, tom kelly wrote: >> Hi Dawid, >> >> You may have mis-read the documentation slightly, or else I misunderstand >> your question. >> >> If trap_exit is set to true then the terminate function is called when the >> gen_server is shutting down, if it is false then the terminate function is >> never called. All cleanup should be done in terminate, nowhere else. >> >> Is this what you're asking? >> >> //TTom. >> >> >> >> On Tue, Mar 20, 2012 at 4:17 PM, Dawid Figiel wrote: >> >>> Hi, >>> >>> There is a recommendation in Erlang doc, which is telling to set up >>> the trap_exit to true in the gen_server:init, when want to do clean up >>> before supervisor will terminate the gen_server. >>> >>> >>> http://www.erlang.org/doc/design_principles/gen_server_concepts.html#id66567 >>> >>> My question is: >>> What for trap_exit has to be set to true when the clean up is needed? >>> >>> The place for doing some clean ups is under terminate function. >>> We want to have trape_exit set to true when we need to handle 'EXIT' >>> signals coming from linked processes to (for example) not crash >>> gen_server. >>> >>> The only explanation for me is that the supervisor may crash too and >>> send 'Exit' to gen_server as well... but this doesn't convince even >>> myself. >>> >>> Has anyone some explanation for this advice ? >>> >>> >>> Best, >>> >>> Dawid >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Tue Mar 20 18:27:22 2012 From: g@REDACTED (Garrett Smith) Date: Tue, 20 Mar 2012 12:27:22 -0500 Subject: [erlang-questions] [ANN] March Chicago Erlang User Group - this Thursday Message-ID: For those in the Chicago area this week, the March Erlang User Group meeting promises to be utterly fantastic! The Future Of Erlang - An Insider's Perspective Thursday, March 22, 2012, 6:30 PM 500 West Madison Street (Orbitz) http://www.meetup.com/ErlangChicago/events/55358672/ Garrett From emeka_1978@REDACTED Tue Mar 20 19:05:24 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Tue, 20 Mar 2012 11:05:24 -0700 (PDT) Subject: [erlang-questions] Help withwxErlang dragging Message-ID: <3db76448-cf44-486e-b6b4-b8b60b692bfc@v2g2000vbx.googlegroups.com> Frame = wxFrame:new(wx:new(),-1,"Frame Without Border",[{style, ? wxCLIP_CHILDREN bor ?wxFRAME_NO_TASKBAR bor ?wxNO_BORDER bor ?wxFRAME_SHAPED}]), wxFrame:connect(Frame,motion,[{callback,fun(Ev,Obj) -> onMouse(Ev,Frame) end}]), ... onMouse(Ev,Obj) -> io:format("on mouse called ..."), case wxMouse:dragging(Ev#wx.event) of %% Error! false ->0 ok; %% no dragging _ -> %% get the position of the mouse and %% pass it around somehow Any idea about how i can go about implementing "onMouse()" ? Thank. From dawid.figiel@REDACTED Tue Mar 20 19:18:13 2012 From: dawid.figiel@REDACTED (Dawid Figiel) Date: Tue, 20 Mar 2012 19:18:13 +0100 Subject: [erlang-questions] How to clean up before termination... In-Reply-To: References: Message-ID: Hi Ulf! :) OK,... but shame on me I'd forgotten that. http://www.erlang.org/doc/man/gen_server.html#Module:terminate-2 Thanks! Dawid On 3/20/12, Ulf Wiger wrote: > > Hi Dawid, > > The gen_server loop will always initiate a shutdown if it receives an > {'EXIT', Parent, shutdown}. As part of this process, it calls > Mod:terminate(shutdown, ModState). If it doesn't trap exits, the Erlang VM > will simply terminate the process, and the EXIT signal will never reach the > erlang code level. Thus, without the trap_exit flag set, there is no way the > terminate/2 function can be called. > > To further clarify, when the supervisor wants to terminate a child, it > doesn't ask nicely; it simply calls exit(Child, shutdown). > > BR, > Ulf W > > On 20 Mar 2012, at 17:44, Dawid Figiel wrote: > >> Heh.. >> >> I read it view times, then I call someone to read this for me... then >> that someone becomes confused too ;) >> >> (1) We jump to terminate/1 when supervisor wants to shut down the >> gen_server. >> (2) Trap_exit lets you handle 'EXIT' signals from linked processes. >> >> What is the relation between (1) and (2). >> In my opinion function terminate has nothing to do with trap_exit. >> >> Regards, >> >> Dawid >> >> >> On 3/20/12, tom kelly wrote: >>> Hi Dawid, >>> >>> You may have mis-read the documentation slightly, or else I misunderstand >>> your question. >>> >>> If trap_exit is set to true then the terminate function is called when >>> the >>> gen_server is shutting down, if it is false then the terminate function >>> is >>> never called. All cleanup should be done in terminate, nowhere else. >>> >>> Is this what you're asking? >>> >>> //TTom. >>> >>> >>> >>> On Tue, Mar 20, 2012 at 4:17 PM, Dawid Figiel >>> wrote: >>> >>>> Hi, >>>> >>>> There is a recommendation in Erlang doc, which is telling to set up >>>> the trap_exit to true in the gen_server:init, when want to do clean up >>>> before supervisor will terminate the gen_server. >>>> >>>> >>>> http://www.erlang.org/doc/design_principles/gen_server_concepts.html#id66567 >>>> >>>> My question is: >>>> What for trap_exit has to be set to true when the clean up is needed? >>>> >>>> The place for doing some clean ups is under terminate function. >>>> We want to have trape_exit set to true when we need to handle 'EXIT' >>>> signals coming from linked processes to (for example) not crash >>>> gen_server. >>>> >>>> The only explanation for me is that the supervisor may crash too and >>>> send 'Exit' to gen_server as well... but this doesn't convince even >>>> myself. >>>> >>>> Has anyone some explanation for this advice ? >>>> >>>> >>>> Best, >>>> >>>> Dawid >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From ransomr@REDACTED Tue Mar 20 22:52:19 2012 From: ransomr@REDACTED (Ransom Richardson) Date: Tue, 20 Mar 2012 21:52:19 +0000 Subject: [erlang-questions] EUnit failure stack traces Message-ID: When running an eunit test that causes the code under test to thrown an exception there doesn't seem to be a stack trace, which can make it hard to debug. I wrote the following macros which I can put around my test code to output the stack trace, but I was wondering if there is an easier or better way to do this. Is there a reason that EUnit doesn't output the stacktrace on failures? -define(TEST_START, try ok). -define(TEST_END, ok catch Type:X -> io:format("~p~n", [{Type, X, erlang:get_stacktrace()}]), ?assert(unhandled_exception) end). some_eunit_test() -> ?TEST_START, ?assert(...), ?TEST_END. Thanks, Ransom -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Tue Mar 20 22:58:56 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 20 Mar 2012 17:58:56 -0400 Subject: [erlang-questions] conf. mgt. tools? Message-ID: <4F68FDA0.2020809@meetinghouse.net> Hi Folks, Out of curiosity, for those of you running production systems, what are you using (if anything) for deployment and configuration management? Any chef or puppet fans (or not)? Erlang/OTP specific tools? What about for managing your underlying platform? (Hopefully this will be about more than curiosity a few months down the road.) Thanks, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From ok@REDACTED Tue Mar 20 23:06:39 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 21 Mar 2012 11:06:39 +1300 Subject: [erlang-questions] math:pow(X, Y). In-Reply-To: References: <20120319182333.GA13317@corelatus.se> Message-ID: On 20/03/2012, at 11:10 PM, Michael Turner wrote: > "As these are the C library, the bugs are the same." > > I'm not sure what the IEEE floating point spec says about cases where > you've exceeded what can be represented in double precision floating > point, in computing an exponential. Raising to a power is not one of the operations defined by the IEEE standard. (Getting that right to half an ULP is a _lot_ harder than getting + and / right to have an ULP.) So the IEEE standard has nothing whatever to say about math:pow/2. One may argue by analogy that - EITHER it should return an appropriately signed Infinity - OR it should raise an exception (the Overflow exception). whichever is consistent with the behaviour of * . 1> 1.0e200 * 1.0e200. ** exception error: bad argument in an arithmetic expression in operator */2 called as 1.0e200 * 1.0e200 2> math:pow(1.0e200, 2). ** exception error: bad argument in an arithmetic expression in function math:pow/2 called as math:pow(1.0e200,2) The behaviour of math:pow/2 *is* consistent with the behaviour of simple multiplication, so no IEEE worries there. What _is_ questionable is that all the arguments are perfectly OK; the problem is *not* a bad argument but a floating point overflow, and so it should be labelled. Now full conformance to IEEE would mean that exception/infinity behaviour would be switchable at run time. Presumably each Erlang thread would have to have its own copy of the floating point state. At this point it starts getting complicated, and for typical Erlang applications, not _usefully_ so. From rstephenson@REDACTED Tue Mar 20 23:10:24 2012 From: rstephenson@REDACTED (Rob Stephenson) Date: Tue, 20 Mar 2012 15:10:24 -0700 Subject: [erlang-questions] Appup & Relup possible failures Message-ID: Hi, Our company has built an application using Erlang and we would like to support continuous deployment ( http://wiki.smartfrog.org/wiki/display/sf/Pattern+-+Continuous+Deployment) for this application. We're considering using the OTP appup and relup code updating features. We've reviewed the OTP documentation for release handling, and it mentions that upgrading or downgrading can fail under certain circumstances, but does not mention what those circumstances are. Can anyone shed any light on this? >From http://www.erlang.org/doc/design_principles/release_handling.html Section 11.1 "If the installation fails, the system may be rebooted. The old release version is then automatically used. " We've found information on other websites indicating that circular dependencies will cause application upgrade failures, but without a lot of details. Of the following use cases, which should be expected to work? - Circular dependencies between functional code modules - Circular dependencies between a functional code module and a residence module (a module that hosts the message loop of an application) - Circular dependencies between residence modules in the same node - Circular dependencies between residence modules in different nodes Is there a definitive list of what cases the application upgrade can fail? Thanks, Rob Stephenson -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Tue Mar 20 23:10:32 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Tue, 20 Mar 2012 17:10:32 -0500 Subject: [erlang-questions] conf. mgt. tools? In-Reply-To: <4F68FDA0.2020809@meetinghouse.net> References: <4F68FDA0.2020809@meetinghouse.net> Message-ID: I've become a big Chef fan, and it doesn't hurt that they use Erlang at Opscode ;) It's tool knife is great if you work with Amazon or Rackspace clouds ( http://wiki.opscode.com/display/chef/Launch+Cloud+Instances+with+Knife). One command and I can have a new VM up, configured and running. If not, its not too bad doing it "manually", http://erlware.wordpress.com/2011/09/29/centos6-chef-node-creation/ For deploying Erlang systems with Chef I use a similar approach to Boundary now, https://github.com/boundary/boundary_cookbooks/tree/master/erlang. I used to do things a bit more complicated, with package management to handle apps and releases, but now I just publish a self contained target system to a CDN that the Chef recipe grabs from, with whatever version it is set to install in its data bag, and boot it up. Tristan On Tue, Mar 20, 2012 at 4:58 PM, Miles Fidelman wrote: > Hi Folks, > > Out of curiosity, for those of you running production systems, what are > you using (if anything) for deployment and configuration management? Any > chef or puppet fans (or not)? Erlang/OTP specific tools? What about for > managing your underlying platform? (Hopefully this will be about more than > curiosity a few months down the road.) > > Thanks, > > Miles Fidelman > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tilman.holschuh@REDACTED Tue Mar 20 23:12:26 2012 From: tilman.holschuh@REDACTED (Tilman Holschuh) Date: Tue, 20 Mar 2012 15:12:26 -0700 Subject: [erlang-questions] conf. mgt. tools? In-Reply-To: <4F68FDA0.2020809@meetinghouse.net> References: <4F68FDA0.2020809@meetinghouse.net> Message-ID: <4F6900CA.70508@gmail.com> Hi Miles, We use Chef and self contained releases built with rebar. This way you don't have to install Erlang on the production machines. Our CI server (jenkins) builds the releases with the installed Erlang version. We deploy a package with a chef recipe and have it put everything in place (config files, etc). We don't do hot-upgrades. Most of the work is getting the self contained release (the riak rebar config on github helps to get an idea). The rest is typical chef shenanigan. Cheers - Tilman On 12-03-20 02:58 PM, Miles Fidelman wrote: > Hi Folks, > > Out of curiosity, for those of you running production systems, what are > you using (if anything) for deployment and configuration management? Any > chef or puppet fans (or not)? Erlang/OTP specific tools? What about for > managing your underlying platform? (Hopefully this will be about more > than curiosity a few months down the road.) > > Thanks, > > Miles Fidelman > From ok@REDACTED Tue Mar 20 23:17:05 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 21 Mar 2012 11:17:05 +1300 Subject: [erlang-questions] math:pow(X, Y). In-Reply-To: <48BCC786-151C-4835-9EA2-DB519871601F@gmail.com> References: <37C435A0-E191-4583-852F-24E495F4BB56@cs.otago.ac.nz> <48BCC786-151C-4835-9EA2-DB519871601F@gmail.com> Message-ID: <211E9A10-13FB-4B75-8243-92A51FDDA2B4@cs.otago.ac.nz> On 21/03/2012, at 4:07 AM, Aleksandr Vinokurov wrote: > Richard, what the point of writing two equal funs? rtpi and rtpf? Really. There was originally one function. However, they have different *types*, and my hope was that HiPE would be able to exploit this to generate code, at least for rtpf, that had no run-time tests for "is this a float", and perhaps might even pass the arguments around unboxed. So yes, *really*, because thanks to the -specs, they don't have exactly the same semantics. From tristan.sloughter@REDACTED Tue Mar 20 23:17:12 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Tue, 20 Mar 2012 17:17:12 -0500 Subject: [erlang-questions] conf. mgt. tools? In-Reply-To: References: <4F68FDA0.2020809@meetinghouse.net> Message-ID: I guess I should have mentioned I build the target systems with sinan ( http://erlware.github.com/sinan/) 'sinan dist' with {include_erts, true}. set in sinan.config builds a tar.gz with erts included. I've found it to be the easiest tool, with the least hassle to handle projects of many applications and building a release of all those applications and their dependencies by default. Tristan On Tue, Mar 20, 2012 at 5:10 PM, Tristan Sloughter < tristan.sloughter@REDACTED> wrote: > I've become a big Chef fan, and it doesn't hurt that they use Erlang at > Opscode ;) > > It's tool knife is great if you work with Amazon or Rackspace clouds ( > http://wiki.opscode.com/display/chef/Launch+Cloud+Instances+with+Knife). > One command and I can have a new VM up, configured and running. If not, its > not too bad doing it "manually", > http://erlware.wordpress.com/2011/09/29/centos6-chef-node-creation/ > > For deploying Erlang systems with Chef I use a similar approach to > Boundary now, > https://github.com/boundary/boundary_cookbooks/tree/master/erlang. I used > to do things a bit more complicated, with package management to handle apps > and releases, but now I just publish a self contained target system to a > CDN that the Chef recipe grabs from, with whatever version it is set to > install in its data bag, and boot it up. > > Tristan > > On Tue, Mar 20, 2012 at 4:58 PM, Miles Fidelman < > mfidelman@REDACTED> wrote: > >> Hi Folks, >> >> Out of curiosity, for those of you running production systems, what are >> you using (if anything) for deployment and configuration management? Any >> chef or puppet fans (or not)? Erlang/OTP specific tools? What about for >> managing your underlying platform? (Hopefully this will be about more than >> curiosity a few months down the road.) >> >> Thanks, >> >> Miles Fidelman >> >> -- >> In theory, there is no difference between theory and practice. >> In practice, there is. .... Yogi Berra >> >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wmknapik@REDACTED Wed Mar 21 02:38:25 2012 From: wmknapik@REDACTED (Wojciech Knapik) Date: Wed, 21 Mar 2012 02:38:25 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: References: Message-ID: <20120321013825.GA3889@h82-143-167-72-static.e-wro.net.pl> On Mon, Mar 19, 2012 at 05:38:37PM +0100, Ulf Wiger wrote: > I much prefer to have a slightly more strict setup, where you e.g. > designate one or two "master" machines that keep a persistent copy of > the database. The other machines can start up with the env variable: > extra_db_nodes : MasterNodes, and access the database without even > having their own copy. Let's assume for a moment, that full replication is indeed unnecessary. We still want to run identical nodes. Or at least the static code delivered to all nodes should be the same and runtime decisions should designate a master node. That is not trivial. Would we have to "manually" implement an election algorithm, or are there any Erlang mechanisms that would make this simpler ? If "manual" is the answer, then would you suggest any specific algorithm ? > Why does the data need to be fully replicated? Nodes starting with > extra_db_nodes as above enjoy full distribution transparency. If very > rapid read response is required, you can add a ram-based replica on > the fly with the function mnesia:add_table_copy_type(Table, Node, > CopyType). > > Update cost will increase noticeably with every replica you add, so in > many cases, it may be counter-productive to use full replication. For > fault tolerance, having two persistent copies of the data goes a long > way. The project's domain is completely dominated by client-server solutions where the central server is a weak point in terms of fault tolerance, performance and scalability. A completely explicit configuration on the server is also very rigid - it doesn't allow the system to adapt to changing conditions and forbids a number of features that could exist in a fully distributed setup. So we aim to push the distribution aspect as far, as it makes sense, to fix these issues. >From what everyone here is saying, full mnesia replication will give us fault tolerance, but at a severe cost to scalability and performance. On the other hand we want to avoid the necessity for the user to explicitly specify nodes that would be special in any way. In fact the user can't know that in advance, because nodes can go offline, or become overloaded, or whatever. And last, but not least, in the spirit of declarativeness - the user shouldn't be bothered with these details - it's up to the system to figure it out. As far as what the database would be used for - it would hold the state of the system as a whole and provide persistence even if all nodes go down (with disc_copies). It's hard to predict a lot of details at this point. I know I'm being a bit cryptic, but I can't really talk about the project without the blessing of the other team members... So anyway - I'm guessing the problems I described are not uncommon among Erlang programmers. How do you usually handle them ? Any feedback would be very much appreciated. br, WK From wmknapik@REDACTED Wed Mar 21 02:56:45 2012 From: wmknapik@REDACTED (Wojciech Knapik) Date: Wed, 21 Mar 2012 02:56:45 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: <1339302F-97CB-45F4-B002-E2D0E635EDF0@feuerlabs.com> References: <1339302F-97CB-45F4-B002-E2D0E635EDF0@feuerlabs.com> Message-ID: <20120321015645.GA7548@h82-143-167-72-static.e-wro.net.pl> On Mon, Mar 19, 2012 at 06:07:16PM +0100, Ulf Wiger wrote: > - Mnesia uses a simple broadcast algorithm for locking, and 2-phase > commit protocols, so running transactions across a large number of > replicas will become increasingly expensive. Ironically, I've often > heard people become frustrated with mnesia's performance when running > fully replicated, and instead switch to PostgreSQL, which until very > recently, couldn't replicate at all. This suggest that replication > wasn't a requirement in the first place (for them, I don't know enough > about your requirements to know if it applies in this case). > *SNIP* > > I assume the SQL database is not running in fully replicated mode? If > so, perhaps you have a layered architecture after all? In our case the mnesia database will only hold data relevant to the system's operation at any given moment. The loss of this data would likely be a severe problem. The SQL database will provide information that's not crucial for the system to operate, but if it goes down, some of the system's functionality will be limited. We're still planning to keep it replicated though. br, WK From wmknapik@REDACTED Wed Mar 21 03:13:16 2012 From: wmknapik@REDACTED (Wojciech Knapik) Date: Wed, 21 Mar 2012 03:13:16 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: <61A30020-4D60-4825-92CC-917A0DDED075@feuerlabs.com> References: <61A30020-4D60-4825-92CC-917A0DDED075@feuerlabs.com> Message-ID: <20120321021316.GB7548@h82-143-167-72-static.e-wro.net.pl> On Mon, Mar 19, 2012 at 06:45:09PM +0100, Ulf Wiger wrote: > Not sure if you've seen the (admittedly short) tutorials on TrapExit: > > http://www.trapexit.org/Distributing_a_Mnesia_schema > > A minor comment: you can do this instead: > > mnesia:start([{extra_db_nodes, Nodes}]) > > If you want the nodes to start from a boot script, you can either roll > your own the old-fashioned way: > > http://www.trapexit.org/OTP_Release_Handling_Tutorial > > or use rebar and Reltool: > > http://www.metabrew.com/article/erlang-rebar-tutorial-generating-releases-upgrades > > In this case, it's better to put the extra_db_nodes setting in the > sys.config file: > > [?, {mnesia, [ {extra_db_nodes, [node@REDACTED, node@REDACTED]} ]} > > Then, calling mnesia:start() will suffice, given that you've started > the node in the right way. Thanks for the hints, that's good to know, especially about Rebar. WK From alisdairsullivan@REDACTED Wed Mar 21 03:52:25 2012 From: alisdairsullivan@REDACTED (alisdair sullivan) Date: Tue, 20 Mar 2012 19:52:25 -0700 Subject: [erlang-questions] [ANNOUNCE] JSX 1.0 and 1.1rc In-Reply-To: References: Message-ID: <98BF8D67D2394D42A19341C94909253C@yahoo.ca> i finally cleaned up and pushed jsx 1.0 earlier this month, it's here: https://github.com/talentdeficit/jsx what i'd really like though is people to try out my 1.1rc and report any problems. i think the tests are fairly comprehensive, but it's possible i missed some things. it mostly adds more options for more lenient parsing (comments, single quoted keys and strings) and better unifies the interfaces between the encoder and the decoder. i also rewrote the README, and it probably needs proofreading replies and comments should probably go to the github issues page, or track me down on irc. thanks! alisdair -------------- next part -------------- An HTML attachment was scrubbed... URL: From aleksandr.vin@REDACTED Wed Mar 21 04:59:41 2012 From: aleksandr.vin@REDACTED (Aleksandr Vinokurov) Date: Wed, 21 Mar 2012 07:59:41 +0400 Subject: [erlang-questions] math:pow(X, Y). In-Reply-To: <211E9A10-13FB-4B75-8243-92A51FDDA2B4@cs.otago.ac.nz> References: <37C435A0-E191-4583-852F-24E495F4BB56@cs.otago.ac.nz> <48BCC786-151C-4835-9EA2-DB519871601F@gmail.com> <211E9A10-13FB-4B75-8243-92A51FDDA2B4@cs.otago.ac.nz> Message-ID: <4FF1A02C-A9FD-4420-847C-20791EFE649D@gmail.com> Got it. Thank you Richard. 21.03.2012, ? 2:17, "Richard O'Keefe" ???????(?): > > On 21/03/2012, at 4:07 AM, Aleksandr Vinokurov wrote: > >> Richard, what the point of writing two equal funs? rtpi and rtpf? Really. > > There was originally one function. However, they have different > *types*, and my hope was that HiPE would be able to exploit this > to generate code, at least for rtpf, that had no run-time tests > for "is this a float", and perhaps might even pass the arguments > around unboxed. > > So yes, *really*, because thanks to the -specs, they don't have > exactly the same semantics. > From aleksandr.vin@REDACTED Wed Mar 21 05:03:11 2012 From: aleksandr.vin@REDACTED (Aleksandr Vinokurov) Date: Wed, 21 Mar 2012 08:03:11 +0400 Subject: [erlang-questions] Different behaviour of the erl shell and compile:file In-Reply-To: References: Message-ID: <1D0A8F0B-2A70-4781-90D2-71A9C8930C69@gmail.com> Any feedback so far? I'll do epp.erl patch and new erlc option. 19.03.2012, ? 12:19, Aleksandr Vinokurov ???????(?): > > > Hello all, > > I'm investigating Unicode support in Erlang. And have found the difference in behaviour of erl shell and compile:file. > > The erl shell: > > 1. We configure our terminal emulator to use UTF-8 > 2. We set io:setopts({encoding,unicode}). > 3. Then in the prompt we enter a Unicode string and see the result -- a list of Unicode codepoints: > (emacs@REDACTED)103> "?????". > [1040,1041,1042,1043,1044] > > > The compile:file: > > 1. We save a file foo in UTF-8 with a function that return a Unicode string bar() -> "?????". > 2. Now we open an erl shell and run compile:file(foo). > 3. Finally running foo:bar() we have a list of UTF-8 bytes. > (emacs@REDACTED)111> foo:bar(). > [208,144,208,145,208,146,208,147,208,148] > > > I dug the compile:file/1 and found epp:server/4 where the file is opened with file:open(Name, [read]) and where encoding of the returned IoDevice is not configured. > > Why not to use one of the Unicode recipes from the Unicode usage doc (lib/stdlib-1.18/doc/html/unicode_usage.html) to set io:setopts(F,[{encoding,Type}]) or add an encoding option to the compile:file/2 and erlc? > > What would be your suggestions, honorable all? > > -- > ????????? ????????? > +7 (921) 982-21-43 > @aleksandrvin > From ulf@REDACTED Wed Mar 21 07:33:22 2012 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 21 Mar 2012 07:33:22 +0100 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: <20120321013825.GA3889@h82-143-167-72-static.e-wro.net.pl> References: <20120321013825.GA3889@h82-143-167-72-static.e-wro.net.pl> Message-ID: <4AF16C06-60D5-47DF-B69C-CBB6E5CE1FA2@feuerlabs.com> On 21 Mar 2012, at 02:38, Wojciech Knapik wrote: > On Mon, Mar 19, 2012 at 05:38:37PM +0100, Ulf Wiger wrote: > >> I much prefer to have a slightly more strict setup, where you e.g. >> designate one or two "master" machines that keep a persistent copy of >> the database. The other machines can start up with the env variable: >> extra_db_nodes : MasterNodes, and access the database without even >> having their own copy. > > Let's assume for a moment, that full replication is indeed unnecessary. > We still want to run identical nodes. Or at least the static code > delivered to all nodes should be the same and runtime decisions should > designate a master node. That is not trivial. Would we have to > "manually" implement an election algorithm, or are there any Erlang > mechanisms that would make this simpler ? If "manual" is the answer, > then would you suggest any specific algorithm ? A simple way to create a critical section is to use global:trans http://www.erlang.org/doc/man/global.html#trans-2 In this case, what you really need to ensure that the initial schema is created only once, on one node, and the other nodes attach themselves to it. > From what everyone here is saying, full mnesia replication will give us > fault tolerance, but at a severe cost to scalability and performance. Well, I don't know about severe? :) In many ways, mnesia is one of the speediest databases you can find for Erlang applications. In terms of scalability, the practical limits for a fully replicated mnesia database seem to be somewhere around 10 nodes, at which point it becomes quite problematic. But some very performant systems have been written on top of mnesia. You can do things like sharing in mnesia, and you can come up with various different ways to distribute your data. But if your access patterns are simple key-value storage patterns without need for transactions, you may be better off going with e.g. riak. Riak, on the other hand, doesn't scale *down* as well as mnesia, and generally wants to run in a node of its own. TANSTAAFL. ;-) BR, Ulf W -------------- next part -------------- An HTML attachment was scrubbed... URL: From aleksandr.vin@REDACTED Wed Mar 21 08:05:59 2012 From: aleksandr.vin@REDACTED (Aleksandr Vinokurov) Date: Wed, 21 Mar 2012 11:05:59 +0400 Subject: [erlang-questions] Please correct my leeway on my first Erlang/OTP application Message-ID: Hello all, I'm quite a newbee in Erlang/OTP and will be very appreciated if you can point any wrong style or misunderstanding of the OTP in my first Erlang/OTP application. I've put it here https://github.com/aleksandr-vin/clustmea and will be glad to here any of your feedback. With best wishes, Aleksandr Vinokurov @aleksandrvin -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Wed Mar 21 08:20:17 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Wed, 21 Mar 2012 09:20:17 +0200 Subject: [erlang-questions] [ANNOUNCE] JSX 1.0 and 1.1rc In-Reply-To: <98BF8D67D2394D42A19341C94909253C@yahoo.ca> References: <98BF8D67D2394D42A19341C94909253C@yahoo.ca> Message-ID: <-451521627403557216@unknownmsgid> Hello, Just spend five minute with the project. It looks pretty interesting, unfortunately I have not heard about it earlier enought. At least, I liked your library and would try to replace a mochiweb JSON with yours. I'll try to run a performance comparison your code and mochiweb in my test env. Best Regards, Dmitry >-|-|-*> On 21.3.2012, at 4.53, alisdair sullivan wrote: > i finally cleaned up and pushed jsx 1.0 earlier this month, it's here: > > https://github.com/talentdeficit/jsx > > what i'd really like though is people to try out my 1.1rc and report any problems. i think the tests are fairly comprehensive, but it's possible i missed some things. it mostly adds more options for more lenient parsing (comments, single quoted keys and strings) and better unifies the interfaces between the encoder and the decoder. i also rewrote the README, and it probably needs proofreading > > replies and comments should probably go to the github issues page, or track me down on irc. thanks! > > alisdair > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lenartlad@REDACTED Wed Mar 21 12:15:29 2012 From: lenartlad@REDACTED (Ladislav Lenart) Date: Wed, 21 Mar 2012 12:15:29 +0100 Subject: [erlang-questions] Please correct my leeway on my first Erlang/OTP application In-Reply-To: References: Message-ID: <4F69B851.7000301@volny.cz> Hello. Disclaimer: I am by no means an Erlang expert, but here you go... I haven't study your application from the architectural POV (i.e. process structure and what is done where) so I have no remarks to the overall semantics. I only looked at a couple of your modules. My mostly stylish and syntactic notes: * It is an OTP convention to split files to the following folders: * /src for *.erl and internal *.hrl, * /include for public *.hrl; i.e. those used in -include_lib(), * /priv for various data files and e.g. C sources and * /ebin for compiled beam code. I suspect that many people adhere to these (or very similar ones). * It is also desirable to prefix all your modules to lower conflict ratio with modules in other apps. Use e.g. full 'clustmea_' or shorter 'clea_' or some such. Usage of the name gen_uploader is a classy antipattern example. The name suggests that the module belongs to the basic OTP libs which is not the case. * I am not sure if it is feasible, but you should at least consider adding API functions to clustmea module so the user does not have to know unnecessary internal details of your code (at least not upfront). This would also allowed you to refactor it (change the process structure) later. These API functions would be simple wrappers calling the right function(s) of the right module(s) with the right arguments. Everything else would be private to your app. * Overall I like your style. It is idiomatic, consistent and thus predictable and reads nicely :-) * However, I wouldn't use records for messages encapsulated behind a functional API. So instead of e.g. -record(stop, {reason}) I would simply write {stop, Reason} But this is just nitpicking (though up until now, I haven't seen this style). * I wouldn't use records such as this -record(state, {tasks}) I would simply use the value of tasks as the entire state and refactor the module later, if/when the need arises. But again, this is just me nitpicking. * You can pattern match in the function clause head. Instead of handle_cast(run, State) -> TaskConf = State#state.config, UploaderConf = get_uploader_config(TaskConf), IterationPolicy = get_policy(iteration, TaskConf, _DefaultPolicy = soft), Module = State#state.module, run(Module, IterationPolicy, UploaderConf), {noreply, State}; I would write handle_cast(run, #state{config=TaskConf, module=Module} = State) -> UploaderConf = get_uploader_config(TaskConf), IterationPolicy = get_policy(iteration, TaskConf, _DefaultPolicy = soft), run(Module, IterationPolicy, UploaderConf), {noreply, State}; * I don't think the following adds any value handle_call(_Request, _From, State) -> Reply = ok, {reply, Reply, State}. so I would write code like this directly handle_call(_Request, _From, State) -> {reply, ok, State}. * I would rename is_policy/1 to something like check_policy/1 which I think better conveys the possibility of throwing an error: get_policy(Name, TaskConf, DefaultPolicy) -> check_policy(Name), case lists:keyfind(Name, 1, TaskConf) of false -> DefaultPolicy; {Name, Policy} -> Policy end. check_policy(iteration) -> ok. * I have seen several occasions where you access a record as a tuple, e.g. {state, Tasks1} = State, vs. #state{tasks=Tasks} = State, I am sure this is just an omission but it may hurt you badly one day nonetheless ;-) HTH, Ladislav Lenart On 21.3.2012 08:05, Aleksandr Vinokurov wrote: > > > Hello all, > > I'm quite a newbee in Erlang/OTP and will be very appreciated if you can point any wrong style or misunderstanding of the OTP in my first Erlang/OTP application. > > I've put it here https://github.com/aleksandr-vin/clustmea and will be glad to here any of your feedback. > > With best wishes, > Aleksandr Vinokurov > @aleksandrvin > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bartek@REDACTED Wed Mar 21 12:32:51 2012 From: bartek@REDACTED (bartek) Date: Wed, 21 Mar 2012 12:32:51 +0100 Subject: [erlang-questions] order of processes in supervisor Message-ID: <20120321123251.09db7005@gorny.edu.pl> Hi Is there a way to configure supervisor so that it makes sure its child processes are started in due order - meaning child B starts only after A started successfully and completed its initialization? I have two processes of which one depends on the other, so I need to make sure that B is running only when A is running. I know can achieve it by monitoring, but I want them both supervised, and when they are being started by a supervisor B goes into a cyclic restart until A gets up, so this is not a clean solution. What is the right way to do it? Start B's supervisor from process A? Or use dynamic child creation somehow? But then a child process would have to get reference to its supervisor and call it after it is done - would it be kosher from OTP point of view? Thanks Bartek G?rny -- Je?li boli ci? gard?o, ciesz si? ?e nie jeste? ?yraf? (zauwa?one w aptece) From ponton@REDACTED Wed Mar 21 12:37:16 2012 From: ponton@REDACTED (Tomasz Maciejewski) Date: Wed, 21 Mar 2012 12:37:16 +0100 Subject: [erlang-questions] order of processes in supervisor In-Reply-To: <20120321123251.09db7005@gorny.edu.pl> References: <20120321123251.09db7005@gorny.edu.pl> Message-ID: W dniu 21 marca 2012 12:32 u?ytkownik bartek napisa?: > Hi > > Is there a way to configure supervisor so that it makes sure its child > processes are started in due order - meaning child B starts only after > A started successfully and completed its initialization? This is the default behavour: "The children of a supervisor is defined as a list of child specifications. When the supervisor is started, the child processes are started in order from left to right according to this list. When the supervisor terminates, it first terminates its child processes in reversed start order, from right to left." (from http://www.erlang.org/doc/man/supervisor.html) -- Tomasz Maciejewski From torben.lehoff@REDACTED Wed Mar 21 12:48:38 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Wed, 21 Mar 2012 12:48:38 +0100 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server Message-ID: <4F69C016.6080800@gmail.com> Hi, I was sitting with a co-worker - who is proficient in OCaml and lots of other stuff and he is now learning Erlang - and then he comes up with a good question on the amount of boilerplate code when implementing a gen_server. From the API functions you call out to gen_server:call/2 and on top of that you need to implement a handle_call/3 function clause that matches whatever format you used for wrapping the incoming message. The question was: why don't you use an IDL (Interface Definition Language) approach instead of writing all that code that has to match up in order to work? So the idea would be to write a my_server.idl file and from that generate two files: * my_server_stub.erl o This holds all the public APIs that clients may call. o It calls my_server_skeleton as dictated by the IDL file. * my_server_skeleton.erl o gets all all the calls from the my_server_stub and calls my_server.erl The job then is to implement my_server.erl according to the type specs derived from the IDL file. Has anyone looked into this before? I have tried searching for it, but since IDL is so tightly coupled with CORBA I didn't really find anything but the ic application from the Erlang/OTP distribution. Cheers, Torben -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias@REDACTED Wed Mar 21 12:52:49 2012 From: matthias@REDACTED (Matthias Lang) Date: Wed, 21 Mar 2012 12:52:49 +0100 Subject: [erlang-questions] math:pow(X, Y). In-Reply-To: References: <20120319182333.GA13317@corelatus.se> Message-ID: <20120321115249.GA3616@corelatus.se> On Wednesday, March 21, Richard O'Keefe wrote: > 1> 1.0e200 * 1.0e200. > ** exception error: bad argument in an arithmetic expression > in operator */2 > called as 1.0e200 * 1.0e200 > What _is_ questionable is that all the arguments are perfectly OK; the > problem is *not* a bad argument but a floating point overflow, > and so it should be labelled. It's "just" the error message pretty-printer which makes it look like a bad argument. The real error is actually a 'badarith': 1> catch 1.0e200 * 1.0e200. {'EXIT',{badarith,[{erlang,'*',[1.0e200,1.0e200],[]}, Matt From bartek@REDACTED Wed Mar 21 12:55:37 2012 From: bartek@REDACTED (bartek) Date: Wed, 21 Mar 2012 12:55:37 +0100 Subject: [erlang-questions] order of processes in supervisor In-Reply-To: <20120321123251.09db7005@gorny.edu.pl> References: <20120321123251.09db7005@gorny.edu.pl> Message-ID: <20120321125537.3ced7270@gorny.edu.pl> On Wed, 21 Mar 2012 12:32:51 +0100 bartek wrote: > Hi > > Is there a way to configure supervisor so that it makes sure its child > processes are started in due order - meaning child B starts only after > A started successfully and completed its initialization? > > I have two processes of which one depends on the other, so I need to > make sure that B is running only when A is running. I know can > achieve it by monitoring, but I want them both supervised, and when > they are being started by a supervisor B goes into a cyclic restart > until A gets up, so this is not a clean solution. What is the right > way to do it? Start B's supervisor from process A? Or use dynamic > child creation somehow? But then a child process would have to get > reference to its supervisor and call it after it is done - would it > be kosher from OTP point of view? > > Thanks > Bartek G?rny > I read this one :) But the supervisor seems to trigger their startup but not wait for it to complete. At least this is what I observed - my child list is [A, B], and B in its init does "monitor(A)", and it crashes. B. -- Co tu pocz??, co tu pocz??, trudna rada / Cz?owiek przesta? si? ju? dobrze zapowiada? / Ale z drugiej strony mo?na cieszy? si? / ?e si? r?wnie? przesta? zapowiada? ?le... (Starsi Panowie) From ponton@REDACTED Wed Mar 21 13:05:00 2012 From: ponton@REDACTED (Tomasz Maciejewski) Date: Wed, 21 Mar 2012 13:05:00 +0100 Subject: [erlang-questions] order of processes in supervisor In-Reply-To: <20120321125537.3ced7270@gorny.edu.pl> References: <20120321123251.09db7005@gorny.edu.pl> <20120321125537.3ced7270@gorny.edu.pl> Message-ID: W dniu 21 marca 2012 12:55 u?ytkownik bartek napisa?: > I read this one :) But the supervisor seems to trigger their startup > but not wait for it to complete. At least this is what I observed - my > child list is [A, B], and B in its init does "monitor(A)", and it > crashes. I'm pretty sure it waits (just have had a quick look at supervisor.erl). What reason does it crash with? Maybe you call monitor(pid()) instead of monitor(process, pid())? -- Tomasz Maciejewski From lenartlad@REDACTED Wed Mar 21 13:09:00 2012 From: lenartlad@REDACTED (Ladislav Lenart) Date: Wed, 21 Mar 2012 13:09:00 +0100 Subject: [erlang-questions] order of processes in supervisor In-Reply-To: <20120321125537.3ced7270@gorny.edu.pl> References: <20120321123251.09db7005@gorny.edu.pl> <20120321125537.3ced7270@gorny.edu.pl> Message-ID: <4F69C4DC.6010106@volny.cz> Can you provide us with a stack trace of the crash? Chances are it is crashing for a different cause than you think. Ladislav Lenart On 21.3.2012 12:55, bartek wrote: > On Wed, 21 Mar 2012 12:32:51 +0100 > bartek wrote: > >> Hi >> >> Is there a way to configure supervisor so that it makes sure its child >> processes are started in due order - meaning child B starts only after >> A started successfully and completed its initialization? >> >> I have two processes of which one depends on the other, so I need to >> make sure that B is running only when A is running. I know can >> achieve it by monitoring, but I want them both supervised, and when >> they are being started by a supervisor B goes into a cyclic restart >> until A gets up, so this is not a clean solution. What is the right >> way to do it? Start B's supervisor from process A? Or use dynamic >> child creation somehow? But then a child process would have to get >> reference to its supervisor and call it after it is done - would it >> be kosher from OTP point of view? >> >> Thanks >> Bartek G?rny >> > > > I read this one :) But the supervisor seems to trigger their startup > but not wait for it to complete. At least this is what I observed - my > child list is [A, B], and B in its init does "monitor(A)", and it > crashes. > > B. > > From bartek@REDACTED Wed Mar 21 13:19:32 2012 From: bartek@REDACTED (bartek) Date: Wed, 21 Mar 2012 13:19:32 +0100 Subject: [erlang-questions] order of processes in supervisor In-Reply-To: <4F69C4DC.6010106@volny.cz> References: <20120321123251.09db7005@gorny.edu.pl> <20120321125537.3ced7270@gorny.edu.pl> <4F69C4DC.6010106@volny.cz> Message-ID: <20120321131932.10c2a638@gorny.edu.pl> On Wed, 21 Mar 2012 13:09:00 +0100 Ladislav Lenart wrote: > Can you provide us with a stack trace of the crash? Chances > are it is crashing for a different cause than you think. > Ehm - indeed it was... I changed the ordering and it crashed for another reason, and it mislead me. Sorry about that, thanks for help. Bartek > Ladislav Lenart > > > On 21.3.2012 12:55, bartek wrote: > > On Wed, 21 Mar 2012 12:32:51 +0100 > > bartek wrote: > > > >> Hi > >> > >> Is there a way to configure supervisor so that it makes sure its > >> child processes are started in due order - meaning child B starts > >> only after A started successfully and completed its initialization? > >> > >> I have two processes of which one depends on the other, so I need > >> to make sure that B is running only when A is running. I know can > >> achieve it by monitoring, but I want them both supervised, and when > >> they are being started by a supervisor B goes into a cyclic restart > >> until A gets up, so this is not a clean solution. What is the right > >> way to do it? Start B's supervisor from process A? Or use dynamic > >> child creation somehow? But then a child process would have to get > >> reference to its supervisor and call it after it is done - would it > >> be kosher from OTP point of view? > >> > >> Thanks > >> Bartek G?rny > >> > > > > > > I read this one :) But the supervisor seems to trigger their startup > > but not wait for it to complete. At least this is what I observed - > > my child list is [A, B], and B in its init does "monitor(A)", and it > > crashes. > > > > B. > > > > > > -- Do you know, where your towel is? (re: Hitchhikers Guide to the Galaxy) From watson.timothy@REDACTED Wed Mar 21 13:22:16 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 21 Mar 2012 12:22:16 +0000 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: <4F69C016.6080800@gmail.com> References: <4F69C016.6080800@gmail.com> Message-ID: <448573B5-136B-43A2-80E2-FA1B07D1B045@gmail.com> That sounds horribly complicated and I always hated corba. What problem exactly are you trying to solve here? Is it (a) the API function simply passes its inputs verbatim to gen_server:call/2 and therefore you don't want to have to bother writing it by hand (b) the API function is transforming its inputs before passing them to the server and you'd prefer to write this declaratively (c) some other issue? Replacing an API function in the same module as a gen_* callback function seems to me to be several orders of magnitude simpler than an IDL that magically generates 2 modules that eventually end up calling your 'backing' module somehow. Here's what I'd consider doing to remove gen_server boilerplate. 1. write a parse_transform that generates and exports default implementations for any missing callbacks (so that cast/info/terminate/etc are optional) 2. write a parse_transform that generates API functions (with some optional parameter transformation/mapping support) based on either some -spec records or something similar You can see an example of something similar to (2) in https://github.com/hyperthunk/delegate which uses https://github.com/hyperthunk/annotations. All in all - and I speak as both an OCaml and Erlang programmer here - using compile/build time code generation feels much more natural that trying to shoehorn an IDL into the problem space. On 21 Mar 2012, at 11:48, Torben Hoffmann wrote: > Hi, > > I was sitting with a co-worker - who is proficient in OCaml and lots of other stuff and he is now learning Erlang - and then he comes up with a good question on the amount of boilerplate code when implementing a gen_server. > > From the API functions you call out to gen_server:call/2 and on top of that you need to implement a handle_call/3 function clause that matches whatever format you used for wrapping the incoming message. > > The question was: why don't you use an IDL (Interface Definition Language) approach instead of writing all that code that has to match up in order to work? > > So the idea would be to write a my_server.idl file and from that generate two files: > my_server_stub.erl > This holds all the public APIs that clients may call. > It calls my_server_skeleton as dictated by the IDL file. > my_server_skeleton.erl > gets all all the calls from the my_server_stub and calls my_server.erl > The job then is to implement my_server.erl according to the type specs derived from the IDL file. > Has anyone looked into this before? > I have tried searching for it, but since IDL is so tightly coupled with CORBA I didn't really find anything but the ic application from the Erlang/OTP distribution. > Cheers, > Torben > -- > http://www.linkedin.com/in/torbenhoffmann > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lenartlad@REDACTED Wed Mar 21 13:25:18 2012 From: lenartlad@REDACTED (Ladislav Lenart) Date: Wed, 21 Mar 2012 13:25:18 +0100 Subject: [erlang-questions] order of processes in supervisor In-Reply-To: <20120321131932.10c2a638@gorny.edu.pl> References: <20120321123251.09db7005@gorny.edu.pl> <20120321125537.3ced7270@gorny.edu.pl> <4F69C4DC.6010106@volny.cz> <20120321131932.10c2a638@gorny.edu.pl> Message-ID: <4F69C8AE.40304@volny.cz> You're welcome! Happy hacking, Ladislav Lenart On 21.3.2012 13:19, bartek wrote: > On Wed, 21 Mar 2012 13:09:00 +0100 > Ladislav Lenart wrote: > >> Can you provide us with a stack trace of the crash? Chances >> are it is crashing for a different cause than you think. >> > > Ehm - indeed it was... I changed the ordering and it crashed for > another reason, and it mislead me. Sorry about that, thanks for help. > > Bartek > >> Ladislav Lenart >> >> >> On 21.3.2012 12:55, bartek wrote: >>> On Wed, 21 Mar 2012 12:32:51 +0100 >>> bartek wrote: >>> >>>> Hi >>>> >>>> Is there a way to configure supervisor so that it makes sure its >>>> child processes are started in due order - meaning child B starts >>>> only after A started successfully and completed its initialization? >>>> >>>> I have two processes of which one depends on the other, so I need >>>> to make sure that B is running only when A is running. I know can >>>> achieve it by monitoring, but I want them both supervised, and when >>>> they are being started by a supervisor B goes into a cyclic restart >>>> until A gets up, so this is not a clean solution. What is the right >>>> way to do it? Start B's supervisor from process A? Or use dynamic >>>> child creation somehow? But then a child process would have to get >>>> reference to its supervisor and call it after it is done - would it >>>> be kosher from OTP point of view? >>>> >>>> Thanks >>>> Bartek G?rny >>>> >>> >>> >>> I read this one :) But the supervisor seems to trigger their startup >>> but not wait for it to complete. At least this is what I observed - >>> my child list is [A, B], and B in its init does "monitor(A)", and it >>> crashes. >>> >>> B. >>> >>> >> >> > > > From tristan.sloughter@REDACTED Wed Mar 21 15:38:22 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Wed, 21 Mar 2012 09:38:22 -0500 Subject: [erlang-questions] [ANNOUNCE] JSX 1.0 and 1.1rc In-Reply-To: <-451521627403557216@unknownmsgid> References: <98BF8D67D2394D42A19341C94909253C@yahoo.ca> <-451521627403557216@unknownmsgid> Message-ID: > > I'll try to run a performance comparison your code and mochiweb in my test > env. > It would be great if you included jiffy (https://github.com/davisp/jiffy) too and please share if you do :) Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Wed Mar 21 15:43:51 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Wed, 21 Mar 2012 09:43:51 -0500 Subject: [erlang-questions] AWS Clients Message-ID: I'm looking around for Erlang clients for Amazon services and there are a number serving the same purpose, so I wanted to check if anyone had experience with them and opinions. I'm mostly, at this time, interested in SQS and Dynamo. These are the ones I've found so far: Dynamo: https://github.com/wagerlabs/ddb https://github.com/SemanticSugar/dinerl SQS and others: https://github.com/x6j8x/erlaws/tree https://github.com/bernied/erlawys https://github.com/mfine/aws_messaging Thanks! Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Wed Mar 21 15:46:16 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Wed, 21 Mar 2012 09:46:16 -0500 Subject: [erlang-questions] AWS Clients In-Reply-To: References: Message-ID: Oh, and erlcloud: https://github.com/bwbuchanan/erlcloud On Wed, Mar 21, 2012 at 9:43 AM, Tristan Sloughter < tristan.sloughter@REDACTED> wrote: > I'm looking around for Erlang clients for Amazon services and there are a > number serving the same purpose, so I wanted to check if anyone had > experience with them and opinions. I'm mostly, at this time, interested in > SQS and Dynamo. These are the ones I've found so far: > > Dynamo: > > https://github.com/wagerlabs/ddb > https://github.com/SemanticSugar/dinerl > > SQS and others: > > https://github.com/x6j8x/erlaws/tree > https://github.com/bernied/erlawys > https://github.com/mfine/aws_messaging > > Thanks! > Tristan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve@REDACTED Wed Mar 21 15:46:23 2012 From: steve@REDACTED (Steve Strong) Date: Wed, 21 Mar 2012 15:46:23 +0100 Subject: [erlang-questions] AWS Clients In-Reply-To: References: Message-ID: <004C29264E7244BDAC8DFB3AFD55A4D9@srstrong.com> We're using erlawys extensively - I think it's fair to say that given a week or so of spare time, we'd probably rip it out - some of the others that have come more recently look considerably better and offer access to far more Amazon services. One that caught my eye a month or two back which isn't in your list is https://github.com/gleber/erlcloud Cheers, Steve -- Steve Strong @srstrong Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Wednesday, 21 March 2012 at 15:43, Tristan Sloughter wrote: > I'm looking around for Erlang clients for Amazon services and there are a number serving the same purpose, so I wanted to check if anyone had experience with them and opinions. I'm mostly, at this time, interested in SQS and Dynamo. These are the ones I've found so far: > > Dynamo: > > https://github.com/wagerlabs/ddb > https://github.com/SemanticSugar/dinerl > > SQS and others: > > https://github.com/x6j8x/erlaws/tree > https://github.com/bernied/erlawys > https://github.com/mfine/aws_messaging > > Thanks! > Tristan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Wed Mar 21 15:51:55 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Wed, 21 Mar 2012 09:51:55 -0500 Subject: [erlang-questions] AWS Clients In-Reply-To: <004C29264E7244BDAC8DFB3AFD55A4D9@srstrong.com> References: <004C29264E7244BDAC8DFB3AFD55A4D9@srstrong.com> Message-ID: Thanks, I also didn't know about this 'fork' of erlcloud. Does anyone know why gleber's is ahead of the original, under bwbuchanan? And then there is https://github.com/nivertech/erlcloud which has a couple other commits not in gleber's... Tristan On Wed, Mar 21, 2012 at 9:46 AM, Steve Strong wrote: > We're using erlawys extensively - I think it's fair to say that given a > week or so of spare time, we'd probably rip it out - some of the others > that have come more recently look considerably better and offer access to > far more Amazon services. One that caught my eye a month or two back which > isn't in your list is https://github.com/gleber/erlcloud > > Cheers, > > Steve > > -- > Steve Strong > @srstrong > > Sent with Sparrow > > On Wednesday, 21 March 2012 at 15:43, Tristan Sloughter wrote: > > I'm looking around for Erlang clients for Amazon services and there are a > number serving the same purpose, so I wanted to check if anyone had > experience with them and opinions. I'm mostly, at this time, interested in > SQS and Dynamo. These are the ones I've found so far: > > Dynamo: > > https://github.com/wagerlabs/ddb > https://github.com/SemanticSugar/dinerl > > SQS and others: > > https://github.com/x6j8x/erlaws/tree > https://github.com/bernied/erlawys > https://github.com/mfine/aws_messaging > > Thanks! > Tristan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Wed Mar 21 15:51:37 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 21 Mar 2012 15:51:37 +0100 Subject: [erlang-questions] AWS Clients In-Reply-To: <004C29264E7244BDAC8DFB3AFD55A4D9@srstrong.com> References: <004C29264E7244BDAC8DFB3AFD55A4D9@srstrong.com> Message-ID: On Wed, Mar 21, 2012 at 15:46, Steve Strong wrote: > One that caught my eye a month or two back which isn't in > your list is?https://github.com/gleber/erlcloud I'm current maintainer of of erlcloud and I'd be glad to receive pull requests with any features you happened to add to it. Unfortunately I don't have much time to add more things (since I don't use them myself), but I will do maintainer's work with due diligence! It would be awesome if it will become full suite for all AWS APIs Best regards, Gleb Peregud From g@REDACTED Wed Mar 21 15:58:47 2012 From: g@REDACTED (Garrett Smith) Date: Wed, 21 Mar 2012 09:58:47 -0500 Subject: [erlang-questions] AWS Clients In-Reply-To: <004C29264E7244BDAC8DFB3AFD55A4D9@srstrong.com> References: <004C29264E7244BDAC8DFB3AFD55A4D9@srstrong.com> Message-ID: On Wed, Mar 21, 2012 at 9:46 AM, Steve Strong wrote: > We're using erlawys extensively - I think it's fair to say that given a week > or so of spare time, we'd probably rip it out - some of the others that have > come more recently look considerably better and offer access to far more > Amazon services. ?One that caught my eye a month or two back which isn't in > your list is?https://github.com/gleber/erlcloud I'm in a similar position -- i.e. looking to possibly replace our current AWS interface. We're using boto + erlport: http://code.google.com/p/boto/ http://erlport.org/ It's admittedly an odd arrangement -- but boto is a hands-down solid AWS interface and I don't have to worry about missing functionality. The one Erlang library that I would consider using at this point is gleber's erlcloud fork. No production experience though. Garrett From tristan.sloughter@REDACTED Wed Mar 21 16:17:10 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Wed, 21 Mar 2012 10:17:10 -0500 Subject: [erlang-questions] AWS Clients In-Reply-To: References: <004C29264E7244BDAC8DFB3AFD55A4D9@srstrong.com> Message-ID: Awesome, seems I'll go with gleber/erlcloud (already sent a pull request for some updates for sinan :) and wagerlabs/ddb. Would it be good to integrate ddb from wagerlabs to gleber/erlcloud? I can look at doing that as well. Tristan On Wed, Mar 21, 2012 at 9:58 AM, Garrett Smith wrote: > On Wed, Mar 21, 2012 at 9:46 AM, Steve Strong wrote: > > We're using erlawys extensively - I think it's fair to say that given a > week > > or so of spare time, we'd probably rip it out - some of the others that > have > > come more recently look considerably better and offer access to far more > > Amazon services. One that caught my eye a month or two back which isn't > in > > your list is https://github.com/gleber/erlcloud > > I'm in a similar position -- i.e. looking to possibly replace our > current AWS interface. > > We're using boto + erlport: > > http://code.google.com/p/boto/ > http://erlport.org/ > > It's admittedly an odd arrangement -- but boto is a hands-down solid > AWS interface and I don't have to worry about missing functionality. > > The one Erlang library that I would consider using at this point is > gleber's erlcloud fork. No production experience though. > > Garrett > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Wed Mar 21 16:29:51 2012 From: g@REDACTED (Garrett Smith) Date: Wed, 21 Mar 2012 10:29:51 -0500 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: <4F69C016.6080800@gmail.com> References: <4F69C016.6080800@gmail.com> Message-ID: On Wed, Mar 21, 2012 at 6:48 AM, Torben Hoffmann wrote: > Hi, > > I was sitting with a co-worker - who is proficient in OCaml and lots of > other stuff and he is now learning Erlang - and then he comes up with a good > question on the amount of boilerplate code when implementing a gen_server. I've had the exact same conversation with coworkers -- it led to this: http://e2project.org/ > From the API functions you call out to gen_server:call/2 and on top of that > you need to implement a handle_call/3 function clause that matches whatever > format you used for wrapping the incoming message. > > The question was: why don't you use an IDL (Interface Definition Language) > approach instead of writing all that code that has to match up in order to > work? Erlang has a "parse transform" feature that would let you embed such an "IDL" in a single module, and could generate the "boilerplate". I initially started down this road with e2, but decided against it for these reasons: - Erlang is already a *very* simple language - let's just use it - Auto generated functions are hard to document, test, etc. - Hiding the client-server distinction in Erlang is a terrible disservice IMO, e2 solves the problem of "too much boiler plate". It's really easy and requires zero magic. Here's a "gen_server" equivalent in e2: https://github.com/gar1t/e2v2/blob/master/examples/ping/src/ping_server.erl You might object to the separation of "ping" and "handle_msg" -- it appears to be just one function, so why break it up into two pieces? The problem is that it's not one function -- it's definitely *two* very separate pieces of code. When you write a gen_server style module, you're writing code that support client-server interactions. You have, in the same module, code that is called by the "client" and code that is called by the "server". If you don't grok this, you're missing the entire point of this type of module. Code that hides this difference -- e.g. a parse transform that gloms the client and server functions into one -- is IMO a Really Bad Idea. As an example of how this client/server difference is important, consider form validation in a web app. There are two places you can run code to validate form input -- you can validate it in the browser using JavaScript, or you can send the form data to the server. Web developers may opt for browser validation to avoid the expense of sending data to the server -- or they might prefer it on the server. Either way, it's an important consideration. This same dynamic applies to gen_server style modules. If you stick with it, I think you'll appreciate the separateness of client and server facing code. As for the boilerplate, I couldn't agree with you more! Garrett P.S. I'm giving a talk on e2 at the SF Factory in a couple weeks, where I'll get into this in more details. Also, the e2 docs and github project are in slight disarray at the moment, but will be put in order this weekend! From gleber.p@REDACTED Wed Mar 21 17:01:04 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 21 Mar 2012 17:01:04 +0100 Subject: [erlang-questions] AWS Clients In-Reply-To: References: <004C29264E7244BDAC8DFB3AFD55A4D9@srstrong.com> Message-ID: On Wed, Mar 21, 2012 at 16:17, Tristan Sloughter wrote: > (already sent a pull request for some updates for sinan :) Reviewing now. > Would it be good to integrate ddb from wagerlabs to gleber/erlcloud? I can > look at doing that as well. I've looked into it, but had not enough time to do it myself. I'd be glad to include it! We briefly discussed it with Joel and he has no objections to integrating ddb into erlcloud. nivertech/erlcloud has changes which switches from httpc to ibrowse. I've been using ibrowse for most of my projects and I am satisfied with it. But for erlcloud httpc is good enough, at least in my use case. Should erlcloud switch to ibrowse? Opinions? From watson.timothy@REDACTED Wed Mar 21 19:28:38 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 21 Mar 2012 18:28:38 +0000 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: References: <4F69C016.6080800@gmail.com> Message-ID: On 21 Mar 2012, at 15:29, Garrett Smith wrote: > > http://e2project.org/ > I've looked at this before and I like the general concept, and I suspect if it gets up to a suitable quality could be a good candidate for moving into OTP proper over time. >> From the API functions you call out to gen_server:call/2 and on top of that >> you need to implement a handle_call/3 function clause that matches whatever >> format you used for wrapping the incoming message. >> >> The question was: why don't you use an IDL (Interface Definition Language) >> approach instead of writing all that code that has to match up in order to >> work? > > Erlang has a "parse transform" feature that would let you embed such > an "IDL" in a single module, and could generate the "boilerplate". > > I initially started down this road with e2, but decided against it for > these reasons: > > - Erlang is already a *very* simple language - let's just use it > - Auto generated functions are hard to document, test, etc. In what way are auto generated functions hard to test? Why can you not generate documentation for them as you would hand coded functions? Good code generation tools will support the latter case and I'm not at all convinced that the charge of being 'hard to test' is at all true, especially for code generated by a parse transform. > - Hiding the client-server distinction in Erlang is a terrible disservice > I agree with this wholeheartedly, but don't see what it has to do with code generation. > IMO, e2 solves the problem of "too much boiler plate". It's really > easy and requires zero magic. > > Here's a "gen_server" equivalent in e2: > > https://github.com/gar1t/e2v2/blob/master/examples/ping/src/ping_server.erl > This is pretty cool though. :) > You might object to the separation of "ping" and "handle_msg" -- it > appears to be just one function, so why break it up into two pieces? > > The problem is that it's not one function -- it's definitely *two* > very separate pieces of code. Absolutely and a good point! > > When you write a gen_server style module, you're writing code that > support client-server interactions. You have, in the same module, code > that is called by the "client" and code that is called by the > "server". If you don't grok this, you're missing the entire point of > this type of module. > > Code that hides this difference -- e.g. a parse transform that gloms > the client and server functions into one -- is IMO a Really Bad Idea. > I don't necessarily agree with this, as the parse transform can be applied to a -spec which is potentially as intention revealing as a hand written function. I do take your point about them being two separate functions and perhaps in this particular case (for gen_server) you are actually correct and having two hand coded parts is actually better. I don't buy 'code-gen is bad' as a general argument (which perhaps you weren't going so far as to make anyway), and I do see your point in this instance. > As an example of how this client/server difference is important, > consider form validation in a web app. There are two places you can > run code to validate form input -- you can validate it in the browser > using JavaScript, or you can send the form data to the server. Web > developers may opt for browser validation to avoid the expense of > sending data to the server -- or they might prefer it on the server. > Either way, it's an important consideration. > > This same dynamic applies to gen_server style modules. If you stick > with it, I think you'll appreciate the separateness of client and > server facing code. > > As for the boilerplate, I couldn't agree with you more! > > Garrett > > P.S. I'm giving a talk on e2 at the SF Factory in a couple weeks, > where I'll get into this in more details. Also, the e2 docs and github > project are in slight disarray at the moment, but will be put in order > this weekend! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Wed Mar 21 20:21:43 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 21 Mar 2012 20:21:43 +0100 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: References: <4F69C016.6080800@gmail.com> Message-ID: On Wed, Mar 21, 2012 at 4:29 PM, Garrett Smith wrote: > On Wed, Mar 21, 2012 at 6:48 AM, Torben Hoffmann > wrote: >> Hi, >> >> I was sitting with a co-worker - who is proficient in OCaml and lots of >> other stuff and he is now learning Erlang - and then he comes up with a good >> question on the amount of boilerplate code when implementing a gen_server. > > I've had the exact same conversation with coworkers -- it led to this: > > http://e2project.org/ > >> From the API functions you call out to gen_server:call/2 and on top of that >> you need to implement a handle_call/3 function clause that matches whatever >> format you used for wrapping the incoming message. >> >> The question was: why don't you use an IDL (Interface Definition Language) >> approach instead of writing all that code that has to match up in order to >> work? > > Erlang has a "parse transform" feature that would let you embed such > an "IDL" in a single module, and could generate the "boilerplate". > > I initially started down this road with e2, but decided against it for > these reasons: > > - Erlang is already a *very* simple language - let's just use it > - Auto generated functions are hard to document, test, etc. > - Hiding the client-server distinction in Erlang is a terrible disservice > > IMO, e2 solves the problem of "too much boiler plate". It's really > easy and requires zero magic. > > Here's a "gen_server" equivalent in e2: > > https://github.com/gar1t/e2v2/blob/master/examples/ping/src/ping_server.erl That's nice - but one could with a little thought and a few conventions make this even shorter. Something like: -module(ping_service). -behavior(e2_service). -interface([ping/0]). ping(_From, State) -> {reply, pong, State}. And just let all undefined values default to sensible values. As in your version you could use init() -> State to prime the state A counter would just be: -module(counter). -behaviour(e2_service). -interface([inc/1, dec/1, get/0]) init() -> 0. deposit(_From, X,N) -> {reply,ack,N+X}. withdraw(_From,N) -> {reply,ack,N-1}. get(_From, N) -> {reply,N,N}. > > You might object to the separation of "ping" and "handle_msg" -- it > appears to be just one function, so why break it up into two pieces? > > The problem is that it's not one function -- it's definitely *two* > very separate pieces of code. > > When you write a gen_server style module, you're writing code that > support client-server interactions. You have, in the same module, code > that is called by the "client" and code that is called by the > "server". If you don't grok this, you're missing the entire point of > this type of module. > > Code that hides this difference -- e.g. a parse transform that gloms > the client and server functions into one -- is IMO a Really Bad Idea. > > As an example of how this client/server difference is important, > consider form validation in a web app. There are two places you can > run code to validate form input -- you can validate it in the browser > using JavaScript, or you can send the form data to the server. Web > developers may opt for browser validation to avoid the expense of > sending data to the server -- or they might prefer it on the server. > Either way, it's an important consideration. > > This same dynamic applies to gen_server style modules. If you stick > with it, I think you'll appreciate the separateness of client and > server facing code. > > As for the boilerplate, I couldn't agree with you more! > > Garrett > > P.S. I'm giving a talk on e2 at the SF Factory in a couple weeks, > where I'll get into this in more details. Also, the e2 docs and github > project are in slight disarray at the moment, but will be put in order > this weekend! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Wed Mar 21 20:22:30 2012 From: g@REDACTED (Garrett Smith) Date: Wed, 21 Mar 2012 14:22:30 -0500 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: References: <4F69C016.6080800@gmail.com> Message-ID: On Wed, Mar 21, 2012 at 1:28 PM, Tim Watson wrote: > On 21 Mar 2012, at 15:29, Garrett Smith wrote: >> >> http://e2project.org/ >> > > I've looked at this before and I like the general concept, and I suspect if it gets up to a suitable quality could be a good candidate for moving into OTP proper over time. > >>> From the API functions you call out to gen_server:call/2 and on top of that >>> you need to implement a handle_call/3 function clause that matches whatever >>> format you used for wrapping the incoming message. >>> >>> The question was: why don't you use an IDL (Interface Definition Language) >>> approach instead of writing all that code that has to match up in order to >>> work? >> >> Erlang has a "parse transform" feature that would let you embed such >> an "IDL" in a single module, and could generate the "boilerplate". >> >> I initially started down this road with e2, but decided against it for >> these reasons: >> >> - Erlang is already a *very* simple language - let's just use it >> - Auto generated functions are hard to document, test, etc. > > In what way are auto generated functions hard to test? Why can you not generate documentation for them as you would hand coded functions? Good code generation tools will support the latter case and I'm not at all convinced that the charge of being 'hard to test' is at all true, especially for code generated by a parse transform. This may come down to personal taste. I don't like having to dig around in parse transform code to understand what my module looks like. eunit is a great example. If you really love eunit, then I'm not going to convince you :) >> - Hiding the client-server distinction in Erlang is a terrible disservice > > I agree with this wholeheartedly, but don't see what it has to do with code generation. If you consolidate both the client and server functions into one function, you've obfuscated an important distinction. >> IMO, e2 solves the problem of "too much boiler plate". It's really >> easy and requires zero magic. >> >> Here's a "gen_server" equivalent in e2: >> >> https://github.com/gar1t/e2v2/blob/master/examples/ping/src/ping_server.erl >> > > This is pretty cool though. :) > >> You might object to the separation of "ping" and "handle_msg" -- it >> appears to be just one function, so why break it up into two pieces? >> >> The problem is that it's not one function -- it's definitely *two* >> very separate pieces of code. > > Absolutely and a good point! > >> >> When you write a gen_server style module, you're writing code that >> support client-server interactions. You have, in the same module, code >> that is called by the "client" and code that is called by the >> "server". If you don't grok this, you're missing the entire point of >> this type of module. >> >> Code that hides this difference -- e.g. a parse transform that gloms >> the client and server functions into one -- is IMO a Really Bad Idea. >> > > I don't necessarily agree with this, as the parse transform can be applied to a -spec which is potentially as intention revealing as a hand written function. I do take your point about them being two separate functions and perhaps in this particular case (for gen_server) you are actually correct and having two hand coded parts is actually better. I don't buy 'code-gen is bad' as a general argument (which perhaps you weren't going so far as to make anyway), and I do see your point in this instance. I'm certainly not arguing against code generation. I think Erlang's parse transform scheme is *very* good -- flexible enough to enable elegant solutions but with enough complexity to scare off casual "meta programmers". But -- I think there should be a very clear payoff for using a parse transform. I don't think the discussion here, which is "boilerplate" is a good application for that. A good example of a value-add parse transform IMO is modlib, which lets you create mods for inets httpd withou causing your brain to explode [1]. >> As an example of how this client/server difference is important, >> consider form validation in a web app. There are two places you can >> run code to validate form input -- you can validate it in the browser >> using JavaScript, or you can send the form data to the server. Web >> developers may opt for browser validation to avoid the expense of >> sending data to the server -- or they might prefer it on the server. >> Either way, it's an important consideration. >> >> This same dynamic applies to gen_server style modules. If you stick >> with it, I think you'll appreciate the separateness of client and >> server facing code. >> >> As for the boilerplate, I couldn't agree with you more! >> >> Garrett >> >> P.S. I'm giving a talk on e2 at the SF Factory in a couple weeks, >> where I'll get into this in more details. Also, the e2 docs and github >> project are in slight disarray at the moment, but will be put in order >> this weekend! [1] https://github.com/gar1t/modlib From g@REDACTED Wed Mar 21 20:28:49 2012 From: g@REDACTED (Garrett Smith) Date: Wed, 21 Mar 2012 14:28:49 -0500 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: References: <4F69C016.6080800@gmail.com> Message-ID: On Wed, Mar 21, 2012 at 2:21 PM, Joe Armstrong wrote: > On Wed, Mar 21, 2012 at 4:29 PM, Garrett Smith wrote: >> >> Here's a "gen_server" equivalent in e2: >> >> https://github.com/gar1t/e2v2/blob/master/examples/ping/src/ping_server.erl > > That's nice - but one could with a little thought and a few conventions > make this even shorter. Something like: > > ? ?-module(ping_service). > ? ?-behavior(e2_service). > ? ?-interface([ping/0]). > > ? ?ping(_From, State) -> {reply, pong, State}. > > And just let all undefined values default to sensible values. > As in your version you could use > > ? ?init() -> State to prime the state > > A counter would just be: > > ? ?-module(counter). > ? ?-behaviour(e2_service). > ? ?-interface([inc/1, dec/1, get/0]) > ? ?init() -> 0. > > ? ?deposit(_From, X,N) -> {reply,ack,N+X}. > ? ?withdraw(_From,N) -> > {reply,ack,N-1}. > ? ?get(_From, N) -> {reply,N,N}. Joe, in this case, what would the "interface" documentation look like? Garrett From erlang@REDACTED Wed Mar 21 20:35:36 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 21 Mar 2012 20:35:36 +0100 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: References: <4F69C016.6080800@gmail.com> Message-ID: My last post got sent before it was ready - (moral don't compose code in the browser) - I meant to say: -module(bank). -behavior(e2_service). -interface([deposit/1, withdraw/1, ask/0]). init() -> 0. deposit(_From, X, N) -> {noreply, X+N}. withdraw(_From, X, N) -> if X < N -> {reply, no, X}; true -> {reply, X, N-X} end. ask(_From, N) -> {reply, N, N}. %% The caller will just say bank:deposit(10) bank:withdraw(12) Pretty easy to code up with a few simple code transformations I had an interesting conversation with a guy who knew Ruby on Rails he attributed part of the success of RoR to the autogeneration of large numbers of stub functions from database schemas. I think there are many users who don't care how things work provided they *do* work and who are quite happy with "do this, and this and this and it will work" (( actually things like eclipse etc. encourage this kind of mind-set - first you ask a load of questions that nobody on this planet understands - then you create a bundle of mysterious files with undocumented and mysterious contents - then you press the big "build" button and with a bit of luck it works )) /Joe On Wed, Mar 21, 2012 at 8:22 PM, Garrett Smith wrote: > On Wed, Mar 21, 2012 at 1:28 PM, Tim Watson wrote: >> On 21 Mar 2012, at 15:29, Garrett Smith wrote: >>> >>> http://e2project.org/ >>> >> >> I've looked at this before and I like the general concept, and I suspect if it gets up to a suitable quality could be a good candidate for moving into OTP proper over time. >> >>>> From the API functions you call out to gen_server:call/2 and on top of that >>>> you need to implement a handle_call/3 function clause that matches whatever >>>> format you used for wrapping the incoming message. >>>> >>>> The question was: why don't you use an IDL (Interface Definition Language) >>>> approach instead of writing all that code that has to match up in order to >>>> work? >>> >>> Erlang has a "parse transform" feature that would let you embed such >>> an "IDL" in a single module, and could generate the "boilerplate". >>> >>> I initially started down this road with e2, but decided against it for >>> these reasons: >>> >>> - Erlang is already a *very* simple language - let's just use it >>> - Auto generated functions are hard to document, test, etc. >> >> In what way are auto generated functions hard to test? Why can you not generate documentation for them as you would hand coded functions? Good code generation tools will support the latter case and I'm not at all convinced that the charge of being 'hard to test' is at all true, especially for code generated by a parse transform. > > This may come down to personal taste. I don't like having to dig > around in parse transform code to understand what my module looks > like. > > eunit is a great example. If you really love eunit, then I'm not going > to convince you :) > >>> - Hiding the client-server distinction in Erlang is a terrible disservice >> >> I agree with this wholeheartedly, but don't see what it has to do with code generation. > > If you consolidate both the client and server functions into one > function, you've obfuscated an important distinction. > >>> IMO, e2 solves the problem of "too much boiler plate". It's really >>> easy and requires zero magic. >>> >>> Here's a "gen_server" equivalent in e2: >>> >>> https://github.com/gar1t/e2v2/blob/master/examples/ping/src/ping_server.erl >>> >> >> This is pretty cool though. :) >> >>> You might object to the separation of "ping" and "handle_msg" -- it >>> appears to be just one function, so why break it up into two pieces? >>> >>> The problem is that it's not one function -- it's definitely *two* >>> very separate pieces of code. >> >> Absolutely and a good point! >> >>> >>> When you write a gen_server style module, you're writing code that >>> support client-server interactions. You have, in the same module, code >>> that is called by the "client" and code that is called by the >>> "server". If you don't grok this, you're missing the entire point of >>> this type of module. >>> >>> Code that hides this difference -- e.g. a parse transform that gloms >>> the client and server functions into one -- is IMO a Really Bad Idea. >>> >> >> I don't necessarily agree with this, as the parse transform can be applied to a -spec which is potentially as intention revealing as a hand written function. I do take your point about them being two separate functions and perhaps in this particular case (for gen_server) you are actually correct and having two hand coded parts is actually better. I don't buy 'code-gen is bad' as a general argument (which perhaps you weren't going so far as to make anyway), and I do see your point in this instance. > > I'm certainly not arguing against code generation. I think Erlang's > parse transform scheme is *very* good -- flexible enough to enable > elegant solutions but with enough complexity to scare off casual "meta > programmers". > > But -- I think there should be a very clear payoff for using a parse > transform. I don't think the discussion here, which is "boilerplate" > is a good application for that. > > A good example of a value-add parse transform IMO is modlib, which > lets you create mods for inets httpd withou causing your brain to > explode [1]. > >>> As an example of how this client/server difference is important, >>> consider form validation in a web app. There are two places you can >>> run code to validate form input -- you can validate it in the browser >>> using JavaScript, or you can send the form data to the server. Web >>> developers may opt for browser validation to avoid the expense of >>> sending data to the server -- or they might prefer it on the server. >>> Either way, it's an important consideration. >>> >>> This same dynamic applies to gen_server style modules. If you stick >>> with it, I think you'll appreciate the separateness of client and >>> server facing code. >>> >>> As for the boilerplate, I couldn't agree with you more! >>> >>> Garrett >>> >>> P.S. I'm giving a talk on e2 at the SF Factory in a couple weeks, >>> where I'll get into this in more details. Also, the e2 docs and github >>> project are in slight disarray at the moment, but will be put in order >>> this weekend! > > [1] https://github.com/gar1t/modlib > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Wed Mar 21 21:01:09 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 21 Mar 2012 21:01:09 +0100 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: References: <4F69C016.6080800@gmail.com> Message-ID: On Wed, Mar 21, 2012 at 8:28 PM, Garrett Smith wrote: > On Wed, Mar 21, 2012 at 2:21 PM, Joe Armstrong wrote: >> On Wed, Mar 21, 2012 at 4:29 PM, Garrett Smith wrote: >>> >>> Here's a "gen_server" equivalent in e2: >>> >>> https://github.com/gar1t/e2v2/blob/master/examples/ping/src/ping_server.erl >> >> That's nice - but one could with a little thought and a few conventions >> make this even shorter. Something like: >> >> ? ?-module(ping_service). >> ? ?-behavior(e2_service). >> ? ?-interface([ping/0]). >> >> ? ?ping(_From, State) -> {reply, pong, State}. >> >> And just let all undefined values default to sensible values. >> As in your version you could use >> >> ? ?init() -> State to prime the state >> >> A counter would just be: >> >> ? ?-module(counter). >> ? ?-behaviour(e2_service). >> ? ?-interface([inc/1, dec/1, get/0]) >> ? ?init() -> 0. >> >> ? ?deposit(_From, X,N) -> {reply,ack,N+X}. >> ? ?withdraw(_From,N) -> >> {reply,ack,N-1}. >> ? ?get(_From, N) -> {reply,N,N}. > > Joe, in this case, what would the "interface" documentation look like? > It should expose the interface from the external point of view. The ping_service interface is. ping_service ? ping => pong meaning if you send the ping service a 'ping' message it will reply with a 'pong' message. All the stuff *inside* the implementation (the From, and State) is basically junk (as seen from the outside) and should not be in seen in the interface type signatures. Re: your comments on parse_transforms - Yes and No :-) In my experience very few people break open abstractions to see how things work if they do in fact work. Very few people seem to look inside the gen_server to see how they work and even fewer modify gen_server etc. How many people have changed error_handler.erl ? I designed it specifically so that you could drop in you own replacement. A lot of the things in the code server/io system etc are designed with hooks so they can be customised later - but virtually nobody does this. Probably a case of 'if it ain't broke don't fix it' - all this would argue in favor of a fixed set of boiler plate transformations aimed at truly minimalistic code for the commonest use cases. /Joe > Garrett From watson.timothy@REDACTED Wed Mar 21 21:10:27 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 21 Mar 2012 20:10:27 +0000 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: References: <4F69C016.6080800@gmail.com> Message-ID: <6F99D60B-C45A-4E21-AAEC-FF5E1630298E@gmail.com> On 21 Mar 2012, at 19:22, Garrett Smith wrote: >> > > This may come down to personal taste. I don't like having to dig > around in parse transform code to understand what my module looks > like. > Neither do I - that's why I simply run 'escript `evm site`/parse_trans/ebin/parse_trans_pp.beam ebin/.beam' over the generated beam to see what it looks like at source level. ;) > eunit is a great example. If you really love eunit, then I'm not going > to convince you :) > Common test and PropEr do exactly the same thing. I'm not sure what you're expected testing frameworks to do instead!? >>> - Hiding the client-server distinction in Erlang is a terrible disservice >> >> I agree with this wholeheartedly, but don't see what it has to do with code generation. > > If you consolidate both the client and server functions into one > function, you've obfuscated an important distinction. > I never suggested doing that at all. Not that I mean to be prickly, as we're all part of the same community and whatnot, but nobody suggested separating both functions. What the OP suggested was that you could define the interface declaratively - e.g., without writing the code directly - and that was what I was responding to. As I mentioned later on, in this particular gen_server case I actually think your approach is probably cleaner and more appropriate, but it's good to separate these points and refine the discussion I think. >>> IMO, e2 solves the problem of "too much boiler plate". It's really >>> easy and requires zero magic. >>> >>> Here's a "gen_server" equivalent in e2: >>> >>> https://github.com/gar1t/e2v2/blob/master/examples/ping/src/ping_server.erl >>> >> >> This is pretty cool though. :) >> >>> You might object to the separation of "ping" and "handle_msg" -- it >>> appears to be just one function, so why break it up into two pieces? >>> >>> The problem is that it's not one function -- it's definitely *two* >>> very separate pieces of code. >> >> Absolutely and a good point! >> >>> >>> When you write a gen_server style module, you're writing code that >>> support client-server interactions. You have, in the same module, code >>> that is called by the "client" and code that is called by the >>> "server". If you don't grok this, you're missing the entire point of >>> this type of module. >>> >>> Code that hides this difference -- e.g. a parse transform that gloms >>> the client and server functions into one -- is IMO a Really Bad Idea. >>> >> >> I don't necessarily agree with this, as the parse transform can be applied to a -spec which is potentially as intention revealing as a hand written function. I do take your point about them being two separate functions and perhaps in this particular case (for gen_server) you are actually correct and having two hand coded parts is actually better. I don't buy 'code-gen is bad' as a general argument (which perhaps you weren't going so far as to make anyway), and I do see your point in this instance. > > I'm certainly not arguing against code generation. I think Erlang's > parse transform scheme is *very* good -- flexible enough to enable > elegant solutions but with enough complexity to scare off casual "meta > programmers". > > But -- I think there should be a very clear payoff for using a parse > transform. I don't think the discussion here, which is "boilerplate" > is a good application for that. > I think it depends on how much boilerplate you're looking at. As an OCaml programmer I have seen a *lot* of work going into removing boilerplate code, and the same in Haskell (although there I am less experienced) and overall I think the payoff you're describing is a value judgement based on the negative impact of using code generation. Generated code is brittle - there's no two ways about it. But it doesn't prevent you from testing or documenting, nor does it have to make your code opaque and difficult to understand, so long as it is not overused - a sprinkle here and there rather than a bucket load. > A good example of a value-add parse transform IMO is modlib, which > lets you create mods for inets httpd withou causing your brain to > explode [1]. > >>> As an example of how this client/server difference is important, >>> consider form validation in a web app. There are two places you can >>> run code to validate form input -- you can validate it in the browser >>> using JavaScript, or you can send the form data to the server. Web >>> developers may opt for browser validation to avoid the expense of >>> sending data to the server -- or they might prefer it on the server. >>> Either way, it's an important consideration. >>> >>> This same dynamic applies to gen_server style modules. If you stick >>> with it, I think you'll appreciate the separateness of client and >>> server facing code. >>> >>> As for the boilerplate, I couldn't agree with you more! >>> >>> Garrett >>> >>> P.S. I'm giving a talk on e2 at the SF Factory in a couple weeks, >>> where I'll get into this in more details. Also, the e2 docs and github >>> project are in slight disarray at the moment, but will be put in order >>> this weekend! > > [1] https://github.com/gar1t/modlib -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Wed Mar 21 21:17:23 2012 From: g@REDACTED (Garrett Smith) Date: Wed, 21 Mar 2012 15:17:23 -0500 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: References: <4F69C016.6080800@gmail.com> Message-ID: On Wed, Mar 21, 2012 at 3:01 PM, Joe Armstrong wrote: > On Wed, Mar 21, 2012 at 8:28 PM, Garrett Smith wrote: >> On Wed, Mar 21, 2012 at 2:21 PM, Joe Armstrong wrote: >>> On Wed, Mar 21, 2012 at 4:29 PM, Garrett Smith wrote: >>>> >>>> Here's a "gen_server" equivalent in e2: >>>> >>>> https://github.com/gar1t/e2v2/blob/master/examples/ping/src/ping_server.erl >>> >>> That's nice - but one could with a little thought and a few conventions >>> make this even shorter. Something like: >>> >>> ? ?-module(ping_service). >>> ? ?-behavior(e2_service). >>> ? ?-interface([ping/0]). >>> >>> ? ?ping(_From, State) -> {reply, pong, State}. >>> >>> And just let all undefined values default to sensible values. >>> As in your version you could use >>> >>> ? ?init() -> State to prime the state >>> >>> A counter would just be: >>> >>> ? ?-module(counter). >>> ? ?-behaviour(e2_service). >>> ? ?-interface([inc/1, dec/1, get/0]) >>> ? ?init() -> 0. >>> >>> ? ?deposit(_From, X,N) -> {reply,ack,N+X}. >>> ? ?withdraw(_From,N) -> >>> {reply,ack,N-1}. >>> ? ?get(_From, N) -> {reply,N,N}. >> >> Joe, in this case, what would the "interface" documentation look like? >> > > It should expose the interface from the external point of view. > The ping_service interface is. > > ? ping_service ? ping => pong > > meaning if you send the ping service a 'ping' message it will reply > with a 'pong' message. > > All the stuff *inside* the implementation (the From, and State) > is basically junk (as seen from the outside) and should not be > in seen in the interface type signatures. > > Re: your comments on parse_transforms - Yes and No :-) > > In my experience very few people break open abstractions to see how > things work if they do in fact work. Very few people seem to > look inside the gen_server to see how they work and even fewer modify > gen_server etc. > > How many people have changed error_handler.erl ? I designed it > specifically so that you could drop in you own replacement. A lot > of the things in the code server/io system etc are designed with hooks > so they can be customised later - but virtually nobody does this. > > Probably a case of 'if it ain't broke don't fix it' - all this would argue > in favor of a fixed set of boiler plate transformations aimed at truly > minimalistic code for the commonest use cases. > > /Joe This is indeed where I started with e2 -- the impulse was get something like Objective C, where the underlying mechanics were message passing, but the user code didn't expose that. It started to move further away from the Erlang that you see everywhere else -- and that wasn't a goal. My main objective is to help people (in particular the folks I work with) leverage the outstanding patterns in OTP, which are currently harder to use than they need to be. If done right, e2 should feel mostly like OTP, but clearer and easier. But dammit, now you've got me thinking again ;) Garrett From g@REDACTED Wed Mar 21 21:23:36 2012 From: g@REDACTED (Garrett Smith) Date: Wed, 21 Mar 2012 15:23:36 -0500 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: <6F99D60B-C45A-4E21-AAEC-FF5E1630298E@gmail.com> References: <4F69C016.6080800@gmail.com> <6F99D60B-C45A-4E21-AAEC-FF5E1630298E@gmail.com> Message-ID: On Wed, Mar 21, 2012 at 3:10 PM, Tim Watson wrote: > On 21 Mar 2012, at 19:22, Garrett Smith wrote: > > I think it depends on how much boilerplate you're looking at. As an OCaml > programmer I have seen a *lot* of work going into removing boilerplate code, > and the same in Haskell (although there I am less experienced) and overall I > think the payoff you're describing is a value judgement based on the > negative impact of using code generation.?Generated code is brittle - > there's no two ways about it. But it doesn't prevent you from testing or > documenting, nor does it have to make your code opaque and difficult to > understand, so long as it is not overused - a sprinkle here and there rather > than a bucket load. Well put! I think we're in heated agreement :) Garrett From watson.timothy@REDACTED Wed Mar 21 21:29:36 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 21 Mar 2012 20:29:36 +0000 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: References: <4F69C016.6080800@gmail.com> <6F99D60B-C45A-4E21-AAEC-FF5E1630298E@gmail.com> Message-ID: <1CF50D4E-A371-41ED-B0E1-E02B1ADC644C@gmail.com> On 21 Mar 2012, at 20:23, Garrett Smith wrote: > On Wed, Mar 21, 2012 at 3:10 PM, Tim Watson wrote: >> On 21 Mar 2012, at 19:22, Garrett Smith wrote: >> >> I think it depends on how much boilerplate you're looking at. As an OCaml >> programmer I have seen a *lot* of work going into removing boilerplate code, >> and the same in Haskell (although there I am less experienced) and overall I >> think the payoff you're describing is a value judgement based on the >> negative impact of using code generation. Generated code is brittle - >> there's no two ways about it. But it doesn't prevent you from testing or >> documenting, nor does it have to make your code opaque and difficult to >> understand, so long as it is not overused - a sprinkle here and there rather >> than a bucket load. > > Well put! > > I think we're in heated agreement :) > > Garrett I suspect we are indeed. :D Good luck with e2 anyway - it looks really cool and I'm going to keep a close eye on it. From ok@REDACTED Wed Mar 21 23:53:47 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 22 Mar 2012 11:53:47 +1300 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: <4F69C016.6080800@gmail.com> References: <4F69C016.6080800@gmail.com> Message-ID: <9B6B38AF-02ED-4CD8-B32E-782535CBD078@cs.otago.ac.nz> On 22/03/2012, at 12:48 AM, Torben Hoffmann wrote: > The question was: why don't you use an IDL (Interface Definition Language) approach instead of writing all that code that has to match up in order to work? > > I have tried searching for it, but since IDL is so tightly coupled with CORBA I didn't really find anything but the ic application from the Erlang/OTP distribution.\ There are two very different concepts here. (1) "An interface definition language *APPROACH*." (2) CORBA IDL. CORBA IDL is language independent as long as the language you are pretending to be independent of is C++ or a close relative. It is about as bad a fit for Erlang data structures as you would expect it to be. This is not to say that Erlang cannot handle the data structures passed through a CORBA interface. It can. What I am saying is that CORBA doesn't handle Erlang data structures. CORBA IDL is by no means the only, nor by a very long time the first, interface definition language, not even if we are talking about interfaces for distributed programming. The first time I used remote procedure call was on a Xerox 1108 running Interlisp... There's no reason why some sort of "little language" for specifying gen_server stuff could not be devised. It might even be something that could be plugged in using a parse transform. But step 1 is to be very very clear about what is repetitive and what is not. Step 2 is to consider whether it can be done using higher-order functions. From steven.charles.davis@REDACTED Thu Mar 22 00:10:39 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 21 Mar 2012 16:10:39 -0700 (PDT) Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: <4F69C016.6080800@gmail.com> References: <4F69C016.6080800@gmail.com> Message-ID: <28323090.418.1332371439093.JavaMail.geo-discussion-forums@yner4> I suspect the acid test for this discussion would be to find out if a non-trivial application written over a "complexity hiding" framework (or other such smarts) was: 1) easier (or even equally easy) to understand and maintain 2) smaller than the "raw" equivalent that didn't use it Such comparisons are, of course, extremely unlikely. /s -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Thu Mar 22 09:09:03 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Thu, 22 Mar 2012 09:09:03 +0100 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: <6F99D60B-C45A-4E21-AAEC-FF5E1630298E@gmail.com> References: <4F69C016.6080800@gmail.com> <6F99D60B-C45A-4E21-AAEC-FF5E1630298E@gmail.com> Message-ID: <4F6ADE1F.6010904@gmail.com> As OP I take the liberty to do a top post and try to summarise a few things at this point. I am not alone when it comes to having an urge to remove some of the boilerplate code. I guess we all want value for our money and when we have to do something that does not provide that we want to improve. If we did not have that craving we might as well be coding enterprise Java applications using dozens of frameworks ;-) I had a look at the e2project and it actually does remove some of the boilerplate that I find the most iritating, namely the tagging of messages from the API function to the handle_* function. It is a lot clearer in e2project where you reference a function in the e2_service:call - I like that a lot. The approach with parse transforms has its merits, but I think that the e2project can do away with most of the pains I have been feeling, so I think I will leave out the semi-black magic of parse transforms for now... it is a tool that should be used with caution as other posters have mentioned. The IDL approach I had in mind was nothing about CORBA - I should never have mentioned CORBA in my original mail, there are too many negative feelings associated to CORBA. What I like about IDLs is that it allows you to spec your API in a nice way and in many cases it will be possible to avoid writing boilerplate code (either through code generation or using a parse transform). Given the good point made about separating the client and the server side of things I think that what is needed is simply to spec up the API function (who doesn't do that already?!?!) and then try out e2project. The spec will give me a slap over my fingers if I do not provide the function (I like that) and e2project seems to minimise the tedious boilerplate stuff from standard gen_server without limiting my ability to express what I need. A big thanks to all posters - I am glad I asked the question! Cheers, Torben On 21/3/12 21:10 , Tim Watson wrote: > On 21 Mar 2012, at 19:22, Garrett Smith wrote: >>> >> >> This may come down to personal taste. I don't like having to dig >> around in parse transform code to understand what my module looks >> like. >> > > Neither do I - that's why I simply run 'escript `evm > site`/parse_trans/ebin/parse_trans_pp.beam ebin/.beam' over > the generated beam to see what it looks like at source level. ;) > >> eunit is a great example. If you really love eunit, then I'm not going >> to convince you :) >> > > Common test and PropEr do exactly the same thing. I'm not sure what > you're expected testing frameworks to do instead!? > >>>> - Hiding the client-server distinction in Erlang is a terrible >>>> disservice >>> >>> I agree with this wholeheartedly, but don't see what it has to do >>> with code generation. >> >> If you consolidate both the client and server functions into one >> function, you've obfuscated an important distinction. >> > > I never suggested doing that at all. Not that I mean to be prickly, as > we're all part of the same community and whatnot, but nobody suggested > separating both functions. What the OP suggested was that you could > define the interface declaratively - e.g., without writing the code > directly - and that was what I was responding to. As I mentioned later > on, in this particular gen_server case I actually think your approach > is probably cleaner and more appropriate, but it's good to separate > these points and refine the discussion I think. > >>>> IMO, e2 solves the problem of "too much boiler plate". It's really >>>> easy and requires zero magic. >>>> >>>> Here's a "gen_server" equivalent in e2: >>>> >>>> https://github.com/gar1t/e2v2/blob/master/examples/ping/src/ping_server.erl >>>> >>> >>> This is pretty cool though. :) >>> >>>> You might object to the separation of "ping" and "handle_msg" -- it >>>> appears to be just one function, so why break it up into two pieces? >>>> >>>> The problem is that it's not one function -- it's definitely *two* >>>> very separate pieces of code. >>> >>> Absolutely and a good point! >>> >>>> >>>> When you write a gen_server style module, you're writing code that >>>> support client-server interactions. You have, in the same module, code >>>> that is called by the "client" and code that is called by the >>>> "server". If you don't grok this, you're missing the entire point of >>>> this type of module. >>>> >>>> Code that hides this difference -- e.g. a parse transform that gloms >>>> the client and server functions into one -- is IMO a Really Bad Idea. >>>> >>> >>> I don't necessarily agree with this, as the parse transform can be >>> applied to a -spec which is potentially as intention revealing as a >>> hand written function. I do take your point about them being two >>> separate functions and perhaps in this particular case (for >>> gen_server) you are actually correct and having two hand coded parts >>> is actually better. I don't buy 'code-gen is bad' as a general >>> argument (which perhaps you weren't going so far as to make anyway), >>> and I do see your point in this instance. >> >> I'm certainly not arguing against code generation. I think Erlang's >> parse transform scheme is *very* good -- flexible enough to enable >> elegant solutions but with enough complexity to scare off casual "meta >> programmers". >> >> But -- I think there should be a very clear payoff for using a parse >> transform. I don't think the discussion here, which is "boilerplate" >> is a good application for that. >> > > I think it depends on how much boilerplate you're looking at. As an > OCaml programmer I have seen a *lot* of work going into removing > boilerplate code, and the same in Haskell (although there I am less > experienced) and overall I think the payoff you're describing is a > value judgement based on the negative impact of using code > generation. Generated code is brittle - there's no two ways about it. > But it doesn't prevent you from testing or documenting, nor does it > have to make your code opaque and difficult to understand, so long as > it is not overused - a sprinkle here and there rather than a bucket load. > >> A good example of a value-add parse transform IMO is modlib, which >> lets you create mods for inets httpd withou causing your brain to >> explode [1]. >> >>>> As an example of how this client/server difference is important, >>>> consider form validation in a web app. There are two places you can >>>> run code to validate form input -- you can validate it in the browser >>>> using JavaScript, or you can send the form data to the server. Web >>>> developers may opt for browser validation to avoid the expense of >>>> sending data to the server -- or they might prefer it on the server. >>>> Either way, it's an important consideration. >>>> >>>> This same dynamic applies to gen_server style modules. If you stick >>>> with it, I think you'll appreciate the separateness of client and >>>> server facing code. >>>> >>>> As for the boilerplate, I couldn't agree with you more! >>>> >>>> Garrett >>>> >>>> P.S. I'm giving a talk on e2 at the SF Factory in a couple weeks, >>>> where I'll get into this in more details. Also, the e2 docs and github >>>> project are in slight disarray at the moment, but will be put in order >>>> this weekend! >> >> [1] https://github.com/gar1t/modlib > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Thu Mar 22 14:40:30 2012 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 22 Mar 2012 14:40:30 +0100 Subject: [erlang-questions] unicode:characters_to_list Message-ID: <4F6B2BCE.5030707@eonblast.com> Hi, I am perplexed about this result: > io:format(" ~s~nLatin: ~w~nUTF-8: ~w~nUTF-8 list: ~s~nUTF-8 list: ~w~n", [ > <<"?">>, > <<"?">>, > <<"?"/utf8>>, > unicode:characters_to_list(<<"?"/utf8>>,utf8), > unicode:characters_to_list(<<"?"/utf8>>,utf8) > ]). ? Latin: <<248>> UTF-8: <<195,184>> UTF-8 list: ? UTF-8 list: [248] ok Should not unicode:characters_to_list return a list with Unicode code points? The docs say: "This function converts a possibly deep list of integers and binaries into a list of integers representing unicode characters." http://www.erlang.org/doc/man/unicode.html#characters_to_list-2 In other words, I'd expect as results: ? Latin: <<248>> UTF-8: <<195,184>> UTF-8 list: bad argument UTF-8 list: [50104] ok Thanks, Henning -------------- next part -------------- An HTML attachment was scrubbed... URL: From masklinn@REDACTED Thu Mar 22 14:53:29 2012 From: masklinn@REDACTED (Masklinn) Date: Thu, 22 Mar 2012 14:53:29 +0100 Subject: [erlang-questions] unicode:characters_to_list In-Reply-To: <4F6B2BCE.5030707@eonblast.com> References: <4F6B2BCE.5030707@eonblast.com> Message-ID: <0EB5D6FF-4B92-417A-A504-9D8734B5A03A@masklinn.net> On 2012-03-22, at 14:40 , Henning Diedrich wrote: > Hi, > > I am perplexed about this result: > > > io:format(" ~s~nLatin: ~w~nUTF-8: ~w~nUTF-8 list: ~s~nUTF-8 list: ~w~n", [ > > <<"?">>, > > <<"?">>, > > <<"?"/utf8>>, > > unicode:characters_to_list(<<"?"/utf8>>,utf8), > > unicode:characters_to_list(<<"?"/utf8>>,utf8) > > ]). > ? > Latin: <<248>> > UTF-8: <<195,184>> > UTF-8 list: ? > UTF-8 list: [248] > ok > > Should not unicode:characters_to_list return a list with Unicode code points? That's what it does? > The docs say: "This function converts a possibly deep list of integers and binaries into a list of integers representing unicode characters." > > http://www.erlang.org/doc/man/unicode.html#characters_to_list-2 > > In other words, I'd expect as results: > > ? > Latin: <<248>> > UTF-8: <<195,184>> > UTF-8 list: bad argument > UTF-8 list: [50104] > ok Why would you expect that? The code point for ? is 248, where would 50104 come from? And why would the third version yield a bad argument when it's a valid string? As far as I can see, everything seems to be working correctly: <<195,184>> is decoded to the unicode list [248] aka the string "?". I don't understand why you're looking for UTF8 as the output of characters_to_list. From hd2010@REDACTED Thu Mar 22 15:32:37 2012 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 22 Mar 2012 15:32:37 +0100 Subject: [erlang-questions] unicode:characters_to_list In-Reply-To: <0EB5D6FF-4B92-417A-A504-9D8734B5A03A@masklinn.net> References: <4F6B2BCE.5030707@eonblast.com> <0EB5D6FF-4B92-417A-A504-9D8734B5A03A@masklinn.net> Message-ID: <4F6B3805.6070600@eonblast.com> Hi Masklinn, no question I am getting something wrong here. On 3/22/12 2:53 PM, Masklinn wrote: > I don't understand why you're looking for UTF8 as the output of > characters_to_list. That's what the doc leads me to believe I guess. "a list of integers representing unicode characters." should that be understood as "formerly unicode now latin-1 characters"? 248 is not a valid unicode Bytecode. Not sure if 50104 is the right one for the same character, but it would have to be two bytes and I would thus expect an integer higher 256. Or "Character code points not encoded as UTF-8"? If 248 is meant as actual number of the letter, which it is no matter the bitcode, then what is the right function to make a Unicode binary again from the list entries, unicode:character_to_binary/1 ? Thanks, Henning -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.eliasson@REDACTED Thu Mar 22 15:46:48 2012 From: daniel.eliasson@REDACTED (Daniel Eliasson) Date: Thu, 22 Mar 2012 15:46:48 +0100 Subject: [erlang-questions] unicode:characters_to_list In-Reply-To: <4F6B3805.6070600@eonblast.com> References: <4F6B2BCE.5030707@eonblast.com> <0EB5D6FF-4B92-417A-A504-9D8734B5A03A@masklinn.net> <4F6B3805.6070600@eonblast.com> Message-ID: unicode:characters_to_list returns a list of unicode codepoints, not utf8 bytecodes. You can get the latter with unicode:characters_to_binary/1,2,3 io:format("~w", [unicode:characters_to_list("?", utf8)]). => [248] vs io:format("~w", [unicode:characters_to_binary("?")]). => <<195,184>> Hope this makes it a little clearer. /Daniel On 22 March 2012 15:32, Henning Diedrich wrote: > Hi Masklinn, > > no question I am getting something wrong here. > > > On 3/22/12 2:53 PM, Masklinn wrote: > > I don't understand why you're looking for UTF8 as the output of > characters_to_list. > > > That's what the doc leads me to believe I guess. > > > "a list of integers representing unicode characters." > > should that be understood as "formerly unicode now latin-1 characters"? > > 248 is not a valid unicode Bytecode. Not sure if 50104 is the right one for > the same character, but it would have to be two bytes and I would thus > expect an integer higher 256. > > Or "Character code points not encoded as UTF-8"? > > If 248 is meant as actual number of the letter, which it is no matter the > bitcode, then what is the right function to make a Unicode binary again from > the list entries, unicode:character_to_binary/1 ? > > Thanks, > Henning > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From masklinn@REDACTED Thu Mar 22 15:57:43 2012 From: masklinn@REDACTED (Masklinn) Date: Thu, 22 Mar 2012 15:57:43 +0100 Subject: [erlang-questions] unicode:characters_to_list In-Reply-To: <4F6B3805.6070600@eonblast.com> References: <4F6B2BCE.5030707@eonblast.com> <0EB5D6FF-4B92-417A-A504-9D8734B5A03A@masklinn.net> <4F6B3805.6070600@eonblast.com> Message-ID: <5B45F6E6-480F-4690-A807-E00C2E4C7B06@masklinn.net> On 2012-03-22, at 15:32 , Henning Diedrich wrote: > Hi Masklinn, > > no question I am getting something wrong here. > > On 3/22/12 2:53 PM, Masklinn wrote: >> I don't understand why you're looking for UTF8 as the output of >> characters_to_list. > > That's what the doc leads me to believe I guess. > > "a list of integers representing unicode characters." "Unicode characters" is an alias for unicode code points: http://en.wikipedia.org/wiki/Code_point > 248 is not a valid unicode Bytecode. 248 is in fact a valid codepoint, it's U+00F8 which you can find in this table: http://en.wikibooks.org/wiki/Unicode/Character_reference/0000-0FFF (row 00F0, column 8). It's not a valid *utf8* byte. > Not sure if 50104 is the right one for the same character, but it would have to be two bytes and I would thus expect an integer higher 256. You're talking about UTF-8 encoded codepoints, "unicode characters" are not UTF. > If 248 is meant as actual number of the letter, which it is no matter the bitcode, then what is the right function to make a Unicode binary again from the list entries, unicode:character_to_binary/1 ? 1. There is no such thing as "a unicode binary" 2. Character_to_binary can be used to encode unicode strings to UTF8 (the default) or an other unicode transformation format. From freeakk@REDACTED Thu Mar 22 16:38:45 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Thu, 22 Mar 2012 19:38:45 +0400 Subject: [erlang-questions] unicode:characters_to_list In-Reply-To: <5B45F6E6-480F-4690-A807-E00C2E4C7B06@masklinn.net> References: <4F6B2BCE.5030707@eonblast.com> <0EB5D6FF-4B92-417A-A504-9D8734B5A03A@masklinn.net> <4F6B3805.6070600@eonblast.com> <5B45F6E6-480F-4690-A807-E00C2E4C7B06@masklinn.net> Message-ID: Hello, 254 is a code point. It is [195,190] in utf8 and [0,254] in utf16. [195,190] are bytes, not code points. Here you can see: (i18n@REDACTED)14> <<254/utf8, 0>>. <<195,190,0>> (i18n@REDACTED)15> <<254/utf16, 0>>. <<0,254,0>> (i18n@REDACTED)16> <<254/utf32, 0>>. <<0,0,0,254,0>> -- ? ?????????, ?????? ??????. Best regards, Uvarov Michael From freeakk@REDACTED Thu Mar 22 16:48:12 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Thu, 22 Mar 2012 19:48:12 +0400 Subject: [erlang-questions] unicode:characters_to_list In-Reply-To: References: <4F6B2BCE.5030707@eonblast.com> <0EB5D6FF-4B92-417A-A504-9D8734B5A03A@masklinn.net> <4F6B3805.6070600@eonblast.com> <5B45F6E6-480F-4690-A807-E00C2E4C7B06@masklinn.net> Message-ID: Also, in utf-8 each code point can be encoded using from 1 to 6 bytes. It is called as variable-width encoding. For example, bigints have the same model of encoding. It can help to save space when we using latin characters set and it "fixes" compatibility problems. This code is an example: io:write([<> || X <- lists:seq(1, 260)]). From masklinn@REDACTED Thu Mar 22 17:17:31 2012 From: masklinn@REDACTED (Masklinn) Date: Thu, 22 Mar 2012 17:17:31 +0100 Subject: [erlang-questions] unicode:characters_to_list In-Reply-To: References: <4F6B2BCE.5030707@eonblast.com> <0EB5D6FF-4B92-417A-A504-9D8734B5A03A@masklinn.net> <4F6B3805.6070600@eonblast.com> <5B45F6E6-480F-4690-A807-E00C2E4C7B06@masklinn.net> Message-ID: <299CD437-9A91-41BF-B87E-9F03761C885F@masklinn.net> On 2012-03-22, at 16:48 , Michael Uvarov wrote: > Also, in utf-8 each code point can be encoded using from 1 to 6 bytes. 1 to 4: Unicode is defined from 0 to 10FFFF, code-points beyond this range are to be considered ill-formed. UTF-8 can encode U+10FFFF in 4 bytes (with room to spare), and its definition was restricted to the same range as Unicode in RFC 3629 (the original definition did indeed allow for encoding 31 bit over up to 6 bytes). From jose.valim@REDACTED Thu Mar 22 19:31:09 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Thu, 22 Mar 2012 19:31:09 +0100 Subject: [erlang-questions] Setting the proper source in a module compiled via compile:forms Message-ID: Hello everyone, I am using compile:forms() to compile some forms and return me the binary. After I load the binary, I can access my code as expected, except the mymodule:module_info(:compile) returns the wrong source: > mymodule:module_info(compile) [{options,[]},{version,"4.8"},{source,"/Users/jose/Work"}] It seems it is returning my PWD as source. To fix the issue, I have already tried: -compile([{source,"real/path"}]). -file("real/path"). And passing a source option to compile:forms(). None of those worked. Does anyone know if there an attribute/option I can pass so Erlang will return the proper source? Thanks in advance, * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Thu Mar 22 20:08:03 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Thu, 22 Mar 2012 21:08:03 +0200 Subject: [erlang-questions] [ANNOUNCE] JSX 1.0 and 1.1rc In-Reply-To: References: <98BF8D67D2394D42A19341C94909253C@yahoo.ca> <-451521627403557216@unknownmsgid> Message-ID: Hello, Just run the test on Mac Mini: 2GHz i7, 4GB 1333 MHz (basic Min Server) with R15B ./configure --prefix=/usr/local/otp-R15B --enable-threads --enable-smp-support --enable-kernel-poll --enable-sctp --enable-hipe --disable-dynamic-ssl-lib --enable-darwin-64bit --enable-m64-build --without-javac 100 unique objects has been converted to_json and to_term about 10000 times (100 time per object) and jsx result was compared with mochijson2. Got the following results: to_term * jsx: 2231 ms, 0.223 ms per object * mochi: 2347 ms, 0.235 ms per object to_json * jsx: 5203 ms, 0.520 ms per object * mochi: 1511ms, 0.151 ms per object The problem with jsx is inefficient json_escape algorithm that causes alot of binary string copying. See the profile: {[{{jsx_utils,json_escape,2}, 3200, 347.542, 7.202}, {{jsx_utils,json_escape,3}, 218000, 0.000, 334.072}], { {jsx_utils,json_escape,3}, 221200, 347.542, 341.274}, % [{garbage_collect, 535, 5.287, 5.287}, {suspend, 107, 0.981, 0.000}, {{jsx_utils,json_escape,3}, 218000, 0.000, 334.072}]}. I've made an fix to it and now to_json * jsx 2157ms, 0.217 ms per object * mochi 1495ms, 0.150 ms per object but the memory usage dynamic fir jsx if much more better (like it is reported by erlang:memory(processes) * jsx from 1.921.751 to max 2.045.904 * mochi from 1.724.560 to max 4.498.640 As a conclusion: fixed jsx ;-) is 1.45 times slower then mochijson but 2x time more memory efficient + I like jsx semantic of object encoding/decoding. BTW, I'll send you a pull request for performance fix. - Dmitry On Mar 21, 2012, at 4:38 PM, Tristan Sloughter wrote: > I'll try to run a performance comparison your code and mochiweb in my test env. > > It would be great if you included jiffy (https://github.com/davisp/jiffy) too and please share if you do :) > > Tristan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Thu Mar 22 20:17:33 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 22 Mar 2012 20:17:33 +0100 Subject: [erlang-questions] Setting the proper source in a module compiled via compile:forms In-Reply-To: References: Message-ID: <4F6B7ACD.6000304@gmail.com> On 03/22/2012 07:31 PM, Jos? Valim wrote: > Hello everyone, > > I am using compile:forms() to compile some forms and return me the binary. > After I load the binary, I can access my code as expected, except the > mymodule:module_info(:compile) returns the wrong source: > > > mymodule:module_info(compile) > [{options,[]},{version,"4.8"},{source,"/Users/jose/Work"}] > > It seems it is returning my PWD as source. To fix the issue, I have > already tried: > > -compile([{source,"real/path"}]). > -file("real/path"). I think you must use `-file("real/path", Line).' (where Line is an integer literal). The version with arity 1 (just the file name) is probably just treated as an unknown user-defined attribute and ignored. /Richard From jose.valim@REDACTED Thu Mar 22 20:43:38 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Thu, 22 Mar 2012 20:43:38 +0100 Subject: [erlang-questions] Setting the proper source in a module compiled via compile:forms In-Reply-To: <4F6B7ACD.6000304@gmail.com> References: <4F6B7ACD.6000304@gmail.com> Message-ID: > > I think you must use `-file("real/path", Line).' (where Line is an integer > literal). The version with arity 1 (just the file name) is probably just > treated as an unknown user-defined attribute and ignored. Thanks! I have tried the following snippet, still with no success: 1> Code = [{attribute,1,file,{"real/path",1}},{attribute,1,module,demo}]. 2> Tuple = compile:forms(Code). 3> code:load_binary(demo, "real/path", element(3, Tuple)). 4> demo:module_info(). module_info still returns pwd as source. I must add I am using R15B. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Thu Mar 22 21:20:30 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 22 Mar 2012 21:20:30 +0100 Subject: [erlang-questions] Setting the proper source in a module compiled via compile:forms In-Reply-To: References: <4F6B7ACD.6000304@gmail.com> Message-ID: <4F6B898E.5000600@gmail.com> On 03/22/2012 08:43 PM, Jos? Valim wrote: > I think you must use `-file("real/path", Line).' (where Line is an > integer literal). The version with arity 1 (just the file name) is > probably just treated as an unknown user-defined attribute and ignored. > > > Thanks! I have tried the following snippet, still with no success: > > 1> Code = [{attribute,1,file,{"real/path",1}},{attribute,1,module,demo}]. > 2> Tuple = compile:forms(Code). > 3> code:load_binary(demo, "real/path", element(3, Tuple)). > 4> demo:module_info(). > > module_info still returns pwd as source. I must add I am using R15B. Sorry, I missed that point. The -file declarations are for getting the correct debugging info and compiler error messages. The source entry in module_info is a different thing. Looking in compile.erl, it seems that if you compile from forms, the source file is always set to "" (and then later, a call to dirname() turns this into ".", which is then passed to absname() before it is sent to beam_asm, so that's why you get your current directory). Perhaps you could patch compile.erl and submit your change to erlang-patches@REDACTED? /Richard From james@REDACTED Thu Mar 22 21:24:00 2012 From: james@REDACTED (james) Date: Thu, 22 Mar 2012 20:24:00 +0000 Subject: [erlang-questions] Erlang Jobs? Message-ID: <4F6B8A60.7050006@mansionfamily.plus.com> Are there any? I guess to be fair (since there aren't many of any sort around right now) - are there normally any (in London)? Weird that there seem to be none on TotalJobs. Even for small software houses trying it on. James From jose.valim@REDACTED Thu Mar 22 21:33:54 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Thu, 22 Mar 2012 21:33:54 +0100 Subject: [erlang-questions] Setting the proper source in a module compiled via compile:forms In-Reply-To: <4F6B898E.5000600@gmail.com> References: <4F6B7ACD.6000304@gmail.com> <4F6B898E.5000600@gmail.com> Message-ID: > > Sorry, I missed that point. The -file declarations are for getting the > correct debugging info and compiler error messages. The source entry in > module_info is a different thing. Looking in compile.erl, it seems that if > you compile from forms, the source file is always set to "" (and then > later, a call to dirname() turns this into ".", which is then passed to > absname() before it is sent to beam_asm, so that's why you get your current > directory). Perhaps you could patch compile.erl and submit your change to > erlang-patches@REDACTED? Thanks for the reply. I will work on a patch, what do you think about compile:forms(Forms, [{source,"path/to/source"]}) as API? -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Thu Mar 22 21:36:06 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 22 Mar 2012 21:36:06 +0100 Subject: [erlang-questions] Setting the proper source in a module compiled via compile:forms In-Reply-To: References: <4F6B7ACD.6000304@gmail.com> <4F6B898E.5000600@gmail.com> Message-ID: <4F6B8D36.4020109@gmail.com> On 2012-03-22 21:33, Jos? Valim wrote: > Sorry, I missed that point. The -file declarations are for getting > the correct debugging info and compiler error messages. The source > entry in module_info is a different thing. Looking in compile.erl, > it seems that if you compile from forms, the source file is always > set to "" (and then later, a call to dirname() turns this into ".", > which is then passed to absname() before it is sent to beam_asm, so > that's why you get your current directory). Perhaps you could patch > compile.erl and submit your change to erlang-patches@REDACTED > ? > > > Thanks for the reply. > I will work on a patch, what do you think about compile:forms(Forms, > [{source,"path/to/source"]}) as API? That is probably good. /Richard From Lennart.Ohman@REDACTED Thu Mar 22 21:42:14 2012 From: Lennart.Ohman@REDACTED (=?iso-8859-1?Q?Lennart_=D6hman?=) Date: Thu, 22 Mar 2012 21:42:14 +0100 Subject: [erlang-questions] Erlang Jobs? In-Reply-To: <4F6B8A60.7050006@mansionfamily.plus.com> References: <4F6B8A60.7050006@mansionfamily.plus.com> Message-ID: Hi James, A good idea is to check the "usual suspects" in the Erlang industry (difficult if you are not familiar with the Erlang community of course) if interested in Erlang jobs. In a quick search I came up with these. https://www.klarna.com/en/about-us/work-at-klarna http://www.campanja.com/join-us/ Best Regards Lennart -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of james Sent: den 22 mars 2012 21:24 To: erlang-questions@REDACTED Subject: [erlang-questions] Erlang Jobs? Are there any? I guess to be fair (since there aren't many of any sort around right now) - are there normally any (in London)? Weird that there seem to be none on TotalJobs. Even for small software houses trying it on. James _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From max.schubert@REDACTED Thu Mar 22 21:43:33 2012 From: max.schubert@REDACTED (Max Schubert) Date: Thu, 22 Mar 2012 16:43:33 -0400 Subject: [erlang-questions] Erlang Jobs? In-Reply-To: <4F6B8A60.7050006@mansionfamily.plus.com> References: <4F6B8A60.7050006@mansionfamily.plus.com> Message-ID: On Thu, Mar 22, 2012 at 4:24 PM, james wrote: > Are there any? > > I guess to be fair (since there aren't many of any sort around right now) - > are there normally any (in London)? > > Weird that there seem to be none on TotalJobs. ?Even for small software > houses trying it on. Check this out http://jobsearch.monster.com/search/?q=Erlang From daniel.widgren@REDACTED Thu Mar 22 21:57:34 2012 From: daniel.widgren@REDACTED (Daniel Widgren) Date: Thu, 22 Mar 2012 21:57:34 +0100 Subject: [erlang-questions] Erlang Jobs? In-Reply-To: References: <4F6B8A60.7050006@mansionfamily.plus.com> Message-ID: Den 22 mars 2012 21:43 skrev Max Schubert : > On Thu, Mar 22, 2012 at 4:24 PM, james wrote: >> Are there any? >> >> I guess to be fair (since there aren't many of any sort around right now) - >> are there normally any (in London)? >> >> Weird that there seem to be none on TotalJobs. ?Even for small software >> houses trying it on. > > Check this out > > http://jobsearch.monster.com/search/?q=Erlang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions You can always send a mail to Erlang Solutions in London. They have a office close to Liverpool Station. http://www.erlang-solutions.com/ They also have offices in Stockholm and Krakow. From mjtruog@REDACTED Thu Mar 22 22:14:01 2012 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 22 Mar 2012 14:14:01 -0700 Subject: [erlang-questions] Erlang Jobs? In-Reply-To: <4F6B8A60.7050006@mansionfamily.plus.com> References: <4F6B8A60.7050006@mansionfamily.plus.com> Message-ID: <4F6B9619.6070608@gmail.com> On 03/22/2012 01:24 PM, james wrote: > Are there any? > You can check the jobs list at "Erlang/OTP Professionals" linkedin group or in the newsfeed at http://planeterlang.org/. There is a separate jobs list at http://totally-erlang.com/ - Michael From hukl@REDACTED Thu Mar 22 22:19:36 2012 From: hukl@REDACTED (John-Paul Bader) Date: Thu, 22 Mar 2012 22:19:36 +0100 Subject: [erlang-questions] Erlang Jobs? In-Reply-To: <4F6B8A60.7050006@mansionfamily.plus.com> References: <4F6B8A60.7050006@mansionfamily.plus.com> Message-ID: <4F6B9768.90208@berlin.ccc.de> Hey, Wooga is also always looking for engineers with Erlang background. Check out http://www.wooga.com/jobs/offers/software-engineer/ ~ John james wrote: > Are there any? > > I guess to be fair (since there aren't many of any sort around right > now) - are there normally any (in London)? > > Weird that there seem to be none on TotalJobs. Even for small software > houses trying it on. > > James > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andre@REDACTED Thu Mar 22 22:41:04 2012 From: andre@REDACTED (=?ISO-8859-1?Q?Andr=E9_Graf?=) Date: Thu, 22 Mar 2012 22:41:04 +0100 Subject: [erlang-questions] Erlang Jobs? In-Reply-To: <4F6B9768.90208@berlin.ccc.de> References: <4F6B8A60.7050006@mansionfamily.plus.com> <4F6B9768.90208@berlin.ccc.de> Message-ID: I know there is a lot of Erlang at MIG (migcan.com) headquartered in London. They presented some of their work at the last year Erlang Factory in London. http://www.erlang-factory.com/conference/London2011/speakers/MarkusKern - Andr? On 22 March 2012 22:19, John-Paul Bader wrote: > Hey, > > > Wooga is also always looking for engineers with Erlang background. Check out > http://www.wooga.com/jobs/offers/software-engineer/ > > ~ John > > > james wrote: >> >> Are there any? >> >> I guess to be fair (since there aren't many of any sort around right >> now) - are there normally any (in London)? >> >> Weird that there seem to be none on TotalJobs. Even for small software >> houses trying it on. >> >> James >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From robert.virding@REDACTED Fri Mar 23 03:35:06 2012 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 23 Mar 2012 02:35:06 -0000 (GMT) Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: <4F6ADE1F.6010904@gmail.com> Message-ID: <7e02f8de-2680-49d1-8293-1d14a96f57d8@knuth> Got in late here but some comments: - I feel it is very important to separate client- and server side. I give quite a few courses and this concept can be difficult for many people. Especially those who come from the OO side who view modules as classes. So anything which hides this is bad. Yes, I know that they will learn but why make it more difficult. Also I like things to be explicit. - I personally don't really see the problem here, but then I am old school. Much of the boiler plate code is very short so specifying it for a "tool" will actually not save much typing, I still have to give the details and the code to be run in the server. My personal (biased) opinion is that has more to do with what they are used to rather than usefulness. Robert ----- Original Message ----- As OP I take the liberty to do a top post and try to summarise a few things at this point. I am not alone when it comes to having an urge to remove some of the boilerplate code. I guess we all want value for our money and when we have to do something that does not provide that we want to improve. If we did not have that craving we might as well be coding enterprise Java applications using dozens of frameworks ;-) I had a look at the e2project and it actually does remove some of the boilerplate that I find the most iritating, namely the tagging of messages from the API function to the handle_* function. It is a lot clearer in e2project where you reference a function in the e2_service:call - I like that a lot. The approach with parse transforms has its merits, but I think that the e2project can do away with most of the pains I have been feeling, so I think I will leave out the semi-black magic of parse transforms for now... it is a tool that should be used with caution as other posters have mentioned. The IDL approach I had in mind was nothing about CORBA - I should never have mentioned CORBA in my original mail, there are too many negative feelings associated to CORBA. What I like about IDLs is that it allows you to spec your API in a nice way and in many cases it will be possible to avoid writing boilerplate code (either through code generation or using a parse transform). Given the good point made about separating the client and the server side of things I think that what is needed is simply to spec up the API function (who doesn't do that already?!?!) and then try out e2project. The spec will give me a slap over my fingers if I do not provide the function (I like that) and e2project seems to minimise the tedious boilerplate stuff from standard gen_server without limiting my ability to express what I need. A big thanks to all posters - I am glad I asked the question! Cheers, Torben On 21/3/12 21:10 , Tim Watson wrote:
On 21 Mar 2012, at 19:22, Garrett Smith wrote:
This may come down to personal taste. I don't like having to dig around in parse transform code to understand what my module looks like.
Neither do I - that's why I simply run 'escript `evm site`/parse_trans/ebin/parse_trans_pp.beam ebin/.beam' over the generated beam to see what it looks like at source level. ;)
eunit is a great example. If you really love eunit, then I'm not going to convince you :)
Common test and PropEr do exactly the same thing. I'm not sure what you're expected testing frameworks to do instead!?
- Hiding the client-server distinction in Erlang is a terrible disservice
I agree with this wholeheartedly, but don't see what it has to do with code generation.
If you consolidate both the client and server functions into one function, you've obfuscated an important distinction.
I never suggested doing that at all. Not that I mean to be prickly, as we're all part of the same community and whatnot, but nobody suggested separating both functions. What the OP suggested was that you could define the interface declaratively - e.g., without writing the code directly - and that was what I was responding to. As I mentioned later on, in this particular gen_server case I actually think your approach is probably cleaner and more appropriate, but it's good to separate these points and refine the discussion I think.
IMO, e2 solves the problem of "too much boiler plate". It's really
easy and requires zero magic.
Here's a "gen_server" equivalent in e2:
https://github.com/gar1t/e2v2/blob/master/examples/ping/src/ping_server.erl
This is pretty cool though. :)
You might object to the separation of "ping" and "handle_msg" -- it
appears to be just one function, so why break it up into two pieces?
The problem is that it's not one function -- it's definitely *two*
very separate pieces of code.
Absolutely and a good point!
When you write a gen_server style module, you're writing code that
support client-server interactions. You have, in the same module, code
that is called by the "client" and code that is called by the
"server". If you don't grok this, you're missing the entire point of
this type of module.
Code that hides this difference -- e.g. a parse transform that gloms
the client and server functions into one -- is IMO a Really Bad Idea.
I don't necessarily agree with this, as the parse transform can be applied to a -spec which is potentially as intention revealing as a hand written function. I do take your point about them being two separate functions and perhaps in this particular case (for gen_server) you are actually correct and having two hand coded parts is actually better. I don't buy 'code-gen is bad' as a general argument (which perhaps you weren't going so far as to make anyway), and I do see your point in this instance.
I'm certainly not arguing against code generation. I think Erlang's parse transform scheme is *very* good -- flexible enough to enable elegant solutions but with enough complexity to scare off casual "meta programmers". But -- I think there should be a very clear payoff for using a parse transform. I don't think the discussion here, which is "boilerplate" is a good application for that.
I think it depends on how much boilerplate you're looking at. As an OCaml programmer I have seen a *lot* of work going into removing boilerplate code, and the same in Haskell (although there I am less experienced) and overall I think the payoff you're describing is a value judgement based on the negative impact of using code generation. Generated code is brittle - there's no two ways about it. But it doesn't prevent you from testing or documenting, nor does it have to make your code opaque and difficult to understand, so long as it is not overused - a sprinkle here and there rather than a bucket load.
A good example of a value-add parse transform IMO is modlib, which lets you create mods for inets httpd withou causing your brain to explode [1].
As an example of how this client/server difference is important,
consider form validation in a web app. There are two places you can
run code to validate form input -- you can validate it in the browser
using JavaScript, or you can send the form data to the server. Web
developers may opt for browser validation to avoid the expense of
sending data to the server -- or they might prefer it on the server.
Either way, it's an important consideration.
This same dynamic applies to gen_server style modules. If you stick
with it, I think you'll appreciate the separateness of client and
server facing code.
As for the boilerplate, I couldn't agree with you more!
Garrett
P.S. I'm giving a talk on e2 at the SF Factory in a couple weeks,
where I'll get into this in more details. Also, the e2 docs and github
project are in slight disarray at the moment, but will be put in order
this weekend!
[1] https://github.com/gar1t/modlib
-- http://www.linkedin.com/in/torbenhoffmann _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions
-------------- next part -------------- An HTML attachment was scrubbed... URL: From spawn.think@REDACTED Fri Mar 23 10:24:59 2012 From: spawn.think@REDACTED (Ahmed Omar) Date: Fri, 23 Mar 2012 10:24:59 +0100 Subject: [erlang-questions] Erlang Jobs? In-Reply-To: References: <4F6B8A60.7050006@mansionfamily.plus.com> <4F6B9768.90208@berlin.ccc.de> Message-ID: As Michael suggested, Erlang/OTP professional group would be a good place to start Jobs feed :This is automatically generated from Jobs posted on LinkedIn that has Erlang in the keywords http://www.linkedin.com/groups?jobs=&gid=90878&trk=anet_ug_jobs Job discussions : These are more relevent Erlang/OTP jobs posted by members of the group (Managers or recruiters) http://www.linkedin.com/groups?careerDiscussion=&gid=90878 It could be useful for you also to join EUG-UK (Erlang Users' Group, UK) http://www.linkedin.com/groups?gid=3661421 Also you can use indeed to do a search like this http://www.indeed.co.uk/jobs?q=erlang&l=london On Thu, Mar 22, 2012 at 10:41 PM, Andr? Graf wrote: > I know there is a lot of Erlang at MIG (migcan.com) headquartered in > London. They presented some of their work at the last year Erlang > Factory in London. > http://www.erlang-factory.com/conference/London2011/speakers/MarkusKern > > - Andr? > > On 22 March 2012 22:19, John-Paul Bader wrote: > > Hey, > > > > > > Wooga is also always looking for engineers with Erlang background. Check > out > > http://www.wooga.com/jobs/offers/software-engineer/ > > > > ~ John > > > > > > james wrote: > >> > >> Are there any? > >> > >> I guess to be fair (since there aren't many of any sort around right > >> now) - are there normally any (in London)? > >> > >> Weird that there seem to be none on TotalJobs. Even for small software > >> houses trying it on. > >> > >> James > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Mar 23 11:30:32 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 23 Mar 2012 13:30:32 +0300 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second Message-ID: I need to load large CSV file into memory very fast. I've tried to use erlang parser, but my results were very bad (in fact file:read_line is very slow), so I've tried to make a NIF for it. Target speed is 1 microsecond per line. My CSV has very strict format: only numbers, no quoting, \n in the end. Also I moved parsing of integers and date into NIF. My results are here: http://github.com/maxlapshin/csv_reader and I get only 15 microseconds: 4,5 seconds for 300K lines CSV. Currently I use fgets to read line by line from file. Maybe it is a bad idea and I should use mmap or implement 1MB buffer for read? From bengt.kleberg@REDACTED Fri Mar 23 11:42:57 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 23 Mar 2012 11:42:57 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: Message-ID: <1332499378.4759.16.camel@seasc1137> Greetings, With a working NIF you probably do not need to consider file:read_file/1 and binary:split/2 on . Otherwise that is an alternative to read_line/1 bengt On Fri, 2012-03-23 at 11:30 +0100, Max Lapshin wrote: > I need to load large CSV file into memory very fast. > > I've tried to use erlang parser, but my results were very bad (in fact > file:read_line is very slow), so I've tried to make a NIF for it. > Target speed is 1 microsecond per line. > > > My CSV has very strict format: only numbers, no quoting, \n in the > end. Also I moved parsing of integers and date into NIF. > > My results are here: http://github.com/maxlapshin/csv_reader and I get > only 15 microseconds: 4,5 seconds for 300K lines CSV. > > Currently I use fgets to read line by line from file. Maybe it is a > bad idea and I should use mmap or implement 1MB buffer for read? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ulf@REDACTED Fri Mar 23 12:29:15 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 23 Mar 2012 12:29:15 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: Message-ID: <54661907-F200-4D92-81C0-3D08EF3B78E3@feuerlabs.com> open_port({spawn, "/bin/cat " ++ File}, [{line, MaxLen}, binary]) will pour the file, one line at a time, into your message queue. :) Wicked fast, but no flow control. Fredrik Svahn made a flow-control hack for stdin back in 2008, but I don't know (a) if that's applicable to you or (b) if it ever made it into the OTP source somehow. http://erlang.org/pipermail/erlang-bugs/2008-December/001136.html Otherwise, opening the file in [raw, binary] mode and using re:split() ought to work reasonably well. file:read_line() is indeed dreadfully slow, but is OTOH extremely nice in distributed embedded systems, as it supports IO redirection across nodes. BR, Ulf W On 23 Mar 2012, at 11:30, Max Lapshin wrote: > I need to load large CSV file into memory very fast. > > I've tried to use erlang parser, but my results were very bad (in fact > file:read_line is very slow), so I've tried to make a NIF for it. > Target speed is 1 microsecond per line. > > > My CSV has very strict format: only numbers, no quoting, \n in the > end. Also I moved parsing of integers and date into NIF. > > My results are here: http://github.com/maxlapshin/csv_reader and I get > only 15 microseconds: 4,5 seconds for 300K lines CSV. > > Currently I use fgets to read line by line from file. Maybe it is a > bad idea and I should use mmap or implement 1MB buffer for read? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Fri Mar 23 13:31:38 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 23 Mar 2012 13:31:38 +0100 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: <7e02f8de-2680-49d1-8293-1d14a96f57d8@knuth> References: <7e02f8de-2680-49d1-8293-1d14a96f57d8@knuth> Message-ID: <4F6C6D2A.7010902@gmail.com> On 23/3/12 3:35 , Robert Virding wrote: > Got in late here but some comments: > > - I feel it is very important to separate client- and server side. I > give quite a few courses and this concept can be difficult for many > people. Especially those who come from the OO side who view modules as > classes. So anything which hides this is bad. Yes, I know that they > will learn but why make it more difficult. Also I like things to be > explicit. After hearing the arguments I am in agreement with this split. > > - I personally don't really see the problem here, but then I am old > school. Much of the boiler plate code is very short so specifying it > for a "tool" will actually not save much typing, I still have to give > the details and the code to be run in the server. For me the biggest annoyance is that in most cases the code in your API function simply wraps the incoming arguments with some tag and that tag has to be matched with a clause in the handle_call function. As far as I can see there is no way to enforce a match unless you start creating records for each message you want to send to the internal server... hmmm, maybe that is not so bad after all... I will toy a bit with that thought. There is a bit of flexibility in the e2project since you do not have to write one big handle_call function - you may choose to have several handle_* functions. It is a bit clearer, but there is no enforcement since the function you specify as handler is dispatched through the e2_service:call/2 function. Looking at this again there seems to be no easy fix to this "hole" since there is a dispatching involved here. The only thing that gives you some warning is to start using records for the messages, but it seems at bit dramatic to do that. Putting specs on the internal protocol will not help due to the dispatching. > > My personal (biased) opinion is that has more to do with what they are > used to rather than usefulness. There is definitely some of that in this equation. Cheers, Torben > > Robert > > ------------------------------------------------------------------------ > > As OP I take the liberty to do a top post and try to summarise a > few things at this point. > > I am not alone when it comes to having an urge to remove some of > the boilerplate code. I guess we all want value for our money and > when we have to do something that does not provide that we want to > improve. If we did not have that craving we might as well be > coding enterprise Java applications using dozens of frameworks ;-) > > I had a look at the e2project and it actually does remove some of > the boilerplate that I find the most iritating, namely the tagging > of messages from the API function to the handle_* function. It is > a lot clearer in e2project where you reference a function in the > e2_service:call - I like that a lot. > > The approach with parse transforms has its merits, but I think > that the e2project can do away with most of the pains I have been > feeling, so I think I will leave out the semi-black magic of parse > transforms for now... it is a tool that should be used with > caution as other posters have mentioned. > > The IDL approach I had in mind was nothing about CORBA - I should > never have mentioned CORBA in my original mail, there are too many > negative feelings associated to CORBA. > What I like about IDLs is that it allows you to spec your API in a > nice way and in many cases it will be possible to avoid writing > boilerplate code (either through code generation or using a parse > transform). > > Given the good point made about separating the client and the > server side of things I think that what is needed is simply to > spec up the API function (who doesn't do that already?!?!) and > then try out e2project. The spec will give me a slap over my > fingers if I do not provide the function (I like that) and > e2project seems to minimise the tedious boilerplate stuff from > standard gen_server without limiting my ability to express what I > need. > > A big thanks to all posters - I am glad I asked the question! > > Cheers, > Torben > > On 21/3/12 21:10 , Tim Watson wrote: > > On 21 Mar 2012, at 19:22, Garrett Smith wrote: > > > > This may come down to personal taste. I don't like having > to dig > around in parse transform code to understand what my > module looks > like. > > > Neither do I - that's why I simply run 'escript `evm > site`/parse_trans/ebin/parse_trans_pp.beam ebin/.beam' > over the generated beam to see what it looks like at source > level. ;) > > eunit is a great example. If you really love eunit, then > I'm not going > to convince you :) > > > Common test and PropEr do exactly the same thing. I'm not sure > what you're expected testing frameworks to do instead!? > > - Hiding the client-server distinction in Erlang > is a terrible disservice > > > I agree with this wholeheartedly, but don't see what > it has to do with code generation. > > > If you consolidate both the client and server functions > into one > function, you've obfuscated an important distinction. > > > I never suggested doing that at all. Not that I mean to be > prickly, as we're all part of the same community and whatnot, > but nobody suggested separating both functions. What the OP > suggested was that you could define the interface > declaratively - e.g., without writing the code directly - and > that was what I was responding to. As I mentioned later on, in > this particular gen_server case I actually think your approach > is probably cleaner and more appropriate, but it's good to > separate these points and refine the discussion I think. > > IMO, e2 solves the problem of "too much boiler > plate". It's really > > easy and requires zero magic. > > > Here's a "gen_server" equivalent in e2: > > > https://github.com/gar1t/e2v2/blob/master/examples/ping/src/ping_server.erl > > > > This is pretty cool though. :) > > > You might object to the separation of "ping" and > "handle_msg" -- it > > appears to be just one function, so why break it > up into two pieces? > > > The problem is that it's not one function -- it's > definitely *two* > > very separate pieces of code. > > > Absolutely and a good point! > > > > When you write a gen_server style module, you're > writing code that > > support client-server interactions. You have, in > the same module, code > > that is called by the "client" and code that is > called by the > > "server". If you don't grok this, you're missing > the entire point of > > this type of module. > > > Code that hides this difference -- e.g. a parse > transform that gloms > > the client and server functions into one -- is IMO > a Really Bad Idea. > > > > I don't necessarily agree with this, as the parse > transform can be applied to a -spec which is > potentially as intention revealing as a hand written > function. I do take your point about them being two > separate functions and perhaps in this particular case > (for gen_server) you are actually correct and having > two hand coded parts is actually better. I don't buy > 'code-gen is bad' as a general argument (which perhaps > you weren't going so far as to make anyway), and I do > see your point in this instance. > > > I'm certainly not arguing against code generation. I think > Erlang's > parse transform scheme is *very* good -- flexible enough > to enable > elegant solutions but with enough complexity to scare off > casual "meta > programmers". > > But -- I think there should be a very clear payoff for > using a parse > transform. I don't think the discussion here, which is > "boilerplate" > is a good application for that. > > > I think it depends on how much boilerplate you're looking at. > As an OCaml programmer I have seen a *lot* of work going into > removing boilerplate code, and the same in Haskell (although > there I am less experienced) and overall I think the payoff > you're describing is a value judgement based on the negative > impact of using code generation. Generated code is brittle - > there's no two ways about it. But it doesn't prevent you from > testing or documenting, nor does it have to make your code > opaque and difficult to understand, so long as it is not > overused - a sprinkle here and there rather than a bucket load. > > A good example of a value-add parse transform IMO is > modlib, which > lets you create mods for inets httpd withou causing your > brain to > explode [1]. > > As an example of how this client/server difference > is important, > > consider form validation in a web app. There are > two places you can > > run code to validate form input -- you can > validate it in the browser > > using JavaScript, or you can send the form data to > the server. Web > > developers may opt for browser validation to avoid > the expense of > > sending data to the server -- or they might prefer > it on the server. > > Either way, it's an important consideration. > > > This same dynamic applies to gen_server style > modules. If you stick > > with it, I think you'll appreciate the > separateness of client and > > server facing code. > > > As for the boilerplate, I couldn't agree with you > more! > > > Garrett > > > P.S. I'm giving a talk on e2 at the SF Factory in > a couple weeks, > > where I'll get into this in more details. Also, > the e2 docs and github > > project are in slight disarray at the moment, but > will be put in order > > this weekend! > > > [1] https://github.com/gar1t/modlib > > > > -- > http://www.linkedin.com/in/torbenhoffmann > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From gordon@REDACTED Fri Mar 23 13:54:00 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Fri, 23 Mar 2012 12:54:00 +0000 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: Message-ID: Max There is a csv parser for RFC 4180 compliant csv files which is now being maintained by Eric Merritt: https://github.com/afiniate/erfc_parsers/tree/master/src Gordon On 23 March 2012 10:30, Max Lapshin wrote: > I need to load large CSV file into memory very fast. > > I've tried to use erlang parser, but my results were very bad (in fact > ?file:read_line is very slow), so I've tried to make a NIF for it. > Target speed is 1 microsecond per line. > > > My CSV has very strict format: only numbers, no quoting, \n in the > end. Also I moved parsing of integers and date into NIF. > > My results are here: http://github.com/maxlapshin/csv_reader and I get > only 15 microseconds: ?4,5 seconds for 300K lines CSV. > > Currently I use fgets to read line by line from file. Maybe it is a > bad idea and I should use mmap or implement 1MB buffer for read? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From watson.timothy@REDACTED Fri Mar 23 13:55:51 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 23 Mar 2012 12:55:51 +0000 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: Message-ID: On 23 Mar 2012, at 12:54, Gordon Guthrie wrote: > Max > > There is a csv parser for RFC 4180 compliant csv files which is now > being maintained by Eric Merritt: > https://github.com/afiniate/erfc_parsers/tree/master/src > This appears to read the whole file into memory, so probably not very space efficient for dealing with large files. > Gordon > > On 23 March 2012 10:30, Max Lapshin wrote: >> I need to load large CSV file into memory very fast. >> >> I've tried to use erlang parser, but my results were very bad (in fact >> file:read_line is very slow), so I've tried to make a NIF for it. >> Target speed is 1 microsecond per line. >> >> >> My CSV has very strict format: only numbers, no quoting, \n in the >> end. Also I moved parsing of integers and date into NIF. >> >> My results are here: http://github.com/maxlapshin/csv_reader and I get >> only 15 microseconds: 4,5 seconds for 300K lines CSV. >> >> Currently I use fgets to read line by line from file. Maybe it is a >> bad idea and I should use mmap or implement 1MB buffer for read? >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Gordon Guthrie > CEO hypernumbers > > http://hypernumbers.com > t: hypernumbers > +44 7776 251669 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ponton@REDACTED Fri Mar 23 14:03:28 2012 From: ponton@REDACTED (Tomasz Maciejewski) Date: Fri, 23 Mar 2012 14:03:28 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: Message-ID: W dniu 23 marca 2012 11:30 u?ytkownik Max Lapshin napisa?: > Currently I use fgets to read line by line from file. Maybe it is a > bad idea and I should use mmap or implement 1MB buffer for read? mmap is the fastest way to read lines is you don't much care about portability. -- Tomasz Maciejewski From max.lapshin@REDACTED Fri Mar 23 14:10:53 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 23 Mar 2012 16:10:53 +0300 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: Message-ID: On Fri, Mar 23, 2012 at 4:54 PM, Gordon Guthrie wrote: > Max > > There is a csv parser for RFC 4180 compliant csv files which is now > being maintained by Eric Merritt: > https://github.com/afiniate/erfc_parsers/tree/master/src > It is great, but: 1) it doesn't support header. Usually first line is a header 2) it is very strict to structure of CSV. It may change and it is impossible to tie to index of column, only by its name 3) it is incredibly slow: NIF: 4812, RFC: 38371 It is 10 times slower than my code. From andy.richards.iit@REDACTED Fri Mar 23 15:28:09 2012 From: andy.richards.iit@REDACTED (Andy Richards) Date: Fri, 23 Mar 2012 14:28:09 +0000 Subject: [erlang-questions] Linux ODBC socket recieve_msg woes Message-ID: Hi all, I'm experiencing performance issues with erlang odbc when running on Linux. I have a simple test application which sends 100 queries from my test gen_server via erlang odbc to sqlserver. When running this test on Windows the test completes in about 200 millis however the exact same test on Linux take about 4 seconds! I added trace logging to the odbc port driver odbcserver.c and can see that it also takes approx 200 millis to execute all messages and send them back to Erlang however the function receive_msg & receive_msg_part adds approx 3.6 seconds receiving messages from the socket? I'm running OTP R15B which I've compiled myself on our Redhat EL 6 server. Has anyone come experienced socket performance issues with ODBC under Linux ? Many thank, Andy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Mar 23 16:22:19 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 23 Mar 2012 08:22:19 -0700 Subject: [erlang-questions] current xml parsers Message-ID: Dear list, does someone have recent considerations on xml parsers in terms of memory footprint, parsing speed and stability? The ones I'm aware of are xmerl, erlsom [1] and the driver used in ejabberd (which unfortunately is GPL). I don't care about DTD validation. Thank you, r. [1] http://erlsom.sourceforge.net/erlsom.htm -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukasp.p017@REDACTED Fri Mar 23 18:12:53 2012 From: lukasp.p017@REDACTED (Lukas P) Date: Fri, 23 Mar 2012 18:12:53 +0100 Subject: [erlang-questions] ssl timing issue in R15B? Message-ID: Hello. I think that I have been hit by a SSL timing issue in R15B. My functional test (= SSL/TCP client) crahes my server unless the test contains a sleep. The tested server implements a TCP based, line oriented protocol. Switch to SSL can be started with a "STARTTLS\r\n" command. The functional test starts a TCP connection and upgrades it to SSL: upgrade_connection_to_ssl(Socket) -> SendResult = gen_tcp:send(Socket, "STARTTLS\r\n"), %timer:sleep(10), % workaround for a SSL timing bug SslConnectResult = case SendResult of ok -> ssl:connect(Socket, [{active, false}, {packet, line}, list], 2000); {error, Reason1} -> {error, Reason1} end, ... After the server receives "STARTTLS\r\n", it performs ssl:ssl_accept on the socket: inet:setopts(Socket, [{active, false}]), ssl:ssl_accept(Socket, [{cacertfile, ?SSL_CACERT_PATH}, {certfile, ?SSL_CERT_PATH}, {keyfile, ?SSL_KEY_PATH}], infinity), First iteration of the test works right, i.e. connection is upgraded to SSL and futher commands are performed over the secure connection, and the connection is closed at the end of the test. The second iteration of the test fails with {error,closed}, and the server crashes with (see below). The problem manifests also with R14B04. Best regards, Lukas =ERROR REPORT==== 23-Mar-2012::16:57:04 === ** State machine <0.102.0> terminating ** Last message in was {tcp,#Port<0.2140>, [22,3,1,0,103,1,0,0,99,3,1,79,108,157,80,51,140, 130,232,83,15,128,38,143,164,30,203,247,208,66, 205,100,0,158,84,84,251,121,182,34,167,164,117, 32,98,92,131,219,248,65,185,199,148,131,224,46, 71,92,29,58,19,77,33,81,29,143,208,128,239,202, 40,141,122,55,5,240,0,28,0,255,0,57,0,56,0,53,0, 22,0,19,0,10]} ** When State == hello ** Data == {state,server, {#Ref<0.0.0.315>,<0.98.0>}, gen_tcp,tcp,tcp_closed,tcp_error,"localhost",8145, #Port<0.2140>, {ssl_options,[],verify_none, {#Fun,[]}, false,false,undefined,1,"../cert/server.crt", undefined,"../cert/server.key",undefined,undefined, undefined,"../cert/server.crt",undefined,undefined, [<<0,57>>, <<0,56>>, <<0,53>>, <<0,22>>, <<0,19>>, <<0,10>>, <<0,51>>, <<0,50>>, <<0,47>>, <<0,5>>, <<0,4>>, <<0,21>>, <<0,9>>], #Fun,true,18446744073709551900, false,[],undefined}, {socket_options,list,line,0,0,false}, {connection_states, {connection_state, {security_parameters, <<0,0>>, 0,0,0,0,0,0,0,0,0,0,undefined,undefined, undefined,undefined}, undefined,undefined,undefined,0,undefined, undefined,undefined}, {connection_state, {security_parameters,undefined,0,undefined, undefined,undefined,undefined,undefined, undefined,undefined,undefined,undefined, undefined,undefined, <<79,108,157,80,49,103,60,230,250,78,4,225,74, 55,176,84,223,46,97,9,131,243,182,44,18,190, 82,140,50,114,198,215>>, undefined}, undefined,undefined,undefined,undefined, undefined,undefined,undefined}, {connection_state, {security_parameters, <<0,0>>, 0,0,0,0,0,0,0,0,0,0,undefined,undefined, undefined,undefined}, undefined,undefined,undefined,0,undefined, undefined,undefined}, {connection_state, {security_parameters,undefined,0,undefined, undefined,undefined,undefined,undefined, undefined,undefined,undefined,undefined, undefined,undefined, <<79,108,157,80,49,103,60,230,250,78,4,225,74, 55,176,84,223,46,97,9,131,243,182,44,18,190, 82,140,50,114,198,215>>, undefined}, undefined,undefined,undefined,undefined, undefined,undefined,undefined}}, [],<<>>,<<>>, {{<<1,35,69,103,137,171,205,239,254,220,186,152,118,84,50, 16,0,0,0,0,0,0,0,0,189,33,38,64,88,194,157,155,162, 159,131,31,18,9,13,106,73,121,160,91,114,138,162,8, 220,130,60,229,78,11,83,20,0,0,12,139,24,32,103,93, 227,42,206,176,196,177,231,20,0,0,12,169,169,42,128, 92,250,14,14,51,66,235,147,0,0,0,0,0>>, <<1,35,69,103,137,171,205,239,254,220,186,152,118,84, 50,16,240,225,210,195,0,0,0,0,0,0,0,0,189,33,38,64, 88,194,157,155,162,159,131,31,18,9,13,106,73,121,160, 91,114,138,162,8,220,130,60,229,78,11,83,20,0,0,12, 139,24,32,103,93,227,42,206,176,196,177,231,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}, {<<1,35,69,103,137,171,205,239,254,220,186,152,118,84,50, 16,0,0,0,0,0,0,0,0,189,33,38,64,88,194,157,155,162, 159,131,31,18,9,13,106,73,121,160,91,114,138,162,8, 220,130,60,229,78,11,83,20,0,0,12,139,24,32,103,93, 227,42,206,176,196,177,231,20,0,0,12,169,169,42,128, 92,250,14,14,51,66,235,147,0,0,0,0,0>>, <<1,35,69,103,137,171,205,239,254,220,186,152,118,84, 50,16,240,225,210,195,0,0,0,0,0,0,0,0,189,33,38,64, 88,194,157,155,162,159,131,31,18,9,13,106,73,121, 160,91,114,138,162,8,220,130,60,229,78,11,83,20,0,0, 12,139,24,32,103,93,227,42,206,176,196,177,231,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}}, [],8207, {session,undefined,undefined, <<48,130,1,225,48,130,1,74,2,9,0,205,239,129,202,242, 227,65,139,48,13,6,9,42,134,72,134,247,13,1,1,5,5,0, 48,53,49,11,48,9,6,3,85,4,6,19,2,67,90,49,19,48,17, 6,3,85,4,8,19,10,83,111,109,101,45,83,116,97,116, 101,49,17,48,15,6,3,85,4,10,19,8,107,101,114,105, 111,100,101,118,48,30,23,13,49,50,48,51,50,51,48,56, 53,57,49,55,90,23,13,49,51,48,51,50,51,48,56,53,57, 49,55,90,48,53,49,11,48,9,6,3,85,4,6,19,2,67,90,49, 19,48,17,6,3,85,4,8,19,10,83,111,109,101,45,83,116, 97,116,101,49,17,48,15,6,3,85,4,10,19,8,107,101,114, 105,111,100,101,118,48,129,159,48,13,6,9,42,134,72, 134,247,13,1,1,1,5,0,3,129,141,0,48,129,137,2,129, 129,0,182,253,69,113,8,80,65,199,149,56,241,149,198, 94,125,152,3,192,219,198,133,171,8,242,113,152,14, 224,26,118,175,206,96,59,123,81,96,146,254,92,103, 250,224,113,228,227,140,94,205,242,108,108,225,220, 172,105,37,99,239,63,19,230,190,16,83,250,45,164,14, 164,224,143,179,72,236,132,28,137,144,46,24,159,152, 59,190,244,148,39,15,43,175,131,21,160,59,192,73, 137,91,70,221,121,198,143,80,106,191,0,130,145,216, 225,250,251,222,206,206,54,152,61,175,23,219,186, 129,100,155,215,2,3,1,0,1,48,13,6,9,42,134,72,134, 247,13,1,1,5,5,0,3,129,129,0,180,149,241,170,86,108, 137,169,31,219,111,106,43,252,244,75,48,216,124,147, 58,28,203,23,140,245,31,244,37,118,253,191,137,206, 133,193,86,9,20,240,89,32,211,234,245,53,100,147,63, 226,151,82,155,101,60,118,42,34,69,131,57,153,80,55, 117,141,243,94,159,60,126,188,217,46,136,159,152,19, 16,183,226,145,231,58,53,129,168,13,44,89,91,102, 186,146,95,214,176,97,185,153,244,5,180,96,0,6,95, 41,98,187,201,109,163,178,66,237,127,136,212,246, 211,75,111,62,17,99,37,192>>, undefined,undefined,undefined,false,undefined}, 20498,ssl_session_cache,undefined,undefined,false, undefined,undefined, {'RSAPrivateKey','two-prime', 128499485073324344640350663317080381746144189677328093733733818765201879966770636497130667046738574044411458592269074082861288662250357992465839585570810117635103294613076680912273164585128543701693490299026933362559737361586906453308454424637758883964954103891002397921983210876969561998056460034621497973719, 65537, 21142407304967520763185699719976162417697984303380210182505329321530919506258873206716205849596137203730545462874977887841879787677885930585152635171125398678264154737861602152192965103330738539906753643553236543949552679329988944514780005135078240922914424542757568267105592140259268438701514178052748528137, 12299880358440743957425671124001695744432582454465136498819080587852714118502292601888074183728961424894245227783505611233127931778382492533775859423958717, 10447214227180842802291362278552737574221362829069766492840722789131867364093400612139963591599836595829208168869120879147173486297418106712618120046723107, 8600926454775507180385181748794569671725229850667993278115886067108872585298733011265798918045239241650570979734539193914762452622788850082211271277006577, 5030482618477285446570769480206429034434037975448290602477603324177706309545683249420950471643743892069858586523686582895881009309056307025498420957847339, 1302418139310728515246909787183496859699357629051756870224989169538451919989096507242998571432647801516438823004677134366483529583874264214103765858188241, asn1_NOVALUE}, {'DHParameter', 179769313486231590770839156793787453197860296048756011706444423684197180216158519368947833795864925541502180565485980503646440548199239100050792877003355816639229553136239076508735759914822574862575007425302077447712589550957937778424442426617334727629299387668709205606050270810842907692932019128194467627007, 2,asn1_NOVALUE}, undefined,undefined,#Ref<0.0.0.294>,undefined,0,<<>>, true, {false,first}, false, {[],[]}, false} ** Reason for termination = ** {badarg,[{erlang,size, [[22,3,1,0,103,1,0,0,99,3,1,79,108,157,80,51,140,130,232, 83,15,128,38,143,164,30,203,247,208,66,205,100,0,158,84, 84,251,121,182,34,167,164,117,32,98,92,131,219,248,65, 185,199,148,131,224,46,71,92,29,58,19,77,33,81,29,143, 208,128,239,202,40,141,122,55,5,240,0,28,0,255,0,57,0, 56,0,53,0,22,0,19,0,10]]}, {ssl_record,get_tls_records_aux,2}, {ssl_connection,next_tls_record,2}, {ssl_connection,handle_info,3}, {gen_fsm,handle_msg,7}, {proc_lib,init_p_do_apply,3}]} From attila.r.nohl@REDACTED Fri Mar 23 18:16:10 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Fri, 23 Mar 2012 18:16:10 +0100 Subject: [erlang-questions] ssl timing issue in R15B? In-Reply-To: References: Message-ID: 2012/3/23 Lukas P : > Hello. > > I think that I have been hit by a SSL timing issue in R15B. My > functional test (= SSL/TCP client) crahes my server unless the test > contains a sleep. > > The tested server implements a TCP based, line oriented protocol. > Switch to SSL can be started with a "STARTTLS\r\n" command. > > The functional test starts a TCP connection and upgrades it to SSL: > > upgrade_connection_to_ssl(Socket) -> > ? ?SendResult = gen_tcp:send(Socket, "STARTTLS\r\n"), > ? ?%timer:sleep(10), % workaround for a SSL timing bug > ? ?SslConnectResult = case SendResult of > ? ? ? ?ok -> > ? ? ? ? ? ?ssl:connect(Socket, [{active, false}, {packet, line}, list], 2000); > ? ? ? ?{error, Reason1} -> > ? ? ? ? ? ?{error, Reason1} > ? ?end, > ? ?... > > After the server receives "STARTTLS\r\n", it performs ssl:ssl_accept > on the socket: > > ? ?inet:setopts(Socket, [{active, false}]), Don't you need a {reuseaddr, true} option here? The default is false. From andy.richards.iit@REDACTED Fri Mar 23 18:16:59 2012 From: andy.richards.iit@REDACTED (Andy Richards) Date: Fri, 23 Mar 2012 17:16:59 +0000 Subject: [erlang-questions] Linux ODBC socket recieve_msg woes In-Reply-To: References: Message-ID: Fixed. I found this forum message from back in 2004. http://erlang.org/pipermail/erlang-questions/2004-July/012808.html Editing odbcserver.c and disabling nagel's algorithm (approx 40ms on Redhat 6) on the socket solved the problem. I wonder why this was never added to odbcserver.c in the past? I still see a packet being sent and a ack in my tcpdump inbetween the initial query msg being sent and my resultset being sent back to erlang which I have no idea what for? However overall performance has improved greatly. Andy. On Friday, 23 March 2012, Andy Richards wrote: > Hi all, > > I'm experiencing performance issues with erlang odbc when running on Linux. I have a simple test application which sends 100 queries from my test gen_server via erlang odbc to sqlserver. When running this test on Windows the test completes in about 200 millis however the exact same test on Linux take about 4 seconds! > > I added trace logging to the odbc port driver odbcserver.c and can see that it also takes approx 200 millis to execute all messages and send them back to Erlang however the function receive_msg & receive_msg_part adds approx 3.6 seconds receiving messages from the socket? > > I'm running OTP R15B which I've compiled myself on our Redhat EL 6 server. Has anyone come experienced socket performance issues with ODBC under Linux ? > > Many thank, > > Andy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Fri Mar 23 18:28:49 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 23 Mar 2012 17:28:49 +0000 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: Message-ID: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> The problem doesn't appear to be anything to do with the speed of read_line, but rather one of binary splitting/processing instead. Consider this example, which simply uses file:read_line to get a list of lines from the 300k csv file: t4@REDACTED:csv_reader $ ./benchmark.erl example.csv ./benchmark.erl:23: Warning: type transform_fun() is unused ./benchmark.erl:149: Warning: variable 'Pattern' is unused read_line time: 1365 read size: 300000 t4@REDACTED:csv_reader $ But just using binary:split/3 on the individual lines, without even processing the column cells, slows down considerably: t4@REDACTED:csv_reader $ ./benchmark.erl example.csv ./benchmark.erl:23: Warning: type transform_fun() is unused read_line time: 12654 read size: 300000 t4@REDACTED:csv_reader $ This is rather unfortunately slow, as binary processing is one of the things that Erlang is supposed to be exceptionally good at. The parsing code (which is part of a larger example I was playing with) boils down to these functions (with a few setup functions and record definitions omitted for brevity): main([Path]) -> {ok, Fd} = file:open(Path, [raw, binary, {read_ahead, 1024 * 1024}]), try T1 = erlang:now(), Res = parse(make_parser(Fd), start), T2 = erlang:now(), io:format("read_line time: ~p~n", [timer:now_diff(T2, T1) div 1000]), io:format("read size: ~p~n", [length(Res)]) catch Ex:R -> io:format("~p~n", [erlang:get_stacktrace()]), throw({Ex, R}) after file:close(Fd) end. parse(P=#csv_parser{ iodevice=Fd, header=true }, start) -> file:read_line(Fd), parse(P, []); parse(P=#csv_parser{ iodevice=Fd, delimiter=Pattern }, Acc) -> case file:read_line(Fd) of {ok, Data} -> % Record = process(P, binary:split(Data, Pattern, [global])), Record = binary:split(Data, Pattern, [global]), parse(P, [Record|Acc]); eof -> Acc; Other -> throw(Other) end. *************************** So it doesn't look like file:read_line is really the cause of the slowness. I've also tried several other processing examples (to actually generate the records and convert to floats etc) and of course these just add processing time. I do wonder if there is a more efficient manner or splitting the line oriented binaries and processing the data, but I am currently assuming that binary:split/3 is a pretty efficient mechanism. I might investigate this a little further at some point, if I get time. Cheers, Tim On 23 Mar 2012, at 13:10, Max Lapshin wrote: > On Fri, Mar 23, 2012 at 4:54 PM, Gordon Guthrie wrote: >> Max >> >> There is a csv parser for RFC 4180 compliant csv files which is now >> being maintained by Eric Merritt: >> https://github.com/afiniate/erfc_parsers/tree/master/src >> > > It is great, but: > > 1) it doesn't support header. Usually first line is a header > 2) it is very strict to structure of CSV. It may change and it is > impossible to tie to index of column, only by its name > 3) it is incredibly slow: NIF: 4812, RFC: 38371 > It is 10 times slower than my code. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Fri Mar 23 18:31:45 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 23 Mar 2012 20:31:45 +0300 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> Message-ID: On Fri, Mar 23, 2012 at 9:28 PM, Tim Watson wrote: > The problem doesn't appear to be anything to do with the speed of read_line, but rather one of binary splitting/processing instead. Consider this example, which simply uses file:read_line to get a list of lines from the 300k csv file: > > read_line time: 1365 > read size: 300000 But to be honest 1365 milliseconds is 10 times slower than plain C variant. > t4@REDACTED:csv_reader $ > > But just using binary:split/3 on the individual lines, without even processing the column cells, slows down considerably: > > t4@REDACTED:csv_reader $ ./benchmark.erl example.csv > ./benchmark.erl:23: Warning: type transform_fun() is unused > read_line time: 12654 And naive splitting in C gives 3 times faster. Can you launch my code on your machine to normalize numbers? From ebegumisa@REDACTED Fri Mar 23 18:59:00 2012 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Fri, 23 Mar 2012 20:59:00 +0300 Subject: [erlang-questions] Erlang is the best choice for building commercial application servers In-Reply-To: <4F5FB8BE.8040902@mansionfamily.plus.com> References: <4F5D1361.9060604@gmail.com> <4F5E1FD8.4060306@meetinghouse.net> <4F5FB8BE.8040902@mansionfamily.plus.com> Message-ID: You're right, COM is still around (though probably only because MS still has legacy code using it). Indeed, if you really want to write new components using COM you can. But the thing is, the vast majority of the COM components for business applications I came across were written mainly in ActiveX VB (so-called "classic" VB), not C++. C++ COM in business software were one-off components. Component development in classic VB was sooooo different from VB.net that upgrading them was essentially re-writing them. There was no continuity. Sure, you could insist on keeping your old components around but you wouldn't be able to extend them to take advantage of new platform features. More importantly, there was no guarantee that you could even compile them in the future (i.e. no guarantee that your old copy of VS would still work on newer versions of Windows). The mass migrations that occurred around that period had nothing to do with fashion. Most teams avoid migrations. Dev teams migrate coz they feel they have no choice. So yes, there was DEFINITELY a feeling of having the rug pulled -- a feeling that the new "shiny" thing was being imposed by the vendor. To the extent that I recall there were a few petitions floating around. Anyways, my point was, relying of infrastructure from a big name vendor guarantees you nothing in terms of continuity, yet this seems to be a common belief amongst managers of business software development teams. - Edmond - On Wed, 14 Mar 2012 00:14:38 +0300, james wrote: > >There are many businesses that invested heavily in the previous > >iteration of MS development infrastructure (COM-driven Visual Studio 6 > >and related tools), and then suddenly had the rug pulled from > >underneath them in 2002 when .Net appeared and they were expected to > >rewrite/migrate much of their code (I worked for such a victim, and > >gathered many now-worthless skills). > > When did COM stop working? When did you have to throw away working COM > code because you want to use CLR? When did an ability to write modular > C++ applications not apply on, say, Linux? > > You're a fashion victim. It can happen to anyone, and probably will, > eventually. > > Suggesting that there was any rug pulling is bizarre; some greener grass > turned up over the fence. Some people will call it progress. > If you are working with or for people who 'expect to migrate' just > because something shiny showed up, that's a problem and it can happen to > you with nearly anything. How long do you think 'cloud' will last? > > James -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From anthonym@REDACTED Fri Mar 23 19:03:32 2012 From: anthonym@REDACTED (Anthony Molinaro) Date: Fri, 23 Mar 2012 11:03:32 -0700 Subject: [erlang-questions] Erlang Jobs? In-Reply-To: <4F6B8A60.7050006@mansionfamily.plus.com> References: <4F6B8A60.7050006@mansionfamily.plus.com> Message-ID: <20120323180332.GC61597@alumni.caltech.edu> OpenX is always looking for Erlang engineers. We have offices in Los Angeles, New York, and London. The office in London is small and doesn't actually do development, but we might be able to swing a development position or two there. If there is anyone interested please forward me a resume. -Anthony On Thu, Mar 22, 2012 at 08:24:00PM +0000, james wrote: > Are there any? > > I guess to be fair (since there aren't many of any sort around right > now) - are there normally any (in London)? > > Weird that there seem to be none on TotalJobs. Even for small > software houses trying it on. > > James > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ------------------------------------------------------------------------ Anthony Molinaro From ebegumisa@REDACTED Fri Mar 23 19:22:03 2012 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Fri, 23 Mar 2012 21:22:03 +0300 Subject: [erlang-questions] Avoiding boilerplate code when using gen_server In-Reply-To: <9B6B38AF-02ED-4CD8-B32E-782535CBD078@cs.otago.ac.nz> References: <4F69C016.6080800@gmail.com> <9B6B38AF-02ED-4CD8-B32E-782535CBD078@cs.otago.ac.nz> Message-ID: > > On 22/03/2012, at 12:48 AM, Torben Hoffmann wrote: >> The question was: why don't you use an IDL (Interface Definition >> Language) approach instead of writing all that code that has to match >> up in order to work? >> >> I have tried searching for it, but since IDL is so tightly coupled with >> CORBA I didn't really find anything but the ic application from the >> Erlang/OTP distribution.\ > > There are two very different concepts here. > (1) "An interface definition language *APPROACH*." > (2) CORBA IDL. > > CORBA IDL is language independent as long as the language you are > pretending > to be independent of is C++ or a close relative. It is about as bad a > fit > for Erlang data structures as you would expect it to be. This is not to > say > that Erlang cannot handle the data structures passed through a CORBA > interface. > It can. What I am saying is that CORBA doesn't handle Erlang data > structures. > Incidentally, I attempted an XPCOM-Erlang bridge to try and marry Erlang with the Mozilla Framework (XPCOM is similar to MSCOM/CORBA with IDLs etc). Eventually I realised that keeping to two sides completely separate and doing application-specific message passing was waaaaaay easier. - Edmond - On Thu, 22 Mar 2012 01:53:47 +0300, Richard O'Keefe wrote: > CORBA IDL is by no means the only, nor by a very long time the first, > interface definition language, not even if we are talking about > interfaces > for distributed programming. The first time I used remote procedure > call was > on a Xerox 1108 running Interlisp... > > There's no reason why some sort of "little language" for specifying > gen_server > stuff could not be devised. It might even be something that could be > plugged > in using a parse transform. > > But step 1 is to be very very clear about what is repetitive and what is > not. > > Step 2 is to consider whether it can be done using higher-order > functions. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From hd2010@REDACTED Fri Mar 23 19:49:58 2012 From: hd2010@REDACTED (Henning Diedrich) Date: Fri, 23 Mar 2012 19:49:58 +0100 Subject: [erlang-questions] unicode:characters_to_list In-Reply-To: References: <4F6B2BCE.5030707@eonblast.com> <0EB5D6FF-4B92-417A-A504-9D8734B5A03A@masklinn.net> <4F6B3805.6070600@eonblast.com> <5B45F6E6-480F-4690-A807-E00C2E4C7B06@masklinn.net> Message-ID: <4F6CC5D6.2040601@eonblast.com> On 3/22/12 4:38 PM, Michael Uvarov wrote: > Hello, > > 254 is a code point. It is [195,190] in utf8 and [0,254] in utf16. > [195,190] are bytes, not code points. > Here you can see: > > (i18n@REDACTED)14> <<254/utf8, 0>>. > <<195,190,0>> > (i18n@REDACTED)15> <<254/utf16, 0>>. > <<0,254,0>> > (i18n@REDACTED)16> <<254/utf32, 0>>. > <<0,0,0,254,0>> > Nice, thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Fri Mar 23 19:56:03 2012 From: hd2010@REDACTED (Henning Diedrich) Date: Fri, 23 Mar 2012 19:56:03 +0100 Subject: [erlang-questions] unicode:characters_to_list In-Reply-To: <5B45F6E6-480F-4690-A807-E00C2E4C7B06@masklinn.net> References: <4F6B2BCE.5030707@eonblast.com> <0EB5D6FF-4B92-417A-A504-9D8734B5A03A@masklinn.net> <4F6B3805.6070600@eonblast.com> <5B45F6E6-480F-4690-A807-E00C2E4C7B06@masklinn.net> Message-ID: <4F6CC743.6070400@eonblast.com> On 3/22/12 3:57 PM, Masklinn wrote: >> 248 is not a valid unicode Bytecode. > 248 is in fact a valid codepoint, Yes, that clarifies it, thanks. Plus that Unicode codepoints can then be presented in different bit patterns, e.g. UTF-8, UTF-16, UTF-32. I was confusing the Unicode codepoint and the UTF-8 bitpattern. Thanks, Henning -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Fri Mar 23 23:11:39 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 23 Mar 2012 22:11:39 +0000 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> Message-ID: So if you are willing to loose some space efficiency (which you would with mmap anyway) then reading the entire binary into memory is a lot faster: t4@REDACTED:csv_reader $ ./benchmark2.erl example.csv ./benchmark2.erl:13: Warning: variable 'Bin' is unused read_file time: 298 t4@REDACTED:csv_reader $ So now the only question is can you split it in acceptable time!? I've started playing with this and can chunk up the file into lines in around 729, but now it's a question of splitting up the lines into fields. What I'm doing up until now is still sequential of course. As this problem actually appears to be cpu bound rather than io bound, we could start looking at how to break up the problem most effectively to utilise many cores. It's not going to be anywhere near fast enough for your needs I suspect, but I'd like to know if it's possible to squeeze a 300k record file through in a couple of seconds with a bit of careful work. Although I haven't finished with results collection and transforming the split binaries into records, I'm hoping this is achievable: t4@REDACTED:csv_reader $ ./csv_reader.erl example.csv ./csv_reader.erl:10: Warning: record evt is unused ./csv_reader.erl:25: Warning: type transform_fun() is unused ./csv_reader.erl:104: Warning: variable 'Size' is unused read_chunk time: 2765 t4@REDACTED:csv_reader $ Which (slightly faster before) is based on (incomplete): #!/usr/bin/env escript %%! -pa ebin -mode(compile). -record(evt, { key,date,time, l_1,l_2,l_3,l_4,l_5,l_6,l_7,l_8,l_9,l_10, l_11,l_12,l_13,l_14,l_15,l_16,l_17,l_18,l_19,l_20, l_21,l_22,l_23,l_24,l_25,l_26,l_27,l_28,l_29,l_30, l_31,l_32,l_33,l_34,l_35,l_36,l_37,l_38,l_39,l_40, last }). -record(csv_field, { name :: atom() | string(), type = 'binary' :: atom(), %% TODO: use erl_types:type() converter :: fun((binary()) -> term()) }). -type transform_fun() :: fun((term()) -> term()). -record(csv_parser, { io_device :: file:io_device(), collector :: pid(), input_file :: file:name(), header :: boolean(), strict = true :: boolean(), size :: non_neg_integer(), fields :: [#csv_field{} | constant()], buffer_size = 16 * 1024 :: non_neg_integer(), transform = fun(E) -> E end :: transform_fun(), delimiter = binary:compile_pattern(<<",">>) :: binary:cp(), supported_newlines = binary:compile_pattern([<<"\n">>, <<"\r\n">>]) :: binary:cp() }). main([Path]) -> %% TODO: don't be so naive try T1 = erlang:now(), chunker(make_parser(Path)), T2 = erlang:now(), io:format("read_chunk time: ~p~n", [timer:now_diff(T2, T1) div 1000]) catch Ex:R -> io:format("~p~n", [erlang:get_stacktrace()]), throw({Ex, R}) % after % file:close(Fd) end. make_parser(F) -> %% TODO: write a function that takes a 'typed record' and %% generates the csv_field mappings for us using erl_types or whatever... Fields = [ #csv_field{ name="KEY" }, #csv_field{ name="Date" }, %% NB: we will worry about converting dates later on... #csv_field{ name="Time" }, %% NB: we will worry about converting time(s) later on... #csv_field{ name="l_1", type=float }, #csv_field{ name="l_2", type=float }, #csv_field{ name="l_3", type=float }, #csv_field{ name="l_4", type=float }, #csv_field{ name="l_5", type=float }, #csv_field{ name="l_6", type=float }, #csv_field{ name="l_7", type=float }, #csv_field{ name="l_8", type=float }, #csv_field{ name="l_9", type=float }, #csv_field{ name="l_10", type=float }, #csv_field{ name="l_11", type=float }, #csv_field{ name="l_12", type=float }, #csv_field{ name="l_13", type=float }, #csv_field{ name="l_14", type=float }, #csv_field{ name="l_15", type=float }, #csv_field{ name="l_16", type=float }, #csv_field{ name="l_17", type=float }, #csv_field{ name="l_18", type=float }, #csv_field{ name="l_19", type=float }, #csv_field{ name="l_20", type=float }, #csv_field{ name="l_21", type=float }, #csv_field{ name="l_22", type=float }, #csv_field{ name="l_23", type=float }, #csv_field{ name="l_24", type=float }, #csv_field{ name="l_25", type=float }, #csv_field{ name="l_26", type=float }, #csv_field{ name="l_27", type=float }, #csv_field{ name="l_28", type=float }, #csv_field{ name="l_29", type=float }, #csv_field{ name="l_30", type=float }, #csv_field{ name="l_31", type=float }, #csv_field{ name="l_32", type=float }, #csv_field{ name="l_33", type=float }, #csv_field{ name="l_34", type=float }, #csv_field{ name="l_35", type=float }, #csv_field{ name="l_36", type=float }, #csv_field{ name="l_37", type=float }, #csv_field{ name="l_38", type=float }, #csv_field{ name="l_39", type=float }, #csv_field{ name="l_40", type=float }, #csv_field{ name="Last" } ], #csv_parser{ input_file=F, header=true, strict=true, transform=fun(L) -> list_to_tuple(['evt'] ++ L) end, size=length(Fields), fields=Fields }. %parse(P=#csv_parser{ iodevice=Fd, buffer_size=Size }) -> % parse(P, file:read(Fd, Size), 0, undefined, []). %% the butcher process chops up the binary into chunks chunker(P=#csv_parser{ input_file=F, buffer_size=Size }) -> {ok, Fd} = file:open(F, [raw, binary, {read_ahead, 1024 * 1024}]), CPid = spawn(fun loop/0), read(P#csv_parser{ io_device=Fd, collector=CPid }, start, 0, <<>>, []). loop() -> receive {ok, _Pid, _X} -> % io:format("Got ~p parts!~n", [length(X)]) loop() end. read(P=#csv_parser{ io_device=Fd, buffer_size=Size }, start, Idx, Rem, Pids) -> read(P, file:read(Fd, Size), Idx, Rem, Pids); read(P=#csv_parser{ io_device=Fd, buffer_size=Size, delimiter=Delim, collector=Parent, supported_newlines=NL }, {ok, Chunks}, Idx, Rem, Pids) -> % io:format("read ~p characters~n", [size(Chunk)]), Lines = binary:split(Chunks, NL, [global]), {Data, [Remaining]} = lists:split(length(Lines) - 1, Lines), Pid = spawn(fun() -> [H|T] = Data, Sz = size(H), %% TODO: fix this code so it properly deals with %% left over data between processed segments %% i.e., where is \n in relation to Rem .... WorkingSet = case size(Rem) of X when X < Sz -> [<>|T]; _ -> [Rem|Data] end, [ return(Parent, binary:split(L, Delim, [global])) || L <- WorkingSet, is_binary(L) ] end), read(P, file:read(Fd, Size), Idx + 1, Remaining, [{Pid, Idx}|Pids]); read(_P, eof, _, _Rem, _Pids) -> %% TODO: in strict mode, fail unless size(Rem) == 0 ok. return(Collector, Matches) -> Collector ! {ok, self(), Matches}. collect_results(Results, []) -> array:to_list(Results); collect_results(Results, Pids) -> receive {ok, Pid, Data} -> {value, {Pid, Idx}, RemainingPids} = lists:keytake(Pid, 1, Pids), collect_results(array:set(Idx, Data, Results), RemainingPids); Other -> throw(Other) end. process(P=#csv_parser{ strict=Strict, fields=Fields }, Data) -> case lists:foldl(fun process_field/2, {Fields, [], Strict}, Data) of {[], Acc, _} -> (P#csv_parser.transform)(Acc); {MissingFields, Result, _} when is_list(MissingFields) -> case Strict of true -> throw({parse_error, {unexpected_eol, MissingFields}}); false -> Result end end. process_field(_E, {[], Acc, false}) -> Acc; process_field(E, {[], _Acc, true}) -> throw({parse_error, {expected_eol, E}}); process_field(E, {[#csv_field{ type=float }|Rest], Acc, Strict}) -> {Rest, [list_to_float(binary_to_list(E))|Acc], Strict}; process_field(E, {[#csv_field{ type=binary }|Rest], Acc, Strict}) -> {Rest, [E|Acc], Strict}; process_field(E, {[#csv_field{ type=list }|Rest], Acc, Strict}) -> {Rest, [binary_to_list(E)|Acc], Strict}. On 23 Mar 2012, at 19:45, Max Lapshin wrote: From watson.timothy@REDACTED Fri Mar 23 23:15:03 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 23 Mar 2012 22:15:03 +0000 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> Message-ID: <2F0ACE91-F13B-46D6-871C-0AA347581644@gmail.com> Forgot to cc the list.... On 23 Mar 2012, at 19:27, Tim Watson wrote: > On 23 Mar 2012, at 19:19, Tim Watson wrote: > >> On 23 Mar 2012, at 17:31, Max Lapshin wrote: >> >>> On Fri, Mar 23, 2012 at 9:28 PM, Tim Watson wrote: >>>> The problem doesn't appear to be anything to do with the speed of read_line, but rather one of binary splitting/processing instead. Consider this example, which simply uses file:read_line to get a list of lines from the 300k csv file: >>>> >>>> read_line time: 1365 >>>> read size: 300000 >>> >>> But to be honest 1365 milliseconds is 10 times slower than plain C variant. >> >> Sure I appreciate that. >> >>> >>>> t4@REDACTED:csv_reader $ >>>> >>>> But just using binary:split/3 on the individual lines, without even processing the column cells, slows down considerably: >>>> >>>> t4@REDACTED:csv_reader $ ./benchmark.erl example.csv >>>> ./benchmark.erl:23: Warning: type transform_fun() is unused >>>> read_line time: 12654 >>> >>> And naive splitting in C gives 3 times faster. Can you launch my code >>> on your machine to normalize numbers? >> >> t4@REDACTED:csv_reader $ ./csv_bench.erl example.csv >> Load csv_reader: ok >> Load time: 5300 >> t4@REDACTED:csv_reader $ ./csv_bench.erl example.csv >> Load csv_reader: ok >> Load time: 5213 >> t4@REDACTED:csv_reader $ evm info >> R15B compiled for i386-apple-darwin10.8.0, 64bit >> > > And again with a lower erts version: > > t4@REDACTED:csv_reader $ ./csv_bench.erl example.csv > Load csv_reader: ok > Load time: 5518 > t4@REDACTED:csv_reader $ evm info > R14B01 compiled for i386-apple-darwin10.5.0, 64bit > t4@REDACTED:csv_reader $ > > Hardware/OS profile: Apple Macbook Pro running OS-X 10.6.8 (Snow Leopard), 2.8GHz Intel Core2 Duo, 8Gb RAM. From toby@REDACTED Sat Mar 24 04:07:02 2012 From: toby@REDACTED (Toby Thain) Date: Fri, 23 Mar 2012 23:07:02 -0400 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: Message-ID: <4F6D3A56.3050203@telegraphics.com.au> On 23/03/12 9:03 AM, Tomasz Maciejewski wrote: > W dniu 23 marca 2012 11:30 u?ytkownik Max Lapshin > napisa?: >> Currently I use fgets to read line by line from file. Maybe it is a >> bad idea and I should use mmap or implement 1MB buffer for read? > > mmap is the fastest way to read lines is you don't much care about portability. > Even Windows offers mmap functionality. Of course machines without a MMU will not, but that doesn't seem a likely venue for this operation (they probably won't have fast disks and network either :) --Toby From w.a.de.jong@REDACTED Sat Mar 24 08:22:05 2012 From: w.a.de.jong@REDACTED (Willem de Jong) Date: Sat, 24 Mar 2012 08:22:05 +0100 Subject: [erlang-questions] current xml parsers In-Reply-To: References: Message-ID: Hi Roberto, The main reason why I wrote erlsom was not memory footprint or speed, but the fact that I didn't understand the documentation of xmerl and I didn't like the output format. I thought (and still think) that it is a waste to translate a generic structure like xml to another generic structure (like a DOM tree or something similar). You must then extract your information from that new structure, while you could have done it in one pass. Both the SAX parser and the "data binder" mode that erlsom offers support this, in a way. There has been a period when the erlsom SAX parser was a lot faster than xmerl, but nowadays xmerl also offers a SAX parser, and I think the difference in speed is small. Also, both erlsom and xmerl allow you to parse in a kind of streaming way, so neither forces you to load the whole XML document in memory, and both support working directly on binaries. Note that, when you are looking at the performance of the parser you should also consider the effort that is required to get the information that you actually want from the output of the parser. If you use the erlsom "data mapper" mode, you will get nice records that are easy to access. If you use the SAX mode of erlsom or xmerl, you can take the information that you want out of the stream as you go along - actually a very nice model, I think. By the way, please note that the best place to get erlsom nowadays is github, not sourceforge. The latest version is in https://github.com/willemdj/erlsom Regards, Willem On Fri, Mar 23, 2012 at 4:22 PM, Roberto Ostinelli wrote: > Dear list, > > does someone have recent considerations on xml parsers in terms of memory > footprint, parsing speed and stability? > > The ones I'm aware of are xmerl, erlsom [1] and the driver used in > ejabberd (which unfortunately is GPL). > > I don't care about DTD validation. > > Thank you, > > r. > > [1] http://erlsom.sourceforge.net/erlsom.htm > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Mar 24 12:36:11 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 24 Mar 2012 14:36:11 +0300 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> Message-ID: On Sat, Mar 24, 2012 at 2:11 AM, Tim Watson wrote: > So if you are willing to loose some space efficiency (which you would with mmap anyway) then reading the entire binary into memory is a lot faster: > Tim, looks like your solutions is twice faster due to using threads. Am I right? From masklinn@REDACTED Sat Mar 24 13:00:21 2012 From: masklinn@REDACTED (Masklinn) Date: Sat, 24 Mar 2012 13:00:21 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <4F6D3A56.3050203@telegraphics.com.au> References: <4F6D3A56.3050203@telegraphics.com.au> Message-ID: On 2012-03-24, at 04:07 , Toby Thain wrote: > On 23/03/12 9:03 AM, Tomasz Maciejewski wrote: >> W dniu 23 marca 2012 11:30 u?ytkownik Max Lapshin >> napisa?: >>> Currently I use fgets to read line by line from file. Maybe it is a >>> bad idea and I should use mmap or implement 1MB buffer for read? >> >> mmap is the fastest way to read lines is you don't much care about portability. >> > > Even Windows offers mmap functionality. Yep, although the exact options may differ all modern OS can memory-map files, there really is no reason *not* to use it. Especially when running on 64b OS, where there is no risk to mmap a file bigger than VMEM. But due to Erlang semantics, I believe mmap would have to be supported by the VM itself to work correctly with an ideal API (so that it can be integrated well with binary handling e.g. as a special kind of refc binary) otherwise you have to use a file-type interface ? la emmap: https://github.com/krestenkrab/emmap From watson.timothy@REDACTED Sat Mar 24 17:37:35 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 24 Mar 2012 16:37:35 +0000 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> Message-ID: <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> On 24 Mar 2012, at 11:36, Max Lapshin wrote: > On Sat, Mar 24, 2012 at 2:11 AM, Tim Watson wrote: >> So if you are willing to loose some space efficiency (which you would with mmap anyway) then reading the entire binary into memory is a lot faster: >> > > Tim, looks like your solutions is twice faster due to using threads. Am I right? Yes, although I still haven't quite hit the sweet spot with this approach. You can sequentially chunk your way through the file (using file:read/2 on a file opened with [raw, binary]) using 64k chunks and break these up into lines in around 533ms. Breaking up the 300k individual lines on the comma and collecting the results requires a bit more though about how best to split the work up, so currently there's not really that much of an improvement. I am playing around with this to see what's possible though, as it's an interesting problem. From max.lapshin@REDACTED Sat Mar 24 17:45:29 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 24 Mar 2012 19:45:29 +0300 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> Message-ID: I'm trying to use file:read from erlang combined with compiled scanf pattern or some regex. There are two ideas: 1) combine two binary:split will give double memory walking. I think it is a good idea to parse memory once 2) intermediate creation of binaries for all those floats (12 millions of them in this example) is also evil. Perhaps line parsing should be combined with float extraction in the same manner as decode_packet does. -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Sat Mar 24 23:14:37 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 24 Mar 2012 22:14:37 +0000 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> Message-ID: <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> On 24 Mar 2012, at 16:45, Max Lapshin wrote: > I'm trying to use file:read from erlang combined with compiled scanf pattern or some regex. There are two ideas: > 1) combine two binary:split will give double memory walking. I think it is a good idea to parse memory once Actually there's a double problem here. In a CSV file it is possible that a delimiter might be present 'inside' one of the fields, providing the contents of the cell are in quotes. For example: >> 1,2,16,John Snow,"Flat A, 128, Watling Street", etc.... So just doing binary:split(Bin, binary:compile_pattern(<<",">>), [global]) isn't enough. I will take a look how binary:split/3 is implemented, but I'm starting to think that you really do need to drop into C to make this fast enough. Maybe for your data set you don't care about this possibility (because you have some control over the inputs) but for a general purpose CSV parser, this has to be handled and requires too much introspection of the data to efficient over 70MiB and more if written in pure Erlang. With file:read_line and binary:split, I can get the data out (without float/int conversion) sequentially in about 10 seconds. It's not particularly useful to parallelise the binary:split work as for the most part it isn't taking a long time per operation, but performs a vast number of them which slows down. Spawning for each processing line isn't really very smart as even in SMP mode the amount of churn due to scheduling seems likely to be prohibitive. Even if you hit the sweet spot with the right number of worker processes (1 per cpu +/- 1 for example) you've got the additional work of reconstituting the data in the correct order which takes up a lot of time. Now if I manually chunk my way through the file, using an optimal segment size (on my mac, 16 * 1024 seems to be best) with a sizeable read_ahead of 1024 * 1024, then I can get through the whole file and binary:split(Bin, CommaPattern, [global]) on each chunk, in an average of 2300ms. Of course this is too slow, is ignoring newlines at the moment (unlike the other tests) and I've removed the code for dealing with fields that sit across the segment boundaries (which slows things down also): read(P=#csv_parser{ io_device=Fd, buffer_size=Size }, start, Idx, Rem) -> read(P, file:read(Fd, Size), Idx, Rem); read(P=#csv_parser{ io_device=Fd, buffer_size=Size, delimiter=Delim, collector=Collector, supported_newlines=NL }, {ok, Chunks}, Idx, Rem) -> binary:split(Chunks, Delim, [global]), read(P, file:read(Fd, Size), Idx + 1, <<>>); read(_P, eof, _, _Rem) -> %% in strict mode, fail unless size(Rem) == 0 ok. >>>> t4@REDACTED:csv_reader $ ./csv_reader.erl example.csv Starting read.... read_chunk time: 2376 t4@REDACTED:csv_reader $ Now if I switch the split to use binary:matches instead, and look for either ',' or a newline, we get a big performance improvement: t4@REDACTED:csv_reader $ ./csv_reader.erl example.csv Starting read.... read_chunk time: 953 t4@REDACTED:csv_reader $ That's getting much faster, *but* we're not doing any hard work of parsing yet and we're not dealing with 'cross-segment' parts, which I'll look at next. > 2) intermediate creation of binaries for all those floats (12 millions of them in this example) is also evil. Perhaps line parsing should be combined with float extraction in the same manner as decode_packet does. Like I said I think that with the approach I mentioned above (using nicely segmented file:read calls and binary:matches) we've got the i/o subsystem and splitting of fields to go as fast as they're likely to go without dropping into C. Now we get into the more cpu intensive parts - taking the {Location, Size} data from binary:matches and combined with your segment Idx working out the correct offset to use in calling binary:part, plus reconstituting stuff that sits across two segments and then finally converting binary to other data types for the final results. If I could make these parts happen in 2 seconds to get combined throughput of 3 seconds I'd be pleased, but I'm not sure it'll be possible. And even then, your desired time of 1 second isn't looking at all likely for even this 300k record file, let alone a larger 1million line file. It is possible that applying some parallelism to this will help, but the timings we're after are so small that I'm not sure. Spawning per split is of course a waste of time, but it is possible that having a worker process per core that does binary:matches and parses might help. The difficulty here is that you have to deal with the data sitting across two segments before you can pass it off to a worker, because you need the previous 'remainder' if any - I suppose you could 'pass this in' along with the data. If this could be made to work efficiently, then you might also benefit from a parallel collector process taking the completed/parsed records from the workers and storing them so they're ready for retrieval. Again, I'm not convinced all of this could be done in pure Erlang in less than 3 - 4 seconds overall. Perhaps putting ets into the mix might provide some speedup, as I would guess that a shared table can be atomically accessed faster than the processes can exchange data via a mailbox. I'm going to see how fast this can be made to go in pure Erlang (allowing for ETS if it helps) just as a fun exercise. I don't know how much spare time I'll have to do it though and I'm convinced a solution written in C is the right answer for 1million records per second, and to do that a segment at a time approach that does almost all the work in C including all the type conversion is probably the way to go. In that case I would probably do the whole i/o part in C too, using a similar series of buffered read operations rather than line oriented fgets calls or things like that. I do not think that mmap or even reading all the file into memory will be of any use, as you mentioned earlier, the problem is really cpu bound. From dmkolesnikov@REDACTED Sat Mar 24 23:32:24 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sun, 25 Mar 2012 00:32:24 +0200 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> Message-ID: <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> Hello, You have raise a freaking interesting issue! I have decided to invest my time here and share binary parsing techniques used by me. In the nutshell, I was capable to parse a line in 2.17 micro seconds using native erlang implementation. I have tried to productize code and put it here: https://github.com/fogfish/csv The algorithm is very simple 1. load a whole file into memory 2. shard the file into N chunks 3. parse each chunk in parallel, parsing is based on binary scan (no binary:split, etc) 4. merge results. The test was run on Mac Mini server with R15B Erlang R15B (erts-5.9) [source] [64-bit] [smp:8:8] [async-threads:0] [hipe] [kernel-poll:false] Since, you have not shared a details of csv file I used the following setups: * set.txt contains 300K rows, each row has 16 fields of 3 hex digit each * set2.txt contains 300K rows, each row has 48 fields of 3 hex digit each Here is the results: csv_example:run("priv/set.txt", 80). {{lines,300000}, {size,18000000}, {read_ms,132.269}, {parse_ms,652.865}, {line_us,2.1762166666666665}} 2> csv_example:run("priv/set2.txt", 80). {{lines,300000}, {size,54000000}, {read_ms,204.259}, {parse_ms,1860.286}, {line_us,6.2009533333333335}} - Dmitry On Mar 25, 2012, at 12:14 AM, Tim Watson wrote: > On 24 Mar 2012, at 16:45, Max Lapshin wrote: > >> I'm trying to use file:read from erlang combined with compiled scanf pattern or some regex. There are two ideas: >> 1) combine two binary:split will give double memory walking. I think it is a good idea to parse memory once > > Actually there's a double problem here. In a CSV file it is possible that a delimiter might be present 'inside' one of the fields, providing the contents of the cell are in quotes. For example: > >>> 1,2,16,John Snow,"Flat A, 128, Watling Street", etc.... > > So just doing binary:split(Bin, binary:compile_pattern(<<",">>), [global]) isn't enough. I will take a look how binary:split/3 is implemented, but I'm starting to think that you really do need to drop into C to make this fast enough. Maybe for your data set you don't care about this possibility (because you have some control over the inputs) but for a general purpose CSV parser, this has to be handled and requires too much introspection of the data to efficient over 70MiB and more if written in pure Erlang. > > With file:read_line and binary:split, I can get the data out (without float/int conversion) sequentially in about 10 seconds. It's not particularly useful to parallelise the binary:split work as for the most part it isn't taking a long time per operation, but performs a vast number of them which slows down. Spawning for each processing line isn't really very smart as even in SMP mode the amount of churn due to scheduling seems likely to be prohibitive. Even if you hit the sweet spot with the right number of worker processes (1 per cpu +/- 1 for example) you've got the additional work of reconstituting the data in the correct order which takes up a lot of time. > > Now if I manually chunk my way through the file, using an optimal segment size (on my mac, 16 * 1024 seems to be best) with a sizeable read_ahead of 1024 * 1024, then I can get through the whole file and binary:split(Bin, CommaPattern, [global]) on each chunk, in an average of 2300ms. Of course this is too slow, is ignoring newlines at the moment (unlike the other tests) and I've removed the code for dealing with fields that sit across the segment boundaries (which slows things down also): > > read(P=#csv_parser{ io_device=Fd, buffer_size=Size }, start, Idx, Rem) -> > read(P, file:read(Fd, Size), Idx, Rem); > read(P=#csv_parser{ io_device=Fd, buffer_size=Size, > delimiter=Delim, collector=Collector, > supported_newlines=NL }, > {ok, Chunks}, > Idx, Rem) -> > binary:split(Chunks, Delim, [global]), > read(P, file:read(Fd, Size), Idx + 1, <<>>); > read(_P, eof, _, _Rem) -> > %% in strict mode, fail unless size(Rem) == 0 > ok. > >>>>> > t4@REDACTED:csv_reader $ ./csv_reader.erl example.csv > Starting read.... > read_chunk time: 2376 > t4@REDACTED:csv_reader $ > > Now if I switch the split to use binary:matches instead, and look for either ',' or a newline, we get a big performance improvement: > > t4@REDACTED:csv_reader $ ./csv_reader.erl example.csv > Starting read.... > read_chunk time: 953 > t4@REDACTED:csv_reader $ > > That's getting much faster, *but* we're not doing any hard work of parsing yet and we're not dealing with 'cross-segment' parts, which I'll look at next. > >> 2) intermediate creation of binaries for all those floats (12 millions of them in this example) is also evil. Perhaps line parsing should be combined with float extraction in the same manner as decode_packet does. > > > Like I said I think that with the approach I mentioned above (using nicely segmented file:read calls and binary:matches) we've got the i/o subsystem and splitting of fields to go as fast as they're likely to go without dropping into C. Now we get into the more cpu intensive parts - taking the {Location, Size} data from binary:matches and combined with your segment Idx working out the correct offset to use in calling binary:part, plus reconstituting stuff that sits across two segments and then finally converting binary to other data types for the final results. If I could make these parts happen in 2 seconds to get combined throughput of 3 seconds I'd be pleased, but I'm not sure it'll be possible. And even then, your desired time of 1 second isn't looking at all likely for even this 300k record file, let alone a larger 1million line file. > > It is possible that applying some parallelism to this will help, but the timings we're after are so small that I'm not sure. Spawning per split is of course a waste of time, but it is possible that having a worker process per core that does binary:matches and parses might help. The difficulty here is that you have to deal with the data sitting across two segments before you can pass it off to a worker, because you need the previous 'remainder' if any - I suppose you could 'pass this in' along with the data. If this could be made to work efficiently, then you might also benefit from a parallel collector process taking the completed/parsed records from the workers and storing them so they're ready for retrieval. Again, I'm not convinced all of this could be done in pure Erlang in less than 3 - 4 seconds overall. Perhaps putting ets into the mix might provide some speedup, as I would guess that a shared table can be atomically accessed faster than the processes can exchange data > via a mailbox. > > I'm going to see how fast this can be made to go in pure Erlang (allowing for ETS if it helps) just as a fun exercise. I don't know how much spare time I'll have to do it though and I'm convinced a solution written in C is the right answer for 1million records per second, and to do that a segment at a time approach that does almost all the work in C including all the type conversion is probably the way to go. In that case I would probably do the whole i/o part in C too, using a similar series of buffered read operations rather than line oriented fgets calls or things like that. > > I do not think that mmap or even reading all the file into memory will be of any use, as you mentioned earlier, the problem is really cpu bound. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Sun Mar 25 04:19:32 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 25 Mar 2012 03:19:32 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> Message-ID: On 24 Mar 2012, at 22:32, Dmitry Kolesnikov wrote: > Hello, > > You have raise a freaking interesting issue! > > I have decided to invest my time here and share binary parsing techniques used by me. > In the nutshell, I was capable to parse a line in 2.17 micro seconds using native erlang implementation. I have tried to productize code and put it here: > > https://github.com/fogfish/csv > > The algorithm is very simple > 1. load a whole file into memory > 2. shard the file into N chunks > 3. parse each chunk in parallel, parsing is based on binary scan (no binary:split, etc) > 4. merge results. > Thanks for sharing this, it sounds great. I was under the impression from reading the documentation that using the binary module to split everything up would be quicker than hand coding. > The test was run on Mac Mini server with R15B > Erlang R15B (erts-5.9) [source] [64-bit] [smp:8:8] [async-threads:0] [hipe] [kernel-poll:false] > > Since, you have not shared a details of csv file I used the following setups: > * set.txt contains 300K rows, each row has 16 fields of 3 hex digit each > * set2.txt contains 300K rows, each row has 48 fields of 3 hex digit each > This is a bit different to the input Max is trying to work with isn't it? Can you try with his generated example.csv? I tried doing this and your output seems to be a small list of numbers, so I'm not sure what's going wrong. 13> csv_example:run("example.csv", 80). total parse time: 5897.321 Result = [3711,3750,3750,3751,3751,3750,3750,3749,3750,3751,3751,3751,3750,3752,3751, 3750,3750,3751,3751,3751,3750,3750,3751,3751,3750,3751,3751,3750,3753,3751, 3749,3750,3751,3751,3750,3751,3750,3750,3751,3751,3750,3751,3751,3750,3750, 3750,3751,3751,3751,3751,3751,3751,3750,3751,3751,3751,3749,3751,3749,3751, 3751,3750,3751,3751,3750,3752,3750,3751,3750,3751,3750,3751,3749,3750,3750, 3751,3750,3749,3750,3750] > Here is the results: > csv_example:run("priv/set.txt", 80). > {{lines,300000}, > {size,18000000}, > {read_ms,132.269}, > {parse_ms,652.865}, > {line_us,2.1762166666666665}} > > 2> csv_example:run("priv/set2.txt", 80). > {{lines,300000}, > {size,54000000}, > {read_ms,204.259}, > {parse_ms,1860.286}, > {line_us,6.2009533333333335}} > I changed the example code a little bit, to account for the fact that we consider the file:read_file and the parsing all together in all our other examples. I get very different results, although I'm admittedly (probably?) on a less powerful machine: 4@REDACTED:csv $ erl -pa src -pa priv Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.2 (abort with ^G) 1> csv_example:run("example.csv", 80). {{lines,300001}, {size,78599457}, {parse_ms,5727.757}, {line_us,19.09245969180103}} 2> > - Dmitry > > > On Mar 25, 2012, at 12:14 AM, Tim Watson wrote: > >> On 24 Mar 2012, at 16:45, Max Lapshin wrote: >> >>> I'm trying to use file:read from erlang combined with compiled scanf pattern or some regex. There are two ideas: >>> 1) combine two binary:split will give double memory walking. I think it is a good idea to parse memory once >> >> Actually there's a double problem here. In a CSV file it is possible that a delimiter might be present 'inside' one of the fields, providing the contents of the cell are in quotes. For example: >> >>>> 1,2,16,John Snow,"Flat A, 128, Watling Street", etc.... >> >> So just doing binary:split(Bin, binary:compile_pattern(<<",">>), [global]) isn't enough. I will take a look how binary:split/3 is implemented, but I'm starting to think that you really do need to drop into C to make this fast enough. Maybe for your data set you don't care about this possibility (because you have some control over the inputs) but for a general purpose CSV parser, this has to be handled and requires too much introspection of the data to efficient over 70MiB and more if written in pure Erlang. >> >> With file:read_line and binary:split, I can get the data out (without float/int conversion) sequentially in about 10 seconds. It's not particularly useful to parallelise the binary:split work as for the most part it isn't taking a long time per operation, but performs a vast number of them which slows down. Spawning for each processing line isn't really very smart as even in SMP mode the amount of churn due to scheduling seems likely to be prohibitive. Even if you hit the sweet spot with the right number of worker processes (1 per cpu +/- 1 for example) you've got the additional work of reconstituting the data in the correct order which takes up a lot of time. >> >> Now if I manually chunk my way through the file, using an optimal segment size (on my mac, 16 * 1024 seems to be best) with a sizeable read_ahead of 1024 * 1024, then I can get through the whole file and binary:split(Bin, CommaPattern, [global]) on each chunk, in an average of 2300ms. Of course this is too slow, is ignoring newlines at the moment (unlike the other tests) and I've removed the code for dealing with fields that sit across the segment boundaries (which slows things down also): >> >> read(P=#csv_parser{ io_device=Fd, buffer_size=Size }, start, Idx, Rem) -> >> read(P, file:read(Fd, Size), Idx, Rem); >> read(P=#csv_parser{ io_device=Fd, buffer_size=Size, >> delimiter=Delim, collector=Collector, >> supported_newlines=NL }, >> {ok, Chunks}, >> Idx, Rem) -> >> binary:split(Chunks, Delim, [global]), >> read(P, file:read(Fd, Size), Idx + 1, <<>>); >> read(_P, eof, _, _Rem) -> >> %% in strict mode, fail unless size(Rem) == 0 >> ok. >> >>>>>> >> t4@REDACTED:csv_reader $ ./csv_reader.erl example.csv >> Starting read.... >> read_chunk time: 2376 >> t4@REDACTED:csv_reader $ >> >> Now if I switch the split to use binary:matches instead, and look for either ',' or a newline, we get a big performance improvement: >> >> t4@REDACTED:csv_reader $ ./csv_reader.erl example.csv >> Starting read.... >> read_chunk time: 953 >> t4@REDACTED:csv_reader $ >> >> That's getting much faster, *but* we're not doing any hard work of parsing yet and we're not dealing with 'cross-segment' parts, which I'll look at next. >> >>> 2) intermediate creation of binaries for all those floats (12 millions of them in this example) is also evil. Perhaps line parsing should be combined with float extraction in the same manner as decode_packet does. >> >> >> Like I said I think that with the approach I mentioned above (using nicely segmented file:read calls and binary:matches) we've got the i/o subsystem and splitting of fields to go as fast as they're likely to go without dropping into C. Now we get into the more cpu intensive parts - taking the {Location, Size} data from binary:matches and combined with your segment Idx working out the correct offset to use in calling binary:part, plus reconstituting stuff that sits across two segments and then finally converting binary to other data types for the final results. If I could make these parts happen in 2 seconds to get combined throughput of 3 seconds I'd be pleased, but I'm not sure it'll be possible. And even then, your desired time of 1 second isn't looking at all likely for even this 300k record file, let alone a larger 1million line file. >> >> It is possible that applying some parallelism to this will help, but the timings we're after are so small that I'm not sure. Spawning per split is of course a waste of time, but it is possible that having a worker process per core that does binary:matches and parses might help. The difficulty here is that you have to deal with the data sitting across two segments before you can pass it off to a worker, because you need the previous 'remainder' if any - I suppose you could 'pass this in' along with the data. If this could be made to work efficiently, then you might also benefit from a parallel collector process taking the completed/parsed records from the workers and storing them so they're ready for retrieval. Again, I'm not convinced all of this could be done in pure Erlang in less than 3 - 4 seconds overall. Perhaps putting ets into the mix might provide some speedup, as I would guess that a shared table can be atomically accessed faster than the processes can exchange data >> via a mailbox. >> >> I'm going to see how fast this can be made to go in pure Erlang (allowing for ETS if it helps) just as a fun exercise. I don't know how much spare time I'll have to do it though and I'm convinced a solution written in C is the right answer for 1million records per second, and to do that a segment at a time approach that does almost all the work in C including all the type conversion is probably the way to go. In that case I would probably do the whole i/o part in C too, using a similar series of buffered read operations rather than line oriented fgets calls or things like that. >> >> I do not think that mmap or even reading all the file into memory will be of any use, as you mentioned earlier, the problem is really cpu bound. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From watson.timothy@REDACTED Sun Mar 25 04:26:54 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 25 Mar 2012 03:26:54 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> Message-ID: <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> On 25 Mar 2012, at 03:19, Tim Watson wrote: > On 24 Mar 2012, at 22:32, Dmitry Kolesnikov wrote: > >> Hello, >> >> You have raise a freaking interesting issue! >> >> I have decided to invest my time here and share binary parsing techniques used by me. >> In the nutshell, I was capable to parse a line in 2.17 micro seconds using native erlang implementation. I have tried to productize code and put it here: >> >> https://github.com/fogfish/csv >> >> The algorithm is very simple >> 1. load a whole file into memory >> 2. shard the file into N chunks >> 3. parse each chunk in parallel, parsing is based on binary scan (no binary:split, etc) >> 4. merge results. >> > > Thanks for sharing this, it sounds great. I was under the impression from reading the documentation that using the binary module to split everything up would be quicker than hand coding. > >> The test was run on Mac Mini server with R15B >> Erlang R15B (erts-5.9) [source] [64-bit] [smp:8:8] [async-threads:0] [hipe] [kernel-poll:false] >> >> Since, you have not shared a details of csv file I used the following setups: >> * set.txt contains 300K rows, each row has 16 fields of 3 hex digit each >> * set2.txt contains 300K rows, each row has 48 fields of 3 hex digit each >> > > This is a bit different to the input Max is trying to work with isn't it? Can you try with his generated example.csv? I tried doing this and your output seems to be a small list of numbers, so I'm not sure what's going wrong. > > 13> csv_example:run("example.csv", 80). > total parse time: 5897.321 > Result = > [3711,3750,3750,3751,3751,3750,3750,3749,3750,3751,3751,3751,3750,3752,3751, > 3750,3750,3751,3751,3751,3750,3750,3751,3751,3750,3751,3751,3750,3753,3751, > 3749,3750,3751,3751,3750,3751,3750,3750,3751,3751,3750,3751,3751,3750,3750, > 3750,3751,3751,3751,3751,3751,3751,3750,3751,3751,3751,3749,3751,3749,3751, > 3751,3750,3751,3751,3750,3752,3750,3751,3750,3751,3750,3751,3749,3750,3750, > 3751,3750,3749,3750,3750] > On further inspection, I can see that this happens because the fun you pass to the parser (to handle the {line, _} event) is simply incrementing a count. My mistake! > >> Here is the results: >> csv_example:run("priv/set.txt", 80). >> {{lines,300000}, >> {size,18000000}, >> {read_ms,132.269}, >> {parse_ms,652.865}, >> {line_us,2.1762166666666665}} >> >> 2> csv_example:run("priv/set2.txt", 80). >> {{lines,300000}, >> {size,54000000}, >> {read_ms,204.259}, >> {parse_ms,1860.286}, >> {line_us,6.2009533333333335}} >> > > I changed the example code a little bit, to account for the fact that we consider the file:read_file and the parsing all together in all our other examples. I get very different results, although I'm admittedly (probably?) on a less powerful machine: > > 4@REDACTED:csv $ erl -pa src -pa priv > Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.2 (abort with ^G) > 1> csv_example:run("example.csv", 80). > {{lines,300001}, > {size,78599457}, > {parse_ms,5727.757}, > {line_us,19.09245969180103}} > 2> > > >> - Dmitry >> >> >> On Mar 25, 2012, at 12:14 AM, Tim Watson wrote: >> >>> On 24 Mar 2012, at 16:45, Max Lapshin wrote: >>> >>>> I'm trying to use file:read from erlang combined with compiled scanf pattern or some regex. There are two ideas: >>>> 1) combine two binary:split will give double memory walking. I think it is a good idea to parse memory once >>> >>> Actually there's a double problem here. In a CSV file it is possible that a delimiter might be present 'inside' one of the fields, providing the contents of the cell are in quotes. For example: >>> >>>>> 1,2,16,John Snow,"Flat A, 128, Watling Street", etc.... >>> >>> So just doing binary:split(Bin, binary:compile_pattern(<<",">>), [global]) isn't enough. I will take a look how binary:split/3 is implemented, but I'm starting to think that you really do need to drop into C to make this fast enough. Maybe for your data set you don't care about this possibility (because you have some control over the inputs) but for a general purpose CSV parser, this has to be handled and requires too much introspection of the data to efficient over 70MiB and more if written in pure Erlang. >>> >>> With file:read_line and binary:split, I can get the data out (without float/int conversion) sequentially in about 10 seconds. It's not particularly useful to parallelise the binary:split work as for the most part it isn't taking a long time per operation, but performs a vast number of them which slows down. Spawning for each processing line isn't really very smart as even in SMP mode the amount of churn due to scheduling seems likely to be prohibitive. Even if you hit the sweet spot with the right number of worker processes (1 per cpu +/- 1 for example) you've got the additional work of reconstituting the data in the correct order which takes up a lot of time. >>> >>> Now if I manually chunk my way through the file, using an optimal segment size (on my mac, 16 * 1024 seems to be best) with a sizeable read_ahead of 1024 * 1024, then I can get through the whole file and binary:split(Bin, CommaPattern, [global]) on each chunk, in an average of 2300ms. Of course this is too slow, is ignoring newlines at the moment (unlike the other tests) and I've removed the code for dealing with fields that sit across the segment boundaries (which slows things down also): >>> >>> read(P=#csv_parser{ io_device=Fd, buffer_size=Size }, start, Idx, Rem) -> >>> read(P, file:read(Fd, Size), Idx, Rem); >>> read(P=#csv_parser{ io_device=Fd, buffer_size=Size, >>> delimiter=Delim, collector=Collector, >>> supported_newlines=NL }, >>> {ok, Chunks}, >>> Idx, Rem) -> >>> binary:split(Chunks, Delim, [global]), >>> read(P, file:read(Fd, Size), Idx + 1, <<>>); >>> read(_P, eof, _, _Rem) -> >>> %% in strict mode, fail unless size(Rem) == 0 >>> ok. >>> >>>>>>> >>> t4@REDACTED:csv_reader $ ./csv_reader.erl example.csv >>> Starting read.... >>> read_chunk time: 2376 >>> t4@REDACTED:csv_reader $ >>> >>> Now if I switch the split to use binary:matches instead, and look for either ',' or a newline, we get a big performance improvement: >>> >>> t4@REDACTED:csv_reader $ ./csv_reader.erl example.csv >>> Starting read.... >>> read_chunk time: 953 >>> t4@REDACTED:csv_reader $ >>> >>> That's getting much faster, *but* we're not doing any hard work of parsing yet and we're not dealing with 'cross-segment' parts, which I'll look at next. >>> >>>> 2) intermediate creation of binaries for all those floats (12 millions of them in this example) is also evil. Perhaps line parsing should be combined with float extraction in the same manner as decode_packet does. >>> >>> >>> Like I said I think that with the approach I mentioned above (using nicely segmented file:read calls and binary:matches) we've got the i/o subsystem and splitting of fields to go as fast as they're likely to go without dropping into C. Now we get into the more cpu intensive parts - taking the {Location, Size} data from binary:matches and combined with your segment Idx working out the correct offset to use in calling binary:part, plus reconstituting stuff that sits across two segments and then finally converting binary to other data types for the final results. If I could make these parts happen in 2 seconds to get combined throughput of 3 seconds I'd be pleased, but I'm not sure it'll be possible. And even then, your desired time of 1 second isn't looking at all likely for even this 300k record file, let alone a larger 1million line file. >>> >>> It is possible that applying some parallelism to this will help, but the timings we're after are so small that I'm not sure. Spawning per split is of course a waste of time, but it is possible that having a worker process per core that does binary:matches and parses might help. The difficulty here is that you have to deal with the data sitting across two segments before you can pass it off to a worker, because you need the previous 'remainder' if any - I suppose you could 'pass this in' along with the data. If this could be made to work efficiently, then you might also benefit from a parallel collector process taking the completed/parsed records from the workers and storing them so they're ready for retrieval. Again, I'm not convinced all of this could be done in pure Erlang in less than 3 - 4 seconds overall. Perhaps putting ets into the mix might provide some speedup, as I would guess that a shared table can be atomically accessed faster than the processes can exchange data >>> via a mailbox. >>> >>> I'm going to see how fast this can be made to go in pure Erlang (allowing for ETS if it helps) just as a fun exercise. I don't know how much spare time I'll have to do it though and I'm convinced a solution written in C is the right answer for 1million records per second, and to do that a segment at a time approach that does almost all the work in C including all the type conversion is probably the way to go. In that case I would probably do the whole i/o part in C too, using a similar series of buffered read operations rather than line oriented fgets calls or things like that. >>> >>> I do not think that mmap or even reading all the file into memory will be of any use, as you mentioned earlier, the problem is really cpu bound. >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > From watson.timothy@REDACTED Sun Mar 25 04:54:11 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 25 Mar 2012 03:54:11 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> Message-ID: On 25 Mar 2012, at 03:26, Tim Watson wrote: >>> The test was run on Mac Mini server with R15B >>> Erlang R15B (erts-5.9) [source] [64-bit] [smp:8:8] [async-threads:0] [hipe] [kernel-poll:false] >>> >>> Since, you have not shared a details of csv file I used the following setups: >>> * set.txt contains 300K rows, each row has 16 fields of 3 hex digit each >>> * set2.txt contains 300K rows, each row has 48 fields of 3 hex digit each >>> >> >> This is a bit different to the input Max is trying to work with isn't it? Can you try with his generated example.csv? I tried doing this and your output seems to be a small list of numbers, so I'm not sure what's going wrong. >> The kind of lines in the generated example.csv BTW look like this: KEY1,20120201,12:15:44.543,34.28,54.74,16.74,88.51,32.48,15.7,54.19,71.52,69.5,55.35,3.9,20.08,33.83,63.43,12.4,9.66,0.29,59.61,52.94,82.49,78.96,70.52,55.73,79.37,61.25,54.19,49.31,14.14,40.18,21.39,82.26,40.79,36.57,86.14,39.58,28.3,20.1,24.07,51.35,8.38,zz >> 13> csv_example:run("example.csv", 80). >> total parse time: 5897.321 >> Result = >> [3711,3750,3750,3751,3751,3750,3750,3749,3750,3751,3751,3751,3750,3752,3751, >> 3750,3750,3751,3751,3751,3750,3750,3751,3751,3750,3751,3751,3750,3753,3751, >> 3749,3750,3751,3751,3750,3751,3750,3750,3751,3751,3750,3751,3751,3750,3750, >> 3750,3751,3751,3751,3751,3751,3751,3750,3751,3751,3751,3749,3751,3749,3751, >> 3751,3750,3751,3751,3750,3752,3750,3751,3750,3751,3750,3751,3749,3750,3750, >> 3751,3750,3749,3750,3750] >> > > On further inspection, I can see that this happens because the fun you pass to the parser (to handle the {line, _} event) is simply incrementing a count. My mistake! > >> >>> Here is the results: >>> csv_example:run("priv/set.txt", 80). >>> {{lines,300000}, >>> {size,18000000}, >>> {read_ms,132.269}, >>> {parse_ms,652.865}, >>> {line_us,2.1762166666666665}} >>> >>> 2> csv_example:run("priv/set2.txt", 80). >>> {{lines,300000}, >>> {size,54000000}, >>> {read_ms,204.259}, >>> {parse_ms,1860.286}, >>> {line_us,6.2009533333333335}} >>> >> >> I changed the example code a little bit, to account for the fact that we consider the file:read_file and the parsing all together in all our other examples. I get very different results, although I'm admittedly (probably?) on a less powerful machine: >> >> 4@REDACTED:csv $ erl -pa src -pa priv >> Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] >> >> Eshell V5.8.2 (abort with ^G) >> 1> csv_example:run("example.csv", 80). >> {{lines,300001}, >> {size,78599457}, >> {parse_ms,5727.757}, >> {line_us,19.09245969180103}} >> 2> Accumulating the results takes even longer than this though. I have the line event handler simply aggregate the lines and do the wall clock time measures as Max did (just for consistency), and try it with Wrk set to 80 and then larger (to 200) which starts to degrade somewhere between the two: test(Filename, Wrk) -> T1 = erlang:now(), {ok, Bin} = file:read_file(Filename), R = csv:pparse(Bin, Wrk, fun({line, Line}, Acc) -> [Line|Acc] end, []), T2 = erlang:now(), io:format("total parse time: ~p~n", [timer:now_diff(T2, T1) div 1000]), io:format("size(Results) = ~p~n", [length(R)]). And here's what I get: Eshell V5.8.2 (abort with ^G) 3> csv_example:test("example.csv", 80). total parse time: 15232 size(Results) = 80 ok 4> 5> csv_example:test("example.csv", 200). total parse time: 17086 size(Results) = 200 ok 6> So these appear to be taking between 50 and 20 seconds, and of course, you've got to then work your way through the results in the data for each shard. This part has switched me on to the idea of having a stream oriented API - I suspect that will work well, although it's worth baring in mind that the ordering isn't going to be maintained if you start 'sharding' the data set so as to process it sequentially. So do we think the additional time is spent because of the comparative width of the line(s) in Max's inputs? I can't imagine that binary:split is doing anything vastly different, except that it uses binary:matches which is implemented as a BIF - I am guessing that a BIF is going to beat hand coded binary pattern matching every time, is it not? Please also do run this code on some other (faster) machines to see how much of the slowness is specific to my machine. >> >>> - Dmitry >>> >>> >>> On Mar 25, 2012, at 12:14 AM, Tim Watson wrote: >>> >>>> On 24 Mar 2012, at 16:45, Max Lapshin wrote: >>>> >>>>> I'm trying to use file:read from erlang combined with compiled scanf pattern or some regex. There are two ideas: >>>>> 1) combine two binary:split will give double memory walking. I think it is a good idea to parse memory once >>>> >>>> Actually there's a double problem here. In a CSV file it is possible that a delimiter might be present 'inside' one of the fields, providing the contents of the cell are in quotes. For example: >>>> >>>>>> 1,2,16,John Snow,"Flat A, 128, Watling Street", etc.... >>>> >>>> So just doing binary:split(Bin, binary:compile_pattern(<<",">>), [global]) isn't enough. I will take a look how binary:split/3 is implemented, but I'm starting to think that you really do need to drop into C to make this fast enough. Maybe for your data set you don't care about this possibility (because you have some control over the inputs) but for a general purpose CSV parser, this has to be handled and requires too much introspection of the data to efficient over 70MiB and more if written in pure Erlang. >>>> >>>> With file:read_line and binary:split, I can get the data out (without float/int conversion) sequentially in about 10 seconds. It's not particularly useful to parallelise the binary:split work as for the most part it isn't taking a long time per operation, but performs a vast number of them which slows down. Spawning for each processing line isn't really very smart as even in SMP mode the amount of churn due to scheduling seems likely to be prohibitive. Even if you hit the sweet spot with the right number of worker processes (1 per cpu +/- 1 for example) you've got the additional work of reconstituting the data in the correct order which takes up a lot of time. >>>> >>>> Now if I manually chunk my way through the file, using an optimal segment size (on my mac, 16 * 1024 seems to be best) with a sizeable read_ahead of 1024 * 1024, then I can get through the whole file and binary:split(Bin, CommaPattern, [global]) on each chunk, in an average of 2300ms. Of course this is too slow, is ignoring newlines at the moment (unlike the other tests) and I've removed the code for dealing with fields that sit across the segment boundaries (which slows things down also): >>>> >>>> read(P=#csv_parser{ io_device=Fd, buffer_size=Size }, start, Idx, Rem) -> >>>> read(P, file:read(Fd, Size), Idx, Rem); >>>> read(P=#csv_parser{ io_device=Fd, buffer_size=Size, >>>> delimiter=Delim, collector=Collector, >>>> supported_newlines=NL }, >>>> {ok, Chunks}, >>>> Idx, Rem) -> >>>> binary:split(Chunks, Delim, [global]), >>>> read(P, file:read(Fd, Size), Idx + 1, <<>>); >>>> read(_P, eof, _, _Rem) -> >>>> %% in strict mode, fail unless size(Rem) == 0 >>>> ok. >>>> >>>>>>>> >>>> t4@REDACTED:csv_reader $ ./csv_reader.erl example.csv >>>> Starting read.... >>>> read_chunk time: 2376 >>>> t4@REDACTED:csv_reader $ >>>> >>>> Now if I switch the split to use binary:matches instead, and look for either ',' or a newline, we get a big performance improvement: >>>> >>>> t4@REDACTED:csv_reader $ ./csv_reader.erl example.csv >>>> Starting read.... >>>> read_chunk time: 953 >>>> t4@REDACTED:csv_reader $ >>>> >>>> That's getting much faster, *but* we're not doing any hard work of parsing yet and we're not dealing with 'cross-segment' parts, which I'll look at next. >>>> >>>>> 2) intermediate creation of binaries for all those floats (12 millions of them in this example) is also evil. Perhaps line parsing should be combined with float extraction in the same manner as decode_packet does. >>>> >>>> >>>> Like I said I think that with the approach I mentioned above (using nicely segmented file:read calls and binary:matches) we've got the i/o subsystem and splitting of fields to go as fast as they're likely to go without dropping into C. Now we get into the more cpu intensive parts - taking the {Location, Size} data from binary:matches and combined with your segment Idx working out the correct offset to use in calling binary:part, plus reconstituting stuff that sits across two segments and then finally converting binary to other data types for the final results. If I could make these parts happen in 2 seconds to get combined throughput of 3 seconds I'd be pleased, but I'm not sure it'll be possible. And even then, your desired time of 1 second isn't looking at all likely for even this 300k record file, let alone a larger 1million line file. >>>> >>>> It is possible that applying some parallelism to this will help, but the timings we're after are so small that I'm not sure. Spawning per split is of course a waste of time, but it is possible that having a worker process per core that does binary:matches and parses might help. The difficulty here is that you have to deal with the data sitting across two segments before you can pass it off to a worker, because you need the previous 'remainder' if any - I suppose you could 'pass this in' along with the data. If this could be made to work efficiently, then you might also benefit from a parallel collector process taking the completed/parsed records from the workers and storing them so they're ready for retrieval. Again, I'm not convinced all of this could be done in pure Erlang in less than 3 - 4 seconds overall. Perhaps putting ets into the mix might provide some speedup, as I would guess that a shared table can be atomically accessed faster than the processes can exchange data >>>> via a mailbox. >>>> >>>> I'm going to see how fast this can be made to go in pure Erlang (allowing for ETS if it helps) just as a fun exercise. I don't know how much spare time I'll have to do it though and I'm convinced a solution written in C is the right answer for 1million records per second, and to do that a segment at a time approach that does almost all the work in C including all the type conversion is probably the way to go. In that case I would probably do the whole i/o part in C too, using a similar series of buffered read operations rather than line oriented fgets calls or things like that. >>>> >>>> I do not think that mmap or even reading all the file into memory will be of any use, as you mentioned earlier, the problem is really cpu bound. >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > From watson.timothy@REDACTED Sun Mar 25 04:57:19 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 25 Mar 2012 03:57:19 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> Message-ID: On 25 Mar 2012, at 03:54, Tim Watson wrote: > Please also do run this code on some other (faster) machines to see how much of the slowness is specific to my machine. Oh and can you please pass on the set and set2 text files so I can test them on my machine to baseline the comparative differences between our environments? Cheers! Tim From thomasl_erlang@REDACTED Sun Mar 25 11:18:53 2012 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Sun, 25 Mar 2012 02:18:53 -0700 (PDT) Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> Message-ID: <1332667133.73532.YahooMailNeo@web111405.mail.gq1.yahoo.com> Not a direct comment, I'll just hook into the general discussion here.? Here's some older work that might be interesting:?http://www.tbray.org/ongoing/When/200x/2007/09/20/Wide-Finder Best, Thomas >________________________________ > From: Tim Watson >To: Tim Watson >Cc: erlang-questions >Sent: Sunday, March 25, 2012 4:57 AM >Subject: Re: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second > >On 25 Mar 2012, at 03:54, Tim Watson wrote: >>? Please also do run this code on some other (faster) machines to see how much of the slowness is specific to my machine. > >Oh and can you please pass on the set and set2 text files so I can test them on my machine to baseline the comparative differences between our environments? > >Cheers! > >Tim >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions > > > From dmkolesnikov@REDACTED Sun Mar 25 12:09:57 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sun, 25 Mar 2012 13:09:57 +0300 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> Message-ID: <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> Hello, 1. Yes, I'd like to admit that I've missed the example file from the beginning of thread. I've put my sets generators into the repository. You can use them like this: sh ./priv/dset line-ex.txt 1000 > ./priv/set-1M-ex.txt The first parameter is line, second is kilolines to generate. line-16/48 are my original examples, line-ex.txt is from original example. 2. I am a bit of curious why we are getting results of different magnitude, especially with Robert's run. My HW config is pretty close. So, I've build Erlang R15B from sources with following config: ./configure --prefix=/usr/local/otp-R15B --enable-threads --enable-smp-support --enable-kernel-poll --enable-sctp --enable-hipe --disable-dynamic-ssl-lib --enable-darwin-64bit --enable-m64-build --without-javac 3. I've re-run cases again with/without +native flag results are very interesting, so we can parse 300K lines in less then second, less the second for 1M rows is challenging: Parse time of data set is 300K: set-300K-16 653 ms / 2.18 us per line set-300K-48 1832 ms / 6.11 us per line set-300K-ex 2561 ms / 8.54 us per line Parse time of data set is 300K +native: set-300K-16 277 ms / 0.92 us per line set-300K-48 672 ms / 2.24 us per line set-300K-ex 925 ms / 3.09 us per line Parse time of data set is 1M: set-300K-16 4406 ms / 2.20 us per line set-300K-48 6076 ms / 6.08 us per line set-300K-ex 8670 ms / 8.67 us per line Parse time of data set is 1M +native: set-300K-16 1908 ms / 0.95 us per line set-300K-48 2293 ms / 2.29 us per line set-300K-ex 3119 ms / 3.12 us per line 4. It might be unfair to comment but I have a good feeling that Max is evaluating/challenging a type of trade system ;-) Otherwise, why you have such hard latency requirements and dataset looks like a price snapshot ;-) IMHO, 2 - 4 us parse time per row is acceptable for "normal" web app. Regards, Dmitry On Mar 25, 2012, at 5:57 AM, Tim Watson wrote: > On 25 Mar 2012, at 03:54, Tim Watson wrote: >> Please also do run this code on some other (faster) machines to see how much of the slowness is specific to my machine. > > Oh and can you please pass on the set and set2 text files so I can test them on my machine to baseline the comparative differences between our environments? > > Cheers! > > Tim From max.lapshin@REDACTED Sun Mar 25 12:35:41 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 25 Mar 2012 14:35:41 +0400 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> Message-ID: I'm not speaking about latency. I'm speaking about speed. From steven.charles.davis@REDACTED Sun Mar 25 16:12:14 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 25 Mar 2012 07:12:14 -0700 (PDT) Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> Message-ID: <20744632.326.1332684734709.JavaMail.geo-discussion-forums@ynbq18> I know there must be a reason why nobody has suggested: re:split(Bin, <<",(?=(?:[^\"]*\"[^\"]*\")*(?![^\"]*\"))">>). ...just not sure what that reason is? /s On Sunday, March 25, 2012 5:35:41 AM UTC-5, Max Lapshin wrote: > > I'm not speaking about latency. I'm speaking about speed. > ______________________________?_________________ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Sun Mar 25 16:52:48 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 25 Mar 2012 15:52:48 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> Message-ID: Thanks for posting this detail Dmitry, it's very helpful. I'm going to recompile with +native and give it a go. I also note that you're not dealing with nested delimiters (e.g., where a field is enclosed in single or double "quotation marks" and can therefore contain the delimiters). If I can get the speed up to what you're seeing, then I might just send you a couple of pull requests. :) Also I'm interested to know how you called the csv:parse/3 function (or pparse instead) for these examples? Are you, for example, accumulating the results, or are you just counting the occurrences? Please let me know so I can replicate the same setup and test +native for myself. Also would you be kind enough to let us all know what kind of hardware your mac mini has, especially in terms of CPU and available memory? All in all, if we can clear up the differences I think parsing 300k in just under a second without having to resort to a NIF is a very good result and makes this the fastest csv parsing facility we've seen so far!!! On 25 March 2012 11:09, Dmitry Kolesnikov wrote: > Hello, > > 1. Yes, I'd like to admit that I've missed the example file from the beginning of thread. I've put my sets generators into the repository. You can use them like this: > sh ./priv/dset line-ex.txt 1000 > ./priv/set-1M-ex.txt > The first parameter is line, second is kilolines to generate. line-16/48 are my original examples, line-ex.txt is from original example. > > 2. I am a bit of curious why we are getting results of different magnitude, especially with Robert's run. My HW config is pretty close. So, I've build Erlang R15B from sources with following config: > ./configure --prefix=/usr/local/otp-R15B --enable-threads --enable-smp-support --enable-kernel-poll --enable-sctp --enable-hipe --disable-dynamic-ssl-lib --enable-darwin-64bit --enable-m64-build --without-javac > > 3. I've re-run cases again with/without +native flag results are very interesting, so we can parse 300K lines in less then second, less the second for 1M rows is challenging: > > Parse time of data set is 300K: > set-300K-16 ? ? 653 ms / 2.18 us per line > set-300K-48 ? 1832 ms / 6.11 us per line > set-300K-ex ? 2561 ms / 8.54 us per line > > Parse time of data set is 300K +native: > set-300K-16 ? ? 277 ms / 0.92 us per line > set-300K-48 ? ? 672 ms / 2.24 us per line > set-300K-ex ? ? 925 ms / 3.09 us per line > > Parse time of data set is 1M: > set-300K-16 ? 4406 ms / 2.20 us per line > set-300K-48 ? 6076 ms / 6.08 us per line > set-300K-ex ? 8670 ms / 8.67 us per line > > Parse time of data set is 1M +native: > set-300K-16 ? ?1908 ms / 0.95 us per line > set-300K-48 ? ?2293 ms / 2.29 us per line > set-300K-ex ? ?3119 ms / 3.12 us per line > > 4. It might be unfair to comment but I have a good feeling that Max is evaluating/challenging a type of trade system ;-) Otherwise, why you have such hard latency requirements and dataset looks like a price snapshot ;-) IMHO, 2 - 4 us parse time per row is acceptable for "normal" web app. > > Regards, Dmitry > > On Mar 25, 2012, at 5:57 AM, Tim Watson wrote: > >> On 25 Mar 2012, at 03:54, Tim Watson wrote: >>> Please also do run this code on some other (faster) machines to see how much of the slowness is specific to my machine. >> >> Oh and can you please pass on the set and set2 text files so I can test them on my machine to baseline the comparative differences between our environments? >> >> Cheers! >> >> Tim > From nicolas.dufour@REDACTED Sun Mar 25 18:59:49 2012 From: nicolas.dufour@REDACTED (Nicolas Dufour) Date: Sun, 25 Mar 2012 12:59:49 -0400 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> Message-ID: <406B347E-A178-41E2-91FF-36F2ACC114C6@nemoworld.info> Hello everybody, While we are talking on csv parsers, I've been working on one for which the priority is not raw speed but to be able to parse huge files in a stream (if your csv is 60G big, no worry), or csv sent over any wire (http, whatever), and using a callback to both process each line and a general state for the whole parsing. You can take a look at https://github.com/refuge/ecsv . It's actually based on a 3 states machine The main goal was to be able to parse a stream and dispatch the processing across many processes. So far it's able to parse double-quoted field and different delimiters and is written in 100% erlang. The last benchmark I did though was showing something like 1700 row/s ( example here http://friendpaste.com/58s9VAVswaczu4Zav49GEq ). But again, my goal was to withstand bug files through a stream. Feel free to take a look at it. Thank you, Nicolas Dufour On Mar 25, 2012, at 10:52 AM, Tim Watson wrote: > Thanks for posting this detail Dmitry, it's very helpful. I'm going to > recompile with +native and give it a go. I also note that you're not > dealing with nested delimiters (e.g., where a field is enclosed in > single or double "quotation marks" and can therefore contain the > delimiters). If I can get the speed up to what you're seeing, then I > might just send you a couple of pull requests. :) > > Also I'm interested to know how you called the csv:parse/3 function > (or pparse instead) for these examples? Are you, for example, > accumulating the results, or are you just counting the occurrences? > Please let me know so I can replicate the same setup and test +native > for myself. > > Also would you be kind enough to let us all know what kind of hardware > your mac mini has, especially in terms of CPU and available memory? > > All in all, if we can clear up the differences I think parsing 300k in > just under a second without having to resort to a NIF is a very good > result and makes this the fastest csv parsing facility we've seen so > far!!! > > On 25 March 2012 11:09, Dmitry Kolesnikov wrote: >> Hello, >> >> 1. Yes, I'd like to admit that I've missed the example file from the beginning of thread. I've put my sets generators into the repository. You can use them like this: >> sh ./priv/dset line-ex.txt 1000 > ./priv/set-1M-ex.txt >> The first parameter is line, second is kilolines to generate. line-16/48 are my original examples, line-ex.txt is from original example. >> >> 2. I am a bit of curious why we are getting results of different magnitude, especially with Robert's run. My HW config is pretty close. So, I've build Erlang R15B from sources with following config: >> ./configure --prefix=/usr/local/otp-R15B --enable-threads --enable-smp-support --enable-kernel-poll --enable-sctp --enable-hipe --disable-dynamic-ssl-lib --enable-darwin-64bit --enable-m64-build --without-javac >> >> 3. I've re-run cases again with/without +native flag results are very interesting, so we can parse 300K lines in less then second, less the second for 1M rows is challenging: >> >> Parse time of data set is 300K: >> set-300K-16 653 ms / 2.18 us per line >> set-300K-48 1832 ms / 6.11 us per line >> set-300K-ex 2561 ms / 8.54 us per line >> >> Parse time of data set is 300K +native: >> set-300K-16 277 ms / 0.92 us per line >> set-300K-48 672 ms / 2.24 us per line >> set-300K-ex 925 ms / 3.09 us per line >> >> Parse time of data set is 1M: >> set-300K-16 4406 ms / 2.20 us per line >> set-300K-48 6076 ms / 6.08 us per line >> set-300K-ex 8670 ms / 8.67 us per line >> >> Parse time of data set is 1M +native: >> set-300K-16 1908 ms / 0.95 us per line >> set-300K-48 2293 ms / 2.29 us per line >> set-300K-ex 3119 ms / 3.12 us per line >> >> 4. It might be unfair to comment but I have a good feeling that Max is evaluating/challenging a type of trade system ;-) Otherwise, why you have such hard latency requirements and dataset looks like a price snapshot ;-) IMHO, 2 - 4 us parse time per row is acceptable for "normal" web app. >> >> Regards, Dmitry >> >> On Mar 25, 2012, at 5:57 AM, Tim Watson wrote: >> >>> On 25 Mar 2012, at 03:54, Tim Watson wrote: >>>> Please also do run this code on some other (faster) machines to see how much of the slowness is specific to my machine. >>> >>> Oh and can you please pass on the set and set2 text files so I can test them on my machine to baseline the comparative differences between our environments? >>> >>> Cheers! >>> >>> Tim >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sun Mar 25 19:27:04 2012 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 25 Mar 2012 19:27:04 +0200 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: Message-ID: I'm joining this thread rather late, ... Would it be possible to change the problem a bit? The fastest approach would seem to be (assume a quad core): - spilt file into four - parse each bit in parallel - combine the results Well splitting (the first part) is essentially sequential (or at least difficult to do in parallel unless you really understand your hardware - and may be impossible to do in parallel) I would assume that the input file has been created by a sequential program - if this were the case could it not produce four files instead of one? If this were the case then the split into four bit would go away. Do the bits have to be recombined later? - if not the last bit can be removed as well. The key to performance might be to redesign the entire processing pipeline making it as parallel as possible as soon as possible and keeping it as parallel as possible as long as possible. Cheers /Joe On Fri, Mar 23, 2012 at 11:30 AM, Max Lapshin wrote: > I need to load large CSV file into memory very fast. > > I've tried to use erlang parser, but my results were very bad (in fact > ?file:read_line is very slow), so I've tried to make a NIF for it. > Target speed is 1 microsecond per line. > > > My CSV has very strict format: only numbers, no quoting, \n in the > end. Also I moved parsing of integers and date into NIF. > > My results are here: http://github.com/maxlapshin/csv_reader and I get > only 15 microseconds: ?4,5 seconds for 300K lines CSV. > > Currently I use fgets to read line by line from file. Maybe it is a > bad idea and I should use mmap or implement 1MB buffer for read? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Sun Mar 25 19:39:43 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 25 Mar 2012 21:39:43 +0400 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <20744632.326.1332684734709.JavaMail.geo-discussion-forums@ynbq18> References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> <20744632.326.1332684734709.JavaMail.geo-discussion-forums@ynbq18> Message-ID: On Sun, Mar 25, 2012 at 6:12 PM, Steve Davis wrote: > I know there must be a reason why nobody has suggested: > > re:split(Bin, <<",(?=(?:[^\"]*\"[^\"]*\")*(?![^\"]*\"))">>). > > ...just not sure what that reason is? > Because after you get binaries with encoded floats you need very expensive translation of binary to float via list. From max.lapshin@REDACTED Sun Mar 25 19:47:19 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 25 Mar 2012 21:47:19 +0400 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> <20744632.326.1332684734709.JavaMail.geo-discussion-forums@ynbq18> Message-ID: So, currently my time is 850 ms for 300K line file which is 2.8 us per line. https://github.com/maxlapshin/csv_reader $ ./csv_bench.erl example.csv ./csv_bench.erl:66: Warning: variable 'Evt' is unused Load csv_reader: ok csv_reader:133 {chunk,210,19650181} .... csv_reader:142 {loader,<0.36.0>,finish} NIF: 851 Ideas are following: 1) Parse file in 4 threads. Detect where are real borders of lines of each part and spawn 4 workers with offsets and limits 2) Don't use prepending of data, left from previous block. If some line is read only partially, than read next block size(Rest) bytes earlier 3) Use special C nif that splits binaries into lines and parses them in the same call. Parsing is done via precompiled pattern. It was a good idea to throw away non-erlang IO, because there are no lags in erlang file:read layer. But there are lags even in binary:split. NIF that returns subbinary till first EOL is several times faster than binary:split(Bin, <<"\n">>). From james@REDACTED Sun Mar 25 20:53:58 2012 From: james@REDACTED (james) Date: Sun, 25 Mar 2012 19:53:58 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: Message-ID: <4F6F69C6.9070805@mansionfamily.plus.com> I think you should be able to parse something like this at approaching the speed of memcpy, but you have to try hard. Have you tried writing something that runs outside erlang and just tries to parse the file into numbers in memory, say using re2c or ragel? You might need to try using mmap and a thread that just reads every 16th byte and forces the file to read in. Or double buffer with readahead. Is the data in your operating system's VM cache already, or on disk? James From watson.timothy@REDACTED Sun Mar 25 23:02:21 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 25 Mar 2012 22:02:21 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> <20744632.326.1332684734709.JavaMail.geo-discussion-forums@ynbq18> Message-ID: <3D651B97-6CBB-4D6F-8BF7-5C2B8010680B@gmail.com> I think all these ideas have merit. For (1) I agree with Joe that to take proper advantage of parallelism, you've got to redesign the problem space a little, possibly by removing the need for splitting, recombining and/or re-sequencing. The problem to my mind is that parsing a csv input stream (be it from a file or a socket or whatever) is more naturally a sequential task. If we add parallelism for performance gains, we pay later on when we have to 'work around' the issues that parallelism introduces (such as out of order data). Having said that, maybe your problem space isn't specifically about parsing a large range of csv inputs files. BTW my macbook pro appears to have appalling performance characteristics when testing this, taking +/- 300ms longer to do the work: t4@REDACTED:csv_reader $ make bench ./rebar compile ==> csv_reader (compile) ./csv_bench.erl example.csv ./csv_bench.erl:66: Warning: variable 'Evt' is unused Load csv_reader: ok NIF: 1202 Note that I also had to remove the io:format/2 debugging from the code, as this caused the code to take much longer: t4@REDACTED:csv_reader $ make bench ./rebar compile ==> csv_reader (compile) ./csv_bench.erl example.csv ./csv_bench.erl:66: Warning: variable 'Evt' is unused Load csv_reader: ok csv_reader:133 {chunk,210,19650023} .... csv_reader:142 {loader,<0.35.0>,finish} NIF: 1828 I would *love* to know why all this io:format takes so much longer on my machine than yours, as I was under the impression that Erlang/OTP behaved nicely on darwin (apart from not doing proper kernel poll as kqueue seems pretty broken on OS-X) and this is making me wonder. On 25 Mar 2012, at 18:47, Max Lapshin wrote: > So, currently my time is 850 ms for 300K line file which is 2.8 us per line. > > https://github.com/maxlapshin/csv_reader > > $ ./csv_bench.erl example.csv > ./csv_bench.erl:66: Warning: variable 'Evt' is unused > Load csv_reader: ok > csv_reader:133 {chunk,210,19650181} > .... > csv_reader:142 {loader,<0.36.0>,finish} > NIF: 851 > > > Ideas are following: > > 1) Parse file in 4 threads. Detect where are real borders of lines of > each part and spawn 4 workers with offsets and limits > 2) Don't use prepending of data, left from previous block. If some > line is read only partially, than read next block size(Rest) bytes > earlier > 3) Use special C nif that splits binaries into lines and parses them > in the same call. Parsing is done via precompiled pattern. > > > > It was a good idea to throw away non-erlang IO, because there are no > lags in erlang file:read layer. But there are lags even in > binary:split. NIF that returns subbinary till first EOL is several > times faster than binary:split(Bin, <<"\n">>). > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Sun Mar 25 23:05:04 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 26 Mar 2012 01:05:04 +0400 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <3D651B97-6CBB-4D6F-8BF7-5C2B8010680B@gmail.com> References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> <20744632.326.1332684734709.JavaMail.geo-discussion-forums@ynbq18> <3D651B97-6CBB-4D6F-8BF7-5C2B8010680B@gmail.com> Message-ID: On Mon, Mar 26, 2012 at 1:02 AM, Tim Watson wrote: > > BTW my macbook pro appears to have appalling performance characteristics when testing this, taking +/- 300ms longer to do the work: > I have changed HDD to SSD and thrown away cdrom. Don't know what affects speed more =) I really want to load CSV file very fast, because I need to load several files and merge them with sorting. From dmkolesnikov@REDACTED Sun Mar 25 23:19:06 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Mon, 26 Mar 2012 00:19:06 +0300 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> Message-ID: <1C468E9B-5697-4BEA-90CB-6E53275D958B@gmail.com> Hello Tim, 1. Yes, you are right about nested/escaped delimiters. They have to be fixed. 2. Reference platform: * MacMini, Lion Server, * 1x Intel Core i7 (2 GHz), 4x cores * L2 Cache 256KB per core * L3 Cache 6MB * Memory 4GB 1333 MHZ DDR3 * Disk 750GB 7200rpm WDC WD7500BTKT-40MD3T0 * erlang R15B + native build of the library 3. I've tried to improve the documentation as part of README & csv.erl file. Please find them in the project: https://github.com/fogfish/csv Then, I have put extra example in csv_example.erl to show how to perform intake procedure to ets table. - Dmitry On Mar 25, 2012, at 5:52 PM, Tim Watson wrote: > Thanks for posting this detail Dmitry, it's very helpful. I'm going to > recompile with +native and give it a go. I also note that you're not > dealing with nested delimiters (e.g., where a field is enclosed in > single or double "quotation marks" and can therefore contain the > delimiters). If I can get the speed up to what you're seeing, then I > might just send you a couple of pull requests. :) > > Also I'm interested to know how you called the csv:parse/3 function > (or pparse instead) for these examples? Are you, for example, > accumulating the results, or are you just counting the occurrences? > Please let me know so I can replicate the same setup and test +native > for myself. > > Also would you be kind enough to let us all know what kind of hardware > your mac mini has, especially in terms of CPU and available memory? > > All in all, if we can clear up the differences I think parsing 300k in > just under a second without having to resort to a NIF is a very good > result and makes this the fastest csv parsing facility we've seen so > far!!! > > On 25 March 2012 11:09, Dmitry Kolesnikov wrote: >> Hello, >> >> 1. Yes, I'd like to admit that I've missed the example file from the beginning of thread. I've put my sets generators into the repository. You can use them like this: >> sh ./priv/dset line-ex.txt 1000 > ./priv/set-1M-ex.txt >> The first parameter is line, second is kilolines to generate. line-16/48 are my original examples, line-ex.txt is from original example. >> >> 2. I am a bit of curious why we are getting results of different magnitude, especially with Robert's run. My HW config is pretty close. So, I've build Erlang R15B from sources with following config: >> ./configure --prefix=/usr/local/otp-R15B --enable-threads --enable-smp-support --enable-kernel-poll --enable-sctp --enable-hipe --disable-dynamic-ssl-lib --enable-darwin-64bit --enable-m64-build --without-javac >> >> 3. I've re-run cases again with/without +native flag results are very interesting, so we can parse 300K lines in less then second, less the second for 1M rows is challenging: >> >> Parse time of data set is 300K: >> set-300K-16 653 ms / 2.18 us per line >> set-300K-48 1832 ms / 6.11 us per line >> set-300K-ex 2561 ms / 8.54 us per line >> >> Parse time of data set is 300K +native: >> set-300K-16 277 ms / 0.92 us per line >> set-300K-48 672 ms / 2.24 us per line >> set-300K-ex 925 ms / 3.09 us per line >> >> Parse time of data set is 1M: >> set-300K-16 4406 ms / 2.20 us per line >> set-300K-48 6076 ms / 6.08 us per line >> set-300K-ex 8670 ms / 8.67 us per line >> >> Parse time of data set is 1M +native: >> set-300K-16 1908 ms / 0.95 us per line >> set-300K-48 2293 ms / 2.29 us per line >> set-300K-ex 3119 ms / 3.12 us per line >> >> 4. It might be unfair to comment but I have a good feeling that Max is evaluating/challenging a type of trade system ;-) Otherwise, why you have such hard latency requirements and dataset looks like a price snapshot ;-) IMHO, 2 - 4 us parse time per row is acceptable for "normal" web app. >> >> Regards, Dmitry >> >> On Mar 25, 2012, at 5:57 AM, Tim Watson wrote: >> >>> On 25 Mar 2012, at 03:54, Tim Watson wrote: >>>> Please also do run this code on some other (faster) machines to see how much of the slowness is specific to my machine. >>> >>> Oh and can you please pass on the set and set2 text files so I can test them on my machine to baseline the comparative differences between our environments? >>> >>> Cheers! >>> >>> Tim >> From max.lapshin@REDACTED Sun Mar 25 23:26:43 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 26 Mar 2012 01:26:43 +0400 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <1C468E9B-5697-4BEA-90CB-6E53275D958B@gmail.com> References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> <1C468E9B-5697-4BEA-90CB-6E53275D958B@gmail.com> Message-ID: > > 3. I've tried to improve the documentation as part of README & csv.erl file. Please find them in the project: https://github.com/fogfish/csv > Can you commit some script to compare my benchmarks against your numbers? From james@REDACTED Mon Mar 26 00:48:27 2012 From: james@REDACTED (james) Date: Sun, 25 Mar 2012 23:48:27 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: Message-ID: <4F6FA0BB.7020202@mansionfamily.plus.com> > mmap is the fastest way to read lines is you don't much care about portability. While I think mmap is useful (and might see it as a way to avoid a split being sequential, since you only really need to divide it roughly and can probe for EoL), I think its worth qualifying your statement. mmap is NOT necessarily the fastest way to read lines. The issue is whether the operating system will perform read-ahead in its disk system and how many times you fault and wait for a real disk IO if not, so its rather important to know if the file is actually in the operating system's VM cache already, or is actually on a disk drive. As a first cut it would be handy to know how fast the OP's hardware can do atof, compared with the number of numbers in the file. From anarchocptlist@REDACTED Mon Mar 26 04:47:14 2012 From: anarchocptlist@REDACTED (Antonio SJ Musumeci) Date: Sun, 25 Mar 2012 22:47:14 -0400 Subject: [erlang-questions] Scheduling/messaging overhead Message-ID: I've got the following gen_server. start() -> gen_server:start(?MODULE,[],[]). launch() -> [test:start() || _ <- lists:seq(1,10000)]. init([]) -> random:seed(now()), {ok, #state{}, timer:seconds(random:uniform(10))}. handle_info(timeout,State) -> {noreply,State,timer:seconds(10)}; I'm just starting beam and issuing test:launch(). Using erl -smp disable I'm getting 8-9% cpu utilization according to top on a Core i7 2.67Ghz laptop. With erl -smp auto I'm getting as high as 25% for a bit and gradually reduces to low teens. After changing it to a single process with a timeout of 10 milliseconds I'm getting around 8% for smp and not. Is waking 100 times a second really that costly? I'm not planning on doing the above but in a platform I'm working on the intent is to have possibly tens of thousands of processes representing individual elements in external systems. I was trying to see what the load would be updating ETS from each process once every few seconds and it nearly pegged all the cpus with a few tens of thousands of processes which were using the timeout and then inserting a few times. A quick test of 1 process timing out every 100 milliseconds creates a beam load of 1%. Adding dozens of ets updates make no notable impact. Still 1% with the handle_info calling 50 individual ets:insert's. Should I not be timing out like that to simulate random writes? If I had the updates being triggered due to messages from ports would it be about the same? Or would it make more sense to not create one process per logical entity? Let the true originator of the data write it to ETS? Or a proxy for a class of objects? It was designed to be 1 process per entity so as to be persuasively concurrent and give the appearance of 1 to 1 mapping but in practice it may be that most processes rely largely on external triggers to do anything and therefore could be normalized. Thanks. From michael.eugene.turner@REDACTED Mon Mar 26 05:00:14 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Mon, 26 Mar 2012 12:00:14 +0900 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> Message-ID: > The kind of lines in the generated example.csv BTW look like this: > > KEY1,20120201,12:15:44.543,34.28,54.74,16.74,88.51,32.48,15.7,54.19,71.52,69.5,55.35,3.9,20.08,33.83,63.43,12.4,9.66,0.29,59.61,52.94,82.49,78.96,70.52,55.73,79.37,61.25,54.19,49.31,14.14,40.18,21.39,82.26,40.79,36.57,86.14,39.58,28.3,20.1,24.07,51.35,8.38,zz Which made me wonder: is Max actually trying to parse a CSV file of *floating point* numbers, or just numbers that *happen* to be expressed in floating point notation but that can be far more narrowly characterized? If you have 1 million lines of positive 4-digit fixed point numbers with only two significant digits past the decimal point (as the above would suggest), the numbers will repeat (on average) about 100 times in the file. So you could at least save yourself a factor of 100 in the text-to-float conversion part of the problem. This *general* problem of parsing CSV is important too, of course. But the *general* solution could also (FTW) admit of such approaches in its API. -michael turner On Sun, Mar 25, 2012 at 11:54 AM, Tim Watson wrote: > On 25 Mar 2012, at 03:26, Tim Watson wrote: >>>> The test was run on Mac Mini server with R15B >>>> Erlang R15B (erts-5.9) [source] [64-bit] [smp:8:8] [async-threads:0] [hipe] [kernel-poll:false] >>>> >>>> Since, you have not shared a details of csv file I used the following setups: >>>> * set.txt contains 300K rows, each row has 16 fields of 3 hex digit each >>>> * set2.txt contains 300K rows, each row has 48 fields of 3 hex digit each >>>> >>> >>> This is a bit different to the input Max is trying to work with isn't it? Can you try with his generated example.csv? I tried doing this and your output seems to be a small list of numbers, so I'm not sure what's going wrong. >>> > > The kind of lines in the generated example.csv BTW look like this: > > KEY1,20120201,12:15:44.543,34.28,54.74,16.74,88.51,32.48,15.7,54.19,71.52,69.5,55.35,3.9,20.08,33.83,63.43,12.4,9.66,0.29,59.61,52.94,82.49,78.96,70.52,55.73,79.37,61.25,54.19,49.31,14.14,40.18,21.39,82.26,40.79,36.57,86.14,39.58,28.3,20.1,24.07,51.35,8.38,zz > >>> 13> csv_example:run("example.csv", 80). >>> total parse time: 5897.321 >>> Result = >>> [3711,3750,3750,3751,3751,3750,3750,3749,3750,3751,3751,3751,3750,3752,3751, >>> 3750,3750,3751,3751,3751,3750,3750,3751,3751,3750,3751,3751,3750,3753,3751, >>> 3749,3750,3751,3751,3750,3751,3750,3750,3751,3751,3750,3751,3751,3750,3750, >>> 3750,3751,3751,3751,3751,3751,3751,3750,3751,3751,3751,3749,3751,3749,3751, >>> 3751,3750,3751,3751,3750,3752,3750,3751,3750,3751,3750,3751,3749,3750,3750, >>> 3751,3750,3749,3750,3750] >>> >> >> On further inspection, I can see that this happens because the fun you pass to the parser (to handle the {line, _} event) is simply incrementing a count. My mistake! >> >>> >>>> Here is the results: >>>> csv_example:run("priv/set.txt", 80). >>>> {{lines,300000}, >>>> {size,18000000}, >>>> {read_ms,132.269}, >>>> {parse_ms,652.865}, >>>> {line_us,2.1762166666666665}} >>>> >>>> 2> csv_example:run("priv/set2.txt", 80). >>>> {{lines,300000}, >>>> {size,54000000}, >>>> {read_ms,204.259}, >>>> {parse_ms,1860.286}, >>>> {line_us,6.2009533333333335}} >>>> >>> >>> I changed the example code a little bit, to account for the fact that we consider the file:read_file and the parsing all together in all our other examples. I get very different results, although I'm admittedly (probably?) on a less powerful machine: >>> >>> 4@REDACTED:csv $ erl -pa src -pa priv >>> Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] >>> >>> Eshell V5.8.2 ?(abort with ^G) >>> 1> csv_example:run("example.csv", 80). >>> {{lines,300001}, >>> {size,78599457}, >>> {parse_ms,5727.757}, >>> {line_us,19.09245969180103}} >>> 2> > > Accumulating the results takes even longer than this though. I have the line event handler simply aggregate the lines and do the wall clock time measures as Max did (just for consistency), and try it with Wrk set to 80 and then larger (to 200) which starts to degrade somewhere between the two: > > test(Filename, Wrk) -> > ? ?T1 = erlang:now(), > ? ?{ok, Bin} = file:read_file(Filename), > ? ?R = csv:pparse(Bin, Wrk, > ? ? ? ? ? ? ? ? ?fun({line, Line}, Acc) -> [Line|Acc] end, []), > ? ?T2 = erlang:now(), > ? ?io:format("total parse time: ~p~n", [timer:now_diff(T2, T1) div 1000]), > ? ?io:format("size(Results) = ~p~n", [length(R)]). > > And here's what I get: > > Eshell V5.8.2 ?(abort with ^G) > 3> csv_example:test("example.csv", 80). > total parse time: 15232 > size(Results) = 80 > ok > 4> > 5> csv_example:test("example.csv", 200). > total parse time: 17086 > size(Results) = 200 > ok > 6> > > So these appear to be taking between 50 and 20 seconds, and of course, you've got to then work your way through the results in the data for each shard. This part has switched me on to the idea of having a stream oriented API - I suspect that will work well, although it's worth baring in mind that the ordering isn't going to be maintained if you start 'sharding' the data set so as to process it sequentially. > > So do we think the additional time is spent because of the comparative width of the line(s) in Max's inputs? I can't imagine that binary:split is doing anything vastly different, except that it uses binary:matches which is implemented as a BIF - I am guessing that a BIF is going to beat hand coded binary pattern matching every time, is it not? Please also do run this code on some other (faster) machines to see how much of the slowness is specific to my machine. > >>> >>>> - Dmitry >>>> >>>> >>>> On Mar 25, 2012, at 12:14 AM, Tim Watson wrote: >>>> >>>>> On 24 Mar 2012, at 16:45, Max Lapshin wrote: >>>>> >>>>>> I'm trying to use file:read from erlang combined with compiled scanf pattern or some regex. There are two ideas: >>>>>> 1) combine two binary:split will give double memory walking. I think it is a good idea to parse memory once >>>>> >>>>> Actually there's a double problem here. In a CSV file it is possible that a delimiter might be present 'inside' one of the fields, providing the contents of the cell are in quotes. For example: >>>>> >>>>>>> 1,2,16,John Snow,"Flat A, 128, Watling Street", etc.... >>>>> >>>>> So just doing binary:split(Bin, binary:compile_pattern(<<",">>), [global]) isn't enough. I will take a look how binary:split/3 is implemented, but I'm starting to think that you really do need to drop into C to make this fast enough. Maybe for your data set you don't care about this possibility (because you have some control over the inputs) but for a general purpose CSV parser, this has to be handled and requires too much introspection of the data to efficient over 70MiB and more if written in pure Erlang. >>>>> >>>>> With file:read_line and binary:split, I can get the data out (without float/int conversion) sequentially in about 10 seconds. It's not particularly useful to parallelise the binary:split work as for the most part it isn't taking a long time per operation, but performs a vast number of them which slows down. Spawning for each processing line isn't really very smart as even in SMP mode the amount of churn due to scheduling seems likely to be prohibitive. Even if you hit the sweet spot with the right number of worker processes (1 per cpu +/- 1 for example) you've got the additional work of reconstituting the data in the correct order which takes up a lot of time. >>>>> >>>>> Now if I manually chunk my way through the file, using an optimal segment size (on my mac, 16 * 1024 seems to be best) with a sizeable read_ahead of 1024 * 1024, then I can get through the whole file and binary:split(Bin, CommaPattern, [global]) on each chunk, in an average of 2300ms. Of course this is too slow, is ignoring newlines at the moment (unlike the other tests) and I've removed the code for dealing with fields that sit across the segment boundaries (which slows things down also): >>>>> >>>>> read(P=#csv_parser{ io_device=Fd, buffer_size=Size }, start, Idx, Rem) -> >>>>> read(P, file:read(Fd, Size), Idx, Rem); >>>>> read(P=#csv_parser{ io_device=Fd, buffer_size=Size, >>>>> ? ? ? ? ? ? ? ? delimiter=Delim, collector=Collector, >>>>> ? ? ? ? ? ? ? ? supported_newlines=NL }, >>>>> ? ? ? ? ? ? ? ? {ok, Chunks}, >>>>> ? ? ? ? ? ? ? ? Idx, Rem) -> >>>>> binary:split(Chunks, Delim, [global]), >>>>> read(P, file:read(Fd, Size), Idx + 1, <<>>); >>>>> read(_P, eof, _, _Rem) -> >>>>> %% in strict mode, fail unless size(Rem) == 0 >>>>> ok. >>>>> >>>>>>>>> >>>>> t4@REDACTED:csv_reader $ ./csv_reader.erl example.csv >>>>> Starting read.... >>>>> read_chunk time: 2376 >>>>> t4@REDACTED:csv_reader $ >>>>> >>>>> Now if I switch the split to use binary:matches instead, and look for either ',' or a newline, we get a big performance improvement: >>>>> >>>>> t4@REDACTED:csv_reader $ ./csv_reader.erl example.csv >>>>> Starting read.... >>>>> read_chunk time: 953 >>>>> t4@REDACTED:csv_reader $ >>>>> >>>>> That's getting much faster, *but* we're not doing any hard work of parsing yet and we're not dealing with 'cross-segment' parts, which I'll look at next. >>>>> >>>>>> 2) intermediate creation of binaries for all those floats (12 millions of them in this example) is also evil. Perhaps line parsing should be combined with float extraction in the same manner as decode_packet does. >>>>> >>>>> >>>>> Like I said I think that with the approach I mentioned above (using nicely segmented file:read calls and binary:matches) we've got the i/o subsystem and splitting of fields to go as fast as they're likely to go without dropping into C. Now we get into the more cpu intensive parts - taking the {Location, Size} data from binary:matches and combined with your segment Idx working out the correct offset to use in calling binary:part, plus reconstituting stuff that sits across two segments and then finally converting binary to other data types for the final results. If I could make these parts happen in 2 seconds to get combined throughput of 3 seconds I'd be pleased, but I'm not sure it'll be possible. And even then, your desired time of 1 second isn't looking at all likely for even this 300k record file, let alone a larger 1million line file. >>>>> >>>>> It is possible that applying some parallelism to this will help, but the timings we're after are so small that I'm not sure. Spawning per split is of course a waste of time, but it is possible that having a worker process per core that does binary:matches and parses might help. The difficulty here is that you have to deal with the data sitting across two segments before you can pass it off to a worker, because you need the previous 'remainder' if any - I suppose you could 'pass this in' along with the data. If this could be made to work efficiently, then you might also benefit from a parallel collector process taking the completed/parsed records from the workers and storing them so they're ready for retrieval. Again, I'm not convinced all of this could be done in pure Erlang in less than 3 - 4 seconds overall. Perhaps putting ets into the mix might provide some speedup, as I would guess that a shared table can be atomically accessed faster than the processes can exchange > ?data >>>>> via a mailbox. >>>>> >>>>> I'm going to see how fast this can be made to go in pure Erlang (allowing for ETS if it helps) just as a fun exercise. I don't know how much spare time I'll have to do it though and I'm convinced a solution written in C is the right answer for 1million records per second, and to do that a segment at a time approach that does almost all the work in C including all the type conversion is probably the way to go. In that case I would probably do the whole i/o part in C too, using a similar series of buffered read operations rather than line oriented fgets calls or things like that. >>>>> >>>>> I do not think that mmap or even reading all the file into memory will be of any use, as you mentioned earlier, the problem is really cpu bound. >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From chandrashekhar.mullaparthi@REDACTED Mon Mar 26 07:10:58 2012 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 26 Mar 2012 06:10:58 +0100 Subject: [erlang-questions] Extract DIAMETER data types Message-ID: Hi, Does anyone know of a relatively easy way to extract the specification for DIAMETER data types out of 3GPP specs? I'm looking at the spec for the S6A interface, and the amount of copy/paste/reformat required to get a spec which I can input to the diameterc utility shipped with erlang is daunting. I've managed to extract the ABNF for the messages relatively easily, but the number of AVP types are huge! If I get any contributions, I'll happily share the end result. cheers, Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon Mar 26 08:50:57 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 26 Mar 2012 07:50:57 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <1C468E9B-5697-4BEA-90CB-6E53275D958B@gmail.com> References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> <1C468E9B-5697-4BEA-90CB-6E53275D958B@gmail.com> Message-ID: Interesting. Like Max said, would you update or post the script you're using to test Max's inputs please, as I can't reproduce on a comparable system. I've tried Macbook Pro (Intel Core 2 Duo, 3.6GHz, 8Gb Ram, twice the L2 cache) and also on an HP with Intel Core i7 8Gb fast RAM and similar disk profile. I think the difference between our results has to do with how the output is captured and that's what I'd like to see (and be able to reproduce) - is this where you're utilising an ets table yes? That sounds like a sensible approach. Cheers, Tim On 25 March 2012 22:19, Dmitry Kolesnikov wrote: > Hello Tim, > > 1. Yes, you are right about nested/escaped delimiters. They have to be fixed. > > 2. Reference platform: > ? ? * MacMini, Lion Server, > ? ? * 1x Intel Core i7 (2 GHz), 4x cores > ? ? * L2 Cache 256KB per core > ? ? * L3 Cache 6MB > ? ? * Memory 4GB 1333 MHZ DDR3 > ? ? * Disk 750GB 7200rpm WDC WD7500BTKT-40MD3T0 > ? ? * erlang R15B + native build of the library > > 3. I've tried to improve the documentation as part of README & csv.erl file. Please find them in the project: https://github.com/fogfish/csv > > Then, I have put extra example in csv_example.erl to show how to perform intake procedure to ets table. > > - Dmitry > > > On Mar 25, 2012, at 5:52 PM, Tim Watson wrote: > >> Thanks for posting this detail Dmitry, it's very helpful. I'm going to >> recompile with +native and give it a go. I also note that you're not >> dealing with nested delimiters (e.g., where a field is enclosed in >> single or double "quotation marks" and can therefore contain the >> delimiters). If I can get the speed up to what you're seeing, then I >> might just send you a couple of pull requests. :) >> >> Also I'm interested to know how you called the csv:parse/3 function >> (or pparse instead) for these examples? Are you, for example, >> accumulating the results, or are you just counting the occurrences? >> Please let me know so I can replicate the same setup and test +native >> for myself. >> >> Also would you be kind enough to let us all know what kind of hardware >> your mac mini has, especially in terms of CPU and available memory? >> >> All in all, if we can clear up the differences I think parsing 300k in >> just under a second without having to resort to a NIF is a very good >> result and makes this the fastest csv parsing facility we've seen so >> far!!! >> >> On 25 March 2012 11:09, Dmitry Kolesnikov wrote: >>> Hello, >>> >>> 1. Yes, I'd like to admit that I've missed the example file from the beginning of thread. I've put my sets generators into the repository. You can use them like this: >>> sh ./priv/dset line-ex.txt 1000 > ./priv/set-1M-ex.txt >>> The first parameter is line, second is kilolines to generate. line-16/48 are my original examples, line-ex.txt is from original example. >>> >>> 2. I am a bit of curious why we are getting results of different magnitude, especially with Robert's run. My HW config is pretty close. So, I've build Erlang R15B from sources with following config: >>> ./configure --prefix=/usr/local/otp-R15B --enable-threads --enable-smp-support --enable-kernel-poll --enable-sctp --enable-hipe --disable-dynamic-ssl-lib --enable-darwin-64bit --enable-m64-build --without-javac >>> >>> 3. I've re-run cases again with/without +native flag results are very interesting, so we can parse 300K lines in less then second, less the second for 1M rows is challenging: >>> >>> Parse time of data set is 300K: >>> set-300K-16 ? ? 653 ms / 2.18 us per line >>> set-300K-48 ? 1832 ms / 6.11 us per line >>> set-300K-ex ? 2561 ms / 8.54 us per line >>> >>> Parse time of data set is 300K +native: >>> set-300K-16 ? ? 277 ms / 0.92 us per line >>> set-300K-48 ? ? 672 ms / 2.24 us per line >>> set-300K-ex ? ? 925 ms / 3.09 us per line >>> >>> Parse time of data set is 1M: >>> set-300K-16 ? 4406 ms / 2.20 us per line >>> set-300K-48 ? 6076 ms / 6.08 us per line >>> set-300K-ex ? 8670 ms / 8.67 us per line >>> >>> Parse time of data set is 1M +native: >>> set-300K-16 ? ?1908 ms / 0.95 us per line >>> set-300K-48 ? ?2293 ms / 2.29 us per line >>> set-300K-ex ? ?3119 ms / 3.12 us per line >>> >>> 4. It might be unfair to comment but I have a good feeling that Max is evaluating/challenging a type of trade system ;-) Otherwise, why you have such hard latency requirements and dataset looks like a price snapshot ;-) IMHO, 2 - 4 us parse time per row is acceptable for "normal" web app. >>> >>> Regards, Dmitry >>> >>> On Mar 25, 2012, at 5:57 AM, Tim Watson wrote: >>> >>>> On 25 Mar 2012, at 03:54, Tim Watson wrote: >>>>> Please also do run this code on some other (faster) machines to see how much of the slowness is specific to my machine. >>>> >>>> Oh and can you please pass on the set and set2 text files so I can test them on my machine to baseline the comparative differences between our environments? >>>> >>>> Cheers! >>>> >>>> Tim >>> > From watson.timothy@REDACTED Mon Mar 26 08:59:52 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 26 Mar 2012 07:59:52 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> Message-ID: On 26 March 2012 04:00, Michael Turner wrote: >> The kind of lines in the generated example.csv BTW look like this: >> >> KEY1,20120201,12:15:44.543,34.28,54.74,16.74,88.51,32.48,15.7,54.19,71.52,69.5,55.35,3.9,20.08,33.83,63.43,12.4,9.66,0.29,59.61,52.94,82.49,78.96,70.52,55.73,79.37,61.25,54.19,49.31,14.14,40.18,21.39,82.26,40.79,36.57,86.14,39.58,28.3,20.1,24.07,51.35,8.38,zz > > Which made me wonder: is Max actually trying to parse a CSV file of > *floating point* numbers, or just numbers that *happen* to be > expressed in floating point notation but that can be far more narrowly > characterized? If you have 1 million lines of positive 4-digit fixed > point numbers with only two significant digits past the decimal point > (as the above would suggest), the numbers will repeat (on average) > about 100 times in the file. So you could at least save yourself a > factor of 100 in the text-to-float conversion part of the problem. > > This *general* problem of parsing CSV is important too, of course. But > the *general* solution could also (FTW) admit of such approaches in > its API. > Hi michael. Yes I agree with this and in my own experiments (and in Dmitry's library) there is no data type conversion going on. Being able to parse a 300k line CSV in around a second is a pretty good achievement (once it's stable) and if turing off any automatic to_int/to_float conversion is necessary to save a few seconds of processing time, then I think most users will be happy with that. As you say, Max seems to have some more specialised requirements of the system in his 1 million rows in 1 second (on commodity hardware?) goal. > -michael turner From clist@REDACTED Mon Mar 26 09:44:58 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Mon, 26 Mar 2012 09:44:58 +0200 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: Message-ID: <201203260944.58509.clist@uah.es> On Domingo, 25 de Marzo de 2012 19:27:04 Joe Armstrong escribi?: > I'm joining this thread rather late, ... > > Would it be possible to change the problem a bit? > > The fastest approach would seem to be > (assume a quad core): > > - spilt file into four > - parse each bit in parallel > - combine the results > > Well splitting (the first part) is essentially sequential > (or at least difficult to do in parallel unless you really > understand your hardware - and may be impossible to do in parallel) Maybe opening the file 4 times (and apropiate fseeks on the second, thirds etc..)? You can read data from every descriptor so you get your file split (while you care not to overlap ranges during reads..) > > I would assume that the input file has been created > by a sequential program - if this were the case > could it not produce four files instead of one? > > If this were the case then the split into four bit would go away. > > Do the bits have to be recombined later? - if not the > last bit can be removed as well. > > The key to performance might be to redesign the entire > processing pipeline making it as parallel as possible > as soon as possible and keeping it as parallel as possible as long as > possible. > > Cheers > > /Joe > > On Fri, Mar 23, 2012 at 11:30 AM, Max Lapshin wrote: > > I need to load large CSV file into memory very fast. > > > > I've tried to use erlang parser, but my results were very bad (in fact > > file:read_line is very slow), so I've tried to make a NIF for it. > > Target speed is 1 microsecond per line. > > > > > > My CSV has very strict format: only numbers, no quoting, \n in the > > end. Also I moved parsing of integers and date into NIF. > > > > My results are here: http://github.com/maxlapshin/csv_reader and I get > > only 15 microseconds: 4,5 seconds for 300K lines CSV. > > > > Currently I use fgets to read line by line from file. Maybe it is a > > bad idea and I should use mmap or implement 1MB buffer for read? > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- - >>---------------------------------------------------------------------------- Angel J. Alvarez Miguel, Servicios Inform?ticos Edificio Torre de Control, Campus Externo UAH Alcal? de Henares 28806, Madrid ** ESPA?A ** -------------[taH pagh taHbe', DaH mu'tlheghvam v?qeln?S]--<<- From max.lapshin@REDACTED Mon Mar 26 10:23:10 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 26 Mar 2012 12:23:10 +0400 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <201203260944.58509.clist@uah.es> References: <201203260944.58509.clist@uah.es> Message-ID: Colleagues. There is absolutely no problem with disk IO. wc -l and erlang file:read both take about 100-200 ms to read whole file. Problem is in handling this data and parsing. From ingela.andin@REDACTED Mon Mar 26 10:33:04 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 26 Mar 2012 10:33:04 +0200 Subject: [erlang-questions] Linux ODBC socket recieve_msg woes In-Reply-To: References: Message-ID: Hi! 2012/3/23, Andy Richards : > Fixed. I found this forum message from back in 2004. > > http://erlang.org/pipermail/erlang-questions/2004-July/012808.html > > Editing odbcserver.c and disabling nagel's algorithm (approx 40ms on Redhat > 6) on the socket solved the problem. I wonder why this was never added to > odbcserver.c in the past? Humm... it was added on the erlang side but apparently not on the c-side. I can not remember why, if it was for some reason or maybe it was just an oversight. Please send your modifications as a patch. [...] Regards Ingela Erlang/OTP team - Ericsson AB From max.lapshin@REDACTED Mon Mar 26 10:37:17 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 26 Mar 2012 12:37:17 +0400 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <201203260944.58509.clist@uah.es> Message-ID: On Mon, Mar 26, 2012 at 12:33 PM, Robert Melton wrote: > > Agreed. ?Do we have any baseline implementation in pure C or (insert > fastest language/implementation you are aware of)? ?I am working on > speeding this up (and having a lot of fun!), but I have no idea the > theory-craft maximum process speed (with proper escaping, etc) on my > hardware. > I really can't understand why should parsing be slower than reading from HDD =) However, it is slower. Currently I have 950 ms for 300K line CSV with 40 float columns when read on cold system and 820 ms when read from disk cache. Copying from kernel cache and byte-by-byte reading all data while searching '\n' takes 100 ms (it is time of wc -l), so it takes about 700 ms for erlang to parse + create all proper objects. From bchesneau@REDACTED Mon Mar 26 10:58:50 2012 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 26 Mar 2012 10:58:50 +0200 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> <20744632.326.1332684734709.JavaMail.geo-discussion-forums@ynbq18> Message-ID: On Sun, Mar 25, 2012 at 7:47 PM, Max Lapshin wrote: > So, currently my time is 850 ms for 300K line file which is 2.8 us per line. > > https://github.com/maxlapshin/csv_reader > > $ ./csv_bench.erl example.csv can you provide the example.csv file somewhere ? From bchesneau@REDACTED Mon Mar 26 11:02:25 2012 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 26 Mar 2012 11:02:25 +0200 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> <20744632.326.1332684734709.JavaMail.geo-discussion-forums@ynbq18> Message-ID: On Mon, Mar 26, 2012 at 10:58 AM, Benoit Chesneau wrote: > On Sun, Mar 25, 2012 at 7:47 PM, Max Lapshin wrote: >> So, currently my time is 850 ms for 300K line file which is 2.8 us per line. >> >> https://github.com/maxlapshin/csv_reader >> >> $ ./csv_bench.erl example.csv > > can you provide the example.csv file somewhere ? nm... $ ruby fill.rb > example.csv did the trick. From lukasp.p017@REDACTED Mon Mar 26 11:59:28 2012 From: lukasp.p017@REDACTED (Lukas P) Date: Mon, 26 Mar 2012 11:59:28 +0200 Subject: [erlang-questions] ssl timing issue in R15B? In-Reply-To: References: Message-ID: Dne 23. b?ezna 2012 18:16 Attila Rajmund Nohl napsal(a): > 2012/3/23 Lukas P : >> Hello. >> >> I think that I have been hit by a SSL timing issue in R15B. My >> functional test (= SSL/TCP client) crahes my server unless the test >> contains a sleep. >> >> The tested server implements a TCP based, line oriented protocol. >> Switch to SSL can be started with a "STARTTLS\r\n" command. >> >> The functional test starts a TCP connection and upgrades it to SSL: >> >> upgrade_connection_to_ssl(Socket) -> >> ? ?SendResult = gen_tcp:send(Socket, "STARTTLS\r\n"), >> ? ?%timer:sleep(10), % workaround for a SSL timing bug >> ? ?SslConnectResult = case SendResult of >> ? ? ? ?ok -> >> ? ? ? ? ? ?ssl:connect(Socket, [{active, false}, {packet, line}, list], 2000); >> ? ? ? ?{error, Reason1} -> >> ? ? ? ? ? ?{error, Reason1} >> ? ?end, >> ? ?... >> >> After the server receives "STARTTLS\r\n", it performs ssl:ssl_accept >> on the socket: >> >> ? ?inet:setopts(Socket, [{active, false}]), > > Don't you need a {reuseaddr, true} option here? The default is false. Unfortunately not, the tcp socket is already connected. When you look at the walkback, you can see that there is definitely a bug in ssl_record:get_tls_records_aux/2 (erlang:size/1 must not be used on a list). Lukas From michael.eugene.turner@REDACTED Mon Mar 26 12:36:28 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Mon, 26 Mar 2012 19:36:28 +0900 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <201203260944.58509.clist@uah.es> Message-ID: > I really can't understand why should parsing be slower than reading from HDD =) Are you converting the ASCII-coded floating point numbers to actual floating point? That's actually quite a lot more overhead per character than ... well, anything else I can think of in processing a CSV file. And what do these numbers look like? Do they repeat? Are they short? Or are they high-precision and varying wildly in order of magnitude, and widely distributed statistically? -michael turner On Mon, Mar 26, 2012 at 5:37 PM, Max Lapshin wrote: > On Mon, Mar 26, 2012 at 12:33 PM, Robert Melton wrote: >> >> Agreed. ?Do we have any baseline implementation in pure C or (insert >> fastest language/implementation you are aware of)? ?I am working on >> speeding this up (and having a lot of fun!), but I have no idea the >> theory-craft maximum process speed (with proper escaping, etc) on my >> hardware. >> > > I really can't understand why should parsing be slower than reading from HDD =) > > However, it is slower. Currently I have 950 ms for 300K line CSV with > 40 float columns when read on cold system and 820 ms when read from > disk cache. > > Copying from kernel cache and byte-by-byte reading all data while > searching '\n' takes 100 ms (it is time of wc -l), so it takes about > 700 ms for erlang to parse + create all proper objects. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From anders.otp@REDACTED Mon Mar 26 12:57:16 2012 From: anders.otp@REDACTED (Anders Svensson) Date: Mon, 26 Mar 2012 12:57:16 +0200 Subject: [erlang-questions] Extract DIAMETER data types In-Reply-To: References: Message-ID: On Mon, Mar 26, 2012 at 7:10 AM, Chandru wrote: > Hi, > > Does anyone know of a relatively easy way to extract the specification for > DIAMETER data types out of 3GPP specs? I'm looking at the spec for the S6A > interface, and the amount of copy/paste/reformat required to get a spec > which I can input to the diameterc utility shipped with erlang is daunting. Saving 3GPP specs as text and then formulating a sed script to extract/unmangle/massage the relevant bits is the best I've been able to do. RFC's are certainly easier to deal with. /Anders, Erlang/OTP Ericsson > > I've managed to extract the ABNF for the messages relatively easily, but the > number of AVP types are huge! If I get any contributions, I'll happily share > the end result. > > cheers, > Chandru > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf@REDACTED Mon Mar 26 13:04:35 2012 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 26 Mar 2012 04:04:35 -0700 Subject: [erlang-questions] Extract DIAMETER data types In-Reply-To: References: Message-ID: <3366FE93-93B9-43D2-A626-9AE5DC58688C@feuerlabs.com> Ahh, the memories? I played around a little bit, but no guarantees: - 2972-910, right? I downloaded the doc and opened in OpenOffice Writer - Delete the two first rows and the last row of the table (the ones that span the whole table) - Delete the columns "Section defined", "May", "should not", "must not", "may encr." - Select the AVP type data and chose Table->Convert->Table to text (tab-delimited) - Copy-paste the result into emacs - Search-replace all "M, V" with "MV" - There is one "M. V", fix that one too - Remove the space in the name "Requested-UTRAN- GERAN-Authentication-Info" - Fix the type type for Alert-Reason ("Enumerate" -> "Enumerated") Then I just threw in a fake @vendor line to make diameter_dict_util:parse/2 happy. The attached file at least makes it through the parser this way. Considering how many errors these specs contain, I can't vouch for the correctness beyond that. ;) BR, Ulf W (in SF, can't sleep) -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: v6_avps.txt URL: -------------- next part -------------- On 25 Mar 2012, at 22:10, Chandru wrote: > Hi, > > Does anyone know of a relatively easy way to extract the specification for DIAMETER data types out of 3GPP specs? I'm looking at the spec for the S6A interface, and the amount of copy/paste/reformat required to get a spec which I can input to the diameterc utility shipped with erlang is daunting. > > I've managed to extract the ABNF for the messages relatively easily, but the number of AVP types are huge! If I get any contributions, I'll happily share the end result. > > cheers, > Chandru > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Mon Mar 26 13:40:25 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 26 Mar 2012 15:40:25 +0400 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <201203260944.58509.clist@uah.es> Message-ID: > > And what do these numbers look like? Do they repeat? Are they short? Right as in example csv. It is trading data. > Or are they high-precision and varying wildly in order of magnitude, > and widely distributed statistically? They are very close to each other and vary not more than several percents. You think ot is a good place for optimization? In fact I have achieved good enough results: less than a second and thank to all community for it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon Mar 26 14:16:28 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 26 Mar 2012 13:16:28 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> Message-ID: Hi Dmitry, On 25 March 2012 11:09, Dmitry Kolesnikov wrote: > Hello, > > 1. Yes, I'd like to admit that I've missed the example file from the beginning of thread. I've put my sets generators into the repository. You can use them like this: > sh ./priv/dset line-ex.txt 1000 > ./priv/set-1M-ex.txt > The first parameter is line, second is kilolines to generate. line-16/48 are my original examples, line-ex.txt is from original example. > What is line-ex.txt supposed to contain - is it a single line from the original example? - I can't find it in the repo. > 2. I am a bit of curious why we are getting results of different magnitude, especially with Robert's run. My HW config is pretty close. So, I've build Erlang R15B from sources with following config: > ./configure --prefix=/usr/local/otp-R15B --enable-threads --enable-smp-support --enable-kernel-poll --enable-sctp --enable-hipe --disable-dynamic-ssl-lib --enable-darwin-64bit --enable-m64-build --without-javac I'll rebuild that way, but essentially this is my build config looked something like this (from config.log): configure:4674: running /bin/sh '/Users/t4/.kerl/builds/r15b-64-hipe-smp/otp_src_R15B/lib/configure' --prefix=/usr/local '--cache-file=/dev/null' '--enable-hipe' '--enable-darwin-64bit' '--enable-threads' '--enable-smp' '--enable-kernel-poll' 'ERL_TOP=/Users/t4/.kerl/builds/r15b-64-hipe-smp/otp_src_R15B' --cache-file=/dev/null --srcdir=/Users/t4/.kerl/builds/r15b-64-hipe-smp/otp_src_R15B/lib > > 3. I've re-run cases again with/without +native flag results are very interesting, so we can parse 300K lines in less then second, less the second for 1M rows is challenging: > > Parse time of data set is 300K: > set-300K-16 ? ? 653 ms / 2.18 us per line > set-300K-48 ? 1832 ms / 6.11 us per line > set-300K-ex ? 2561 ms / 8.54 us per line > > Parse time of data set is 300K +native: > set-300K-16 ? ? 277 ms / 0.92 us per line > set-300K-48 ? ? 672 ms / 2.24 us per line > set-300K-ex ? ? 925 ms / 3.09 us per line > I can't replicate this even with +native turned on (for both the csv modules and the csv_example) and I'm bemused as you don't have 'that much more' horsepower in your 4 cores than my mbpro's 2. Here's what I get for parsing the original file - as you would expect, there is very little difference in setting the segments to any number higher than the # cpus: 5> csv_example:import("example.csv", 40). size (MB): 74.958283 read (ms): 109.798000 parse (ms): 3239.264000 266257 6> csv_example:import("example.csv", 80). size (MB): 74.958283 read (ms): 105.949000 parse (ms): 3231.541000 270352 7> csv_example:import("example.csv", 4). size (MB): 74.958283 read (ms): 102.250000 parse (ms): 3275.317000 274450 8> csv_example:parse("example.csv", 4). lines: 300001 size (MB): 74.958283 read (ms): 102.768000 parse (ms): 2737.098000 per line (us): 9.123630 ok 9> csv_example:parse("example.csv", 44). lines: 300001 size (MB): 74.958283 read (ms): 106.153000 parse (ms): 2775.781000 per line (us): 9.252572 ok 10> csv_example:parse("example.csv", 2). lines: 300001 size (MB): 74.958283 read (ms): 104.410000 parse (ms): 2758.367000 per line (us): 9.194526 ok 11> csv_example:import("example.csv", 2). size (MB): 74.958283 read (ms): 108.705000 parse (ms): 3390.453000 278547 How did you build Max's example file? I'm really struggling to understand how I've got 2 seconds more processing time for such a similar setup. > Parse time of data set is 1M: > set-300K-16 ? 4406 ms / 2.20 us per line > set-300K-48 ? 6076 ms / 6.08 us per line > set-300K-ex ? 8670 ms / 8.67 us per line > > Parse time of data set is 1M +native: > set-300K-16 ? ?1908 ms / 0.95 us per line > set-300K-48 ? ?2293 ms / 2.29 us per line > set-300K-ex ? ?3119 ms / 3.12 us per line > > 4. It might be unfair to comment but I have a good feeling that Max is evaluating/challenging a type of trade system ;-) Otherwise, why you have such hard latency requirements and dataset looks like a price snapshot ;-) IMHO, 2 - 4 us parse time per row is acceptable for "normal" web app. I agree your numbers seem perfectly reasonable, but I'm not able to reproduce them. I'm going to try on a 2 core linux machine later on and see how I get on. > > Regards, Dmitry > > On Mar 25, 2012, at 5:57 AM, Tim Watson wrote: > >> On 25 Mar 2012, at 03:54, Tim Watson wrote: >>> Please also do run this code on some other (faster) machines to see how much of the slowness is specific to my machine. >> >> Oh and can you please pass on the set and set2 text files so I can test them on my machine to baseline the comparative differences between our environments? >> >> Cheers! >> >> Tim > From attila.r.nohl@REDACTED Mon Mar 26 14:21:07 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 26 Mar 2012 14:21:07 +0200 Subject: [erlang-questions] Performance SNMP counters for the erlang VM itself Message-ID: Hello! I would like to get "relevant for performance" data out of an Erlang VM via SNMP. What do you think, which data is important? Number of processes? Length of messages queues? Number of reductions? Is there already an interface like this? This would be used for performance monitoring, so these counters could be queried about every 1 or 5 or 15 minutes and it would be good if these could indicate that there will be problems with performance. From joelr1@REDACTED Mon Mar 26 14:42:23 2012 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 26 Mar 2012 13:42:23 +0100 Subject: [erlang-questions] say no to aws dynamodb boilerplate and welcome dsl! Message-ID: Here's a DSL I whipped up to simplify working with Amazon DynamoDB by automatically generating Erlang code. With a bit of thinking it could be extended to SimpleDB, Python, etc. I don't have a name for this language so I'm calling it DSL for "Declarative Schema Language" or maybe "Dynamo Schema Language". Feel free to propose a better name. Here's an example schema. More comments further down. --- record Publisher id string (auto primary key) # auto-generated hash key stacks [Stack.id] # set of stack ids, starts empty sub-count count (Subscription.stack.publisher.id) end record Stack id string (auto primary key) publisher Publisher.id subscriptions [Subscription.id] end #enum Frequency # disabled # immediate # daily # weekly # monthly #end record Subscription id string (auto primary key) user User.id stack Stack.id # notify-freq Frequency notify-freq string language string last-notified date end record NewDocs stack Stack.id (primary key) add-date date (secondary key) doc-id string end record User id string (primary key) # Issuu id? subscriptions [Subscription.id] end record UnconfirmedEmails id string (auto primary key) email string end --- This generates Erlang code using http://github.com/wagerlabs/ddb that looks like this --- -module(publisher). -export([setup/0, teardown/0, put/1, get/1, delete/1]). setup() -> ddb:create_table(<<"Publisher">>, ddb:key_type(<<"id">>, 'string'), 10, 10). teardown() -> ddb:remove_table(<<"Publisher">>). --- and this --- -module(newdocs). -export([setup/0, teardown/0, put/3, get/3, delete/3]). setup() -> ddb:create_table(<<"NewDocs">>, ddb:key_type(<<"stack">>, 'string', <<"add_date">>, 'string'), 10, 10). teardown() -> ddb:remove_table(<<"NewDocs">>). --- A bunch of code is missing but you should be able to auto-magically update Publisher.stacks with Stack.id when a new stack is created where Stack.publisher matches Publisher.id. It's gets even better and you should be able to bump Publisher.sub-count (subscription count) when a new subscription is added to a publisher's stack! Of course you should also be able to have put and get functions to create, read and update the items, as well as functions to manipulate the sets, e.g. Publisher.stacks, Stack.subscriptions and User.subscriptions. What do you think? Do you think this can be accomplished using the SQL syntax? How would you update the Publisher.sub-count counter then? Thanks, Joel P.S. I wrote the compiler in OCaml and it has a proper lexer, parser, AST and code generator for Erlang. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From ingela@REDACTED Mon Mar 26 15:16:29 2012 From: ingela@REDACTED (Ingela Andin) Date: Mon, 26 Mar 2012 15:16:29 +0200 Subject: [erlang-questions] ssl timing issue in R15B? In-Reply-To: References: Message-ID: Hi! The problem is that your server and client needs to agree to upgrade to SSL/TLS and your client may not start the handshake until the server is ready to receive it. If you have a erlang-server this will be when you have set your tcp-socket in passive mode. When your server has set its socket in passive mode, you can ack to the client that it is ready to start and then call ssl:ssl_accept with the socket. Otherwhise a quick clients handshake may arrive to the inet driver and be processed with socket options that the ssl-application has no controll over. This may, depending on timing, result in that the ssl-connection process recives data on an unexpected format. It could also lead to that the some other process receives the handshake data and if you are lucky it may work as expected ;) Regards Ingela Erlang/OTP team - Ericsson AB 2012/3/26 Lukas P : > Dne 23. b?ezna 2012 18:16 Attila Rajmund Nohl > napsal(a): >> 2012/3/23 Lukas P : >>> Hello. >>> >>> I think that I have been hit by a SSL timing issue in R15B. My >>> functional test (= SSL/TCP client) crahes my server unless the test >>> contains a sleep. >>> >>> The tested server implements a TCP based, line oriented protocol. >>> Switch to SSL can be started with a "STARTTLS\r\n" command. >>> >>> The functional test starts a TCP connection and upgrades it to SSL: >>> >>> upgrade_connection_to_ssl(Socket) -> >>> ? ?SendResult = gen_tcp:send(Socket, "STARTTLS\r\n"), >>> ? ?%timer:sleep(10), % workaround for a SSL timing bug >>> ? ?SslConnectResult = case SendResult of >>> ? ? ? ?ok -> >>> ? ? ? ? ? ?ssl:connect(Socket, [{active, false}, {packet, line}, list], 2000); >>> ? ? ? ?{error, Reason1} -> >>> ? ? ? ? ? ?{error, Reason1} >>> ? ?end, >>> ? ?... >>> >>> After the server receives "STARTTLS\r\n", it performs ssl:ssl_accept >>> on the socket: >>> >>> ? ?inet:setopts(Socket, [{active, false}]), >> >> Don't you need a {reuseaddr, true} option here? The default is false. > > Unfortunately not, the tcp socket is already connected. > > When you look at the walkback, you can see that there is definitely a > bug in ssl_record:get_tls_records_aux/2 (erlang:size/1 must not be > used on a list). > > Lukas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From chandrashekhar.mullaparthi@REDACTED Mon Mar 26 15:24:59 2012 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 26 Mar 2012 14:24:59 +0100 Subject: [erlang-questions] Extract DIAMETER data types In-Reply-To: <3366FE93-93B9-43D2-A626-9AE5DC58688C@feuerlabs.com> References: <3366FE93-93B9-43D2-A626-9AE5DC58688C@feuerlabs.com> Message-ID: Nearly. 29.272-960. Thanks very much for the AVPs, Ulf. Once I have a working version, I'll host the entire spec on github. cheers Chandru PS: Hopefully this email won't wake you up if you've finally managed to sleep! On 26 March 2012 12:04, Ulf Wiger wrote: > > Ahh, the memories? > > I played around a little bit, but no guarantees: > > - 2972-910, right? I downloaded the doc and opened in OpenOffice Writer > - Delete the two first rows and the last row of the table (the ones that > span the whole table) > - Delete the columns "Section defined", "May", "should not", "must not", > "may encr." > - Select the AVP type data and chose Table->Convert->Table to text > (tab-delimited) > - Copy-paste the result into emacs > - Search-replace all "M, V" with "MV" > - There is one "M. V", fix that one too > - Remove the space in the name "Requested-UTRAN- GERAN-Authentication-Info" > - Fix the type type for Alert-Reason ("Enumerate" -> "Enumerated") > > Then I just threw in a fake @vendor line to make > diameter_dict_util:parse/2 happy. > > The attached file at least makes it through the parser this way. > Considering how many errors these specs contain, I can't vouch for the > correctness beyond that. ;) > > BR, > Ulf W (in SF, can't sleep) > > > > On 25 Mar 2012, at 22:10, Chandru wrote: > > > Hi, > > > > Does anyone know of a relatively easy way to extract the specification > for DIAMETER data types out of 3GPP specs? I'm looking at the spec for the > S6A interface, and the amount of copy/paste/reformat required to get a spec > which I can input to the diameterc utility shipped with erlang is daunting. > > > > I've managed to extract the ABNF for the messages relatively easily, but > the number of AVP types are huge! If I get any contributions, I'll happily > share the end result. > > > > cheers, > > Chandru > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andy.richards.iit@REDACTED Mon Mar 26 15:26:08 2012 From: andy.richards.iit@REDACTED (Andy Richards) Date: Mon, 26 Mar 2012 14:26:08 +0100 Subject: [erlang-questions] Performance SNMP counters for the erlang VM itself In-Reply-To: References: Message-ID: Hi Attila, I too have a similar requirement. My thoughts we to expose such stats via a restful interface by bundling Mochiweb or something similar with my application. My operations team can then use a monitoring probe which understands rest and the stats I'm exposing to monitor the health of my Erlang cluster. I'm pretty new to erlang so I also would be interested to hear what the community suggests? Cheers, Andy. On Monday, 26 March 2012, Attila Rajmund Nohl wrote: > Hello! > > I would like to get "relevant for performance" data out of an Erlang > VM via SNMP. What do you think, which data is important? Number of > processes? Length of messages queues? Number of reductions? Is there > already an interface like this? This would be used for performance > monitoring, so these counters could be queried about every 1 or 5 or > 15 minutes and it would be good if these could indicate that there > will be problems with performance. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Mon Mar 26 15:27:13 2012 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 26 Mar 2012 14:27:13 +0100 Subject: [erlang-questions] Extract DIAMETER data types In-Reply-To: References: Message-ID: On 26 March 2012 11:57, Anders Svensson wrote: > On Mon, Mar 26, 2012 at 7:10 AM, Chandru > wrote: > > Hi, > > > > Does anyone know of a relatively easy way to extract the specification > for > > DIAMETER data types out of 3GPP specs? I'm looking at the spec for the > S6A > > interface, and the amount of copy/paste/reformat required to get a spec > > which I can input to the diameterc utility shipped with erlang is > daunting. > > Saving 3GPP specs as text and then formulating a sed script to > extract/unmangle/massage the relevant bits is the best I've been able > to do. RFC's are certainly easier to deal with. > > Thanks Anders. I can never understand why these bits of the specification cannot be published separately as a text file, like they do with ASN.1 specs. Surely I can't be the first person to want them... Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal.ptaszek@REDACTED Mon Mar 26 15:34:40 2012 From: michal.ptaszek@REDACTED (=?utf-8?Q?Micha=C5=82?= Ptaszek) Date: Mon, 26 Mar 2012 14:34:40 +0100 (BST) Subject: [erlang-questions] current xml parsers In-Reply-To: Message-ID: <9d415d09-da0f-448c-bdeb-56bf8d69399a@knuth> Hi Roberto, you might want to be interested in looking at exml: it's a very simple NIF-based parser built around expat library: https://github.com/paulgray/exml Basing on some of the simple benchmarks of my own it should be 2-3 times faster than xmerl. Best regards, Michal Ptaszek ----- Original Message ----- > Dear list, > > does someone have recent considerations on xml parsers in terms of > memory > footprint, parsing speed and stability? > > The ones I'm aware of are xmerl, erlsom [1] and the driver used in > ejabberd > (which unfortunately is GPL). > > I don't care about DTD validation. > > Thank you, > > r. > > [1] http://erlsom.sourceforge.net/erlsom.htm > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From vances@REDACTED Mon Mar 26 17:14:37 2012 From: vances@REDACTED (Vance Shipley) Date: Mon, 26 Mar 2012 20:44:37 +0530 Subject: [erlang-questions] Performance SNMP counters for the erlang VM itself In-Reply-To: References: Message-ID: <20120326151424.GA356@aluminum.wavenet.lk> Attila, OTP has great SNMP support. The otp_mibs application makes VM performance data available: http://www.erlang.org/doc/apps/otp_mibs/introduction.html The os_mon application handles operating system stuff: http://www.erlang.org/doc/man/os_mon_app.html An example is here: http://www.trapexit.org/SNMP_Quick_Start#Walking_the_OTP_MIB On Mon, Mar 26, 2012 at 02:21:07PM +0200, Attila Rajmund Nohl wrote: } I would like to get "relevant for performance" data out of an Erlang } VM via SNMP. What do you think, which data is important? Number of } processes? Length of messages queues? Number of reductions? Is there } already an interface like this? This would be used for performance } monitoring, so these counters could be queried about every 1 or 5 or } 15 minutes and it would be good if these could indicate that there } will be problems with performance. } _______________________________________________ } erlang-questions mailing list } erlang-questions@REDACTED } http://erlang.org/mailman/listinfo/erlang-questions -- -Vance From marc@REDACTED Mon Mar 26 17:18:06 2012 From: marc@REDACTED (Marc Worrell) Date: Mon, 26 Mar 2012 17:18:06 +0200 Subject: [erlang-questions] Erlang Jobs? In-Reply-To: <4F6B8A60.7050006@mansionfamily.plus.com> References: <4F6B8A60.7050006@mansionfamily.plus.com> Message-ID: <7F71E240-81C5-4E45-96CD-DF9EABAF8490@worrell.nl> You can also check: http://totally-erlang.com/ There is a steady stream of Erlang jobs posted there. And all jobs are mailed to registered interested people. - Marc On 22 mrt 2012, at 21:24, james wrote: > Are there any? > > I guess to be fair (since there aren't many of any sort around right now) - are there normally any (in London)? > > Weird that there seem to be none on TotalJobs. Even for small software houses trying it on. > > James > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From andy.richards.iit@REDACTED Mon Mar 26 18:50:35 2012 From: andy.richards.iit@REDACTED (Andy Richards) Date: Mon, 26 Mar 2012 17:50:35 +0100 Subject: [erlang-questions] Linux ODBC socket recieve_msg woes In-Reply-To: References: Message-ID: Hi Ingela, I shall do. Should I send this patch to the erlang-patches mailing list or submit a patch via GitHub? Andy. On Monday, 26 March 2012, Ingela Andin wrote: > Hi! > > 2012/3/23, Andy Richards : >> Fixed. I found this forum message from back in 2004. >> >> http://erlang.org/pipermail/erlang-questions/2004-July/012808.html >> >> Editing odbcserver.c and disabling nagel's algorithm (approx 40ms on Redhat >> 6) on the socket solved the problem. I wonder why this was never added to >> odbcserver.c in the past? > > Humm... it was added on the erlang side but apparently not on the c-side. I can > not remember why, if it was for some reason or maybe it was just an oversight. > Please send your modifications as a patch. > > [...] > > Regards Ingela Erlang/OTP team - Ericsson AB > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rapsey@REDACTED Mon Mar 26 20:54:27 2012 From: rapsey@REDACTED (Rapsey) Date: Mon, 26 Mar 2012 20:54:27 +0200 Subject: [erlang-questions] current xml parsers In-Reply-To: <9d415d09-da0f-448c-bdeb-56bf8d69399a@knuth> References: <9d415d09-da0f-448c-bdeb-56bf8d69399a@knuth> Message-ID: I think I tried that lib once, because I had the requirement to parse WURFL http://wurfl.sourceforge.net/ It is a giant XML that has info about mobile devices. It worked fine with normal sized XMLs, but WURFL crashed it (and erlang along with it). Sergej On Mon, Mar 26, 2012 at 3:34 PM, Micha? Ptaszek < michal.ptaszek@REDACTED> wrote: > Hi Roberto, > > you might want to be interested in looking at exml: > it's a very simple NIF-based parser built around > expat library: > https://github.com/paulgray/exml > > Basing on some of the simple benchmarks of my own it > should be 2-3 times faster than xmerl. > > Best regards, > Michal Ptaszek > > ----- Original Message ----- > > Dear list, > > > > does someone have recent considerations on xml parsers in terms of > > memory > > footprint, parsing speed and stability? > > > > The ones I'm aware of are xmerl, erlsom [1] and the driver used in > > ejabberd > > (which unfortunately is GPL). > > > > I don't care about DTD validation. > > > > Thank you, > > > > r. > > > > [1] http://erlsom.sourceforge.net/erlsom.htm > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela@REDACTED Mon Mar 26 21:22:22 2012 From: ingela@REDACTED (Ingela Andin) Date: Mon, 26 Mar 2012 21:22:22 +0200 Subject: [erlang-questions] Linux ODBC socket recieve_msg woes In-Reply-To: References: Message-ID: Hi! Please follow the instructions on github on how to make a patch. https://github.com/erlang/otp/wiki/submitting-patches Regards Ingela Erlang/OTP team - Ericsson AB 2012/3/26 Andy Richards : > Hi Ingela, I shall do. Should I send this patch to the erlang-patches > mailing list or submit a patch via GitHub? > > Andy. > > > On Monday, 26 March 2012, Ingela Andin wrote: >> Hi! >> >> 2012/3/23, Andy Richards : >>> Fixed. I found this forum message from back in 2004. >>> >>> http://erlang.org/pipermail/erlang-questions/2004-July/012808.html >>> >>> Editing odbcserver.c and disabling nagel's algorithm (approx 40ms on >>> Redhat >>> 6) on the socket solved the problem. I wonder why this was never added to >>> odbcserver.c in the past? >> >> Humm... ?it was added on the erlang side but apparently not on the c-side. >> I can >> not remember why, if it was for some reason or maybe it was just an >> oversight. >> Please send your modifications as a patch. >> >> [...] >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> From watson.timothy@REDACTED Mon Mar 26 22:05:45 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 26 Mar 2012 21:05:45 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <201203260944.58509.clist@uah.es> Message-ID: Max have you seen http://blogtrader.net/blog/tim_bray_s_erlang_exercise2. This states "0.93 sec on 1 million lines file on my 4-core linux box" which sounds pretty impressive and is based on pure Erlang (with some ets thrown into the mix by the looks of things). Might be worth looking at whether this can potentially out-perform the NIF! On 26 March 2012 12:40, Max Lapshin wrote: > > >> >> And what do these numbers look like? Do they repeat? Are they short? > > Right as in example csv. It is trading data. > > >> Or are they high-precision and varying wildly in order of magnitude, >> and widely distributed statistically? > > > They are very close to each other and vary not more than several percents. > You think ot is a good place for optimization? > > > In fact I have achieved good enough results: less than a second and thank to > all community for it. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From dmkolesnikov@REDACTED Mon Mar 26 22:13:44 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Mon, 26 Mar 2012 23:13:44 +0300 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> Message-ID: Hello Tim, Max, et al, Let's start from beginning... looks like we are mixed a bit. 1. I've made a clean up in the repository. The folder ./priv contains a perl script gen_set.pl to produce a data sets according to Max original format: As an example: key299991,20120326,21:06:31.543,24.16,92.39,22.68,1.71,43.50,53.29,90.53,10.05,91.01,80.66,23.09,18.55,41.38,98.90,61.31,40.44,14.26,42.23,78.22,54.78,18.86,11.72,97.45,47.39,61.50,39.98,73.25,51.65,9.84,42.33,18.84,23.52,60.11,94.82,55.87,86.04,25.12,20.40,32.69,4.51,zz 2. For historical reasons I run tests against three data sets, they differs each other by the number of fields (float numbers). They are * 300K lines, each line contains 8 fields, ~23MB * 300K lines, each line contains 24 fields, ~50MB * 300K lines, each line contains 40 fields, ~77MB <- Max's original file you can generate those data set's via perl priv/gen_set.pl 300 8 > priv/set-300K-8.txt perl priv/gen_set.pl 300 24 > priv/set-300K-24.txt perl priv/gen_set.pl 300 40 > priv/set-300K-40.txt or via make example if you are fun of GNU build like I am 2. I have made number of tests / examples to validate a performance. Essentially we are talking about ETL processes here: * Extract data from csv file * Transform csv lines into some other format * Load data to some storage for processing The following test cases were implemented: * extract data from CSV, this operation just parses a file and does nothing with data. The objective here is to validate a speed of the parser as such * extract/transform, it parses a CSV file and calculates a rolling hash agains data * extract/transform, it parses a CSV file and converts list into tuple. The tuple is not stored anywhere. The objective here is to validate speed of parser + transform operation * extract/transform/load, it parses a CSV file, convers it to tuple and stores to some in-memory storage. You can find those cases at priv/csv_example.erl and run them by csv_example:run(80) 3. The results what I got are following (they are also available at README) E/Parse Size (MB) Read (ms) Handle (ms) Per Line (us) ------------------------------------------------------------------- 300K, 8 flds 23.41 91.722 350.000 1.16 300K, 24 flds 50.42 489.303 697.739 2.33 300K, 40 flds 77.43 780.296 946.003 3.15 ET/hash Size (MB) Read (ms) Handle (ms) Per Line (us) ------------------------------------------------------------------- 300K, 8 flds 23.41 91.722 384.598 1.28 300K, 24 flds 50.42 489.303 761.414 2.54 300K, 40 flds 77.43 780.296 1047.329 3.49 ET/tuple Size (MB) Read (ms) Handle (ms) Per Line (us) ------------------------------------------------------------------- 300K, 8 flds 23.41 91.722 228.306 0.76 300K, 24 flds 50.42 489.303 601.025 2.00 300K, 40 flds 77.43 780.296 984.676 3.28 ETL/ets Size (MB) Read (ms) Handle (ms) Per Line (us) ------------------------------------------------------------------- 300K, 8 flds 23.41 91.722 1489.543 4.50 300K, 24 flds 50.42 489.303 2249.689 7.50 300K, 40 flds 77.43 780.296 2519.401 8.39 ETL/pts Size (MB) Read (ms) Handle (ms) Per Line (us) ------------------------------------------------------------------- 300K, 8 flds 23.41 91.722 592.886 1.98 300K, 24 flds 50.42 489.303 1190.745 3.97 300K, 40 flds 77.43 780.296 1734.898 5.78 The biggest frustration came from ets table. It is was to slow to load data into ets. Now, I believe that your results are slow because you are doing ets load... In my final test, I swap ets with lightweight pts (process term storage), this is a process that holds data an addressable by key. You can see that Max's original file is parsed / transformed and loaded into in-memory process based storage pertty fast, just 5.78 us per line. My original file is even faster 1.98 us per line 4. If you wish to replicate the results on you HW then I propose for you to compile csv library, generate data sets and try to compile csv_example. Please let me know you you have a trouble on each of those phases. and keep in-ming that +native flag were used... Regards, Dmitry On Mar 26, 2012, at 3:16 PM, Tim Watson wrote: > Hi Dmitry, > > On 25 March 2012 11:09, Dmitry Kolesnikov wrote: >> Hello, >> >> 1. Yes, I'd like to admit that I've missed the example file from the beginning of thread. I've put my sets generators into the repository. You can use them like this: >> sh ./priv/dset line-ex.txt 1000 > ./priv/set-1M-ex.txt >> The first parameter is line, second is kilolines to generate. line-16/48 are my original examples, line-ex.txt is from original example. >> > > What is line-ex.txt supposed to contain - is it a single line from the > original example? - I can't find it in the repo. > >> 2. I am a bit of curious why we are getting results of different magnitude, especially with Robert's run. My HW config is pretty close. So, I've build Erlang R15B from sources with following config: >> ./configure --prefix=/usr/local/otp-R15B --enable-threads --enable-smp-support --enable-kernel-poll --enable-sctp --enable-hipe --disable-dynamic-ssl-lib --enable-darwin-64bit --enable-m64-build --without-javac > > I'll rebuild that way, but essentially this is my build config looked > something like this (from config.log): > > configure:4674: running /bin/sh > '/Users/t4/.kerl/builds/r15b-64-hipe-smp/otp_src_R15B/lib/configure' > --prefix=/usr/local '--cache-file=/dev/null' '--enable-hipe' > '--enable-darwin-64bit' '--enable-threads' '--enable-smp' > '--enable-kernel-poll' > 'ERL_TOP=/Users/t4/.kerl/builds/r15b-64-hipe-smp/otp_src_R15B' > --cache-file=/dev/null > --srcdir=/Users/t4/.kerl/builds/r15b-64-hipe-smp/otp_src_R15B/lib > >> >> 3. I've re-run cases again with/without +native flag results are very interesting, so we can parse 300K lines in less then second, less the second for 1M rows is challenging: >> >> Parse time of data set is 300K: >> set-300K-16 653 ms / 2.18 us per line >> set-300K-48 1832 ms / 6.11 us per line >> set-300K-ex 2561 ms / 8.54 us per line >> >> Parse time of data set is 300K +native: >> set-300K-16 277 ms / 0.92 us per line >> set-300K-48 672 ms / 2.24 us per line >> set-300K-ex 925 ms / 3.09 us per line >> > > I can't replicate this even with +native turned on (for both the csv > modules and the csv_example) and I'm bemused as you don't have 'that > much more' horsepower in your 4 cores than my mbpro's 2. Here's what I > get for parsing the original file - as you would expect, there is very > little difference in setting the segments to any number higher than > the # cpus: > > 5> csv_example:import("example.csv", 40). > size (MB): 74.958283 > read (ms): 109.798000 > parse (ms): 3239.264000 > 266257 > 6> csv_example:import("example.csv", 80). > size (MB): 74.958283 > read (ms): 105.949000 > parse (ms): 3231.541000 > 270352 > 7> csv_example:import("example.csv", 4). > size (MB): 74.958283 > read (ms): 102.250000 > parse (ms): 3275.317000 > 274450 > 8> csv_example:parse("example.csv", 4). > lines: 300001 > size (MB): 74.958283 > read (ms): 102.768000 > parse (ms): 2737.098000 > per line (us): 9.123630 > ok > 9> csv_example:parse("example.csv", 44). > lines: 300001 > size (MB): 74.958283 > read (ms): 106.153000 > parse (ms): 2775.781000 > per line (us): 9.252572 > ok > 10> csv_example:parse("example.csv", 2). > lines: 300001 > size (MB): 74.958283 > read (ms): 104.410000 > parse (ms): 2758.367000 > per line (us): 9.194526 > ok > 11> csv_example:import("example.csv", 2). > size (MB): 74.958283 > read (ms): 108.705000 > parse (ms): 3390.453000 > 278547 > > How did you build Max's example file? I'm really struggling to > understand how I've got 2 seconds more processing time for such a > similar setup. > >> Parse time of data set is 1M: >> set-300K-16 4406 ms / 2.20 us per line >> set-300K-48 6076 ms / 6.08 us per line >> set-300K-ex 8670 ms / 8.67 us per line >> >> Parse time of data set is 1M +native: >> set-300K-16 1908 ms / 0.95 us per line >> set-300K-48 2293 ms / 2.29 us per line >> set-300K-ex 3119 ms / 3.12 us per line >> >> 4. It might be unfair to comment but I have a good feeling that Max is evaluating/challenging a type of trade system ;-) Otherwise, why you have such hard latency requirements and dataset looks like a price snapshot ;-) IMHO, 2 - 4 us parse time per row is acceptable for "normal" web app. > > I agree your numbers seem perfectly reasonable, but I'm not able to > reproduce them. I'm going to try on a 2 core linux machine later on > and see how I get on. > >> >> Regards, Dmitry >> >> On Mar 25, 2012, at 5:57 AM, Tim Watson wrote: >> >>> On 25 Mar 2012, at 03:54, Tim Watson wrote: >>>> Please also do run this code on some other (faster) machines to see how much of the slowness is specific to my machine. >>> >>> Oh and can you please pass on the set and set2 text files so I can test them on my machine to baseline the comparative differences between our environments? >>> >>> Cheers! >>> >>> Tim >> From dmkolesnikov@REDACTED Mon Mar 26 22:25:13 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Mon, 26 Mar 2012 23:25:13 +0300 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <201203260944.58509.clist@uah.es> Message-ID: <4E89F17B-C927-4183-89A6-88A770D1C422@gmail.com> Oh my bad.... I've completely forget of theses aspects of VM. I boosted performance so that Max's original file is parsed with 2us per line vs 3.15us, a full ETL cycle (see my previous mail) takes just 7.8 us per line vs 8.39us. and very good hint on Boyer-Moore searching... - dmitry On Mar 26, 2012, at 11:05 PM, Tim Watson wrote: > Max have you seen > http://blogtrader.net/blog/tim_bray_s_erlang_exercise2. This states > "0.93 sec on 1 million lines file on my 4-core linux box" which sounds > pretty impressive and is based on pure Erlang (with some ets thrown > into the mix by the looks of things). Might be worth looking at > whether this can potentially out-perform the NIF! > > On 26 March 2012 12:40, Max Lapshin wrote: >> >> >>> >>> And what do these numbers look like? Do they repeat? Are they short? >> >> Right as in example csv. It is trading data. >> >> >>> Or are they high-precision and varying wildly in order of magnitude, >>> and widely distributed statistically? >> >> >> They are very close to each other and vary not more than several percents. >> You think ot is a good place for optimization? >> >> >> In fact I have achieved good enough results: less than a second and thank to >> all community for it. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Mar 26 22:27:23 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 27 Mar 2012 00:27:23 +0400 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> Message-ID: Dmitry. I tried to compile your library, but it has so many external dependencies, that I've failed. Sorry. From watson.timothy@REDACTED Mon Mar 26 22:47:07 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 26 Mar 2012 21:47:07 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> Message-ID: Max, just ignore the makefile and do this instead: $ echo '{erl_opts, [native]}.' >> rebar.config $ rebar clean compile And then you'll need to compile the csv_example module, which is in priv - I did this lazilly in the shell with `make:all([native]).' before running the test functions. On 26 March 2012 21:27, Max Lapshin wrote: > Dmitry. I tried to compile your library, but it has so many external > dependencies, that I've failed. > Sorry. From watson.timothy@REDACTED Mon Mar 26 22:48:02 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 26 Mar 2012 21:48:02 +0100 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <84475051-E0F3-492D-A47E-91A8402BADD2@gmail.com> <7D73671C-1510-46C0-AB11-375C82FD7AB0@gmail.com> <48CCED27-4D7B-4C8D-A303-3617F2ACD9D8@gmail.com> <30179758-B40F-4F75-9E34-B04630767C5A@gmail.com> <58EC3C39-207E-4C70-A5BC-D1CFEF3EC138@gmail.com> <706ED0BE-EF87-4B72-BC59-A805A368E623@gmail.com> <40DC69BB-C743-46A8-9FEA-9A0BF6CF60CE@gmail.com> Message-ID: Thanks for posting back Dmitry. I'll go through this exercise tomorrow morning and let you know how I get on. For now, I need a little sleep. :) On 26 March 2012 21:13, Dmitry Kolesnikov wrote: > Hello Tim, Max, et al, > > Let's start from beginning... looks like we are mixed a bit. > > 1. I've made a clean up in the repository. The folder ./priv contains a perl script ?gen_set.pl to produce a data sets according to Max original format: > > As an example: > key299991,20120326,21:06:31.543,24.16,92.39,22.68,1.71,43.50,53.29,90.53,10.05,91.01,80.66,23.09,18.55,41.38,98.90,61.31,40.44,14.26,42.23,78.22,54.78,18.86,11.72,97.45,47.39,61.50,39.98,73.25,51.65,9.84,42.33,18.84,23.52,60.11,94.82,55.87,86.04,25.12,20.40,32.69,4.51,zz > > 2. For historical reasons I run tests against three data sets, they differs each other by the number of fields (float numbers). They are > ?* 300K lines, ?each line contains ? 8 fields, ~23MB > ?* 300K lines, ?each line contains 24 fields, ~50MB > ?* 300K lines, ?each line contains 40 fields, ~77MB <- Max's original file > > you can generate those data set's via > perl priv/gen_set.pl 300 ? 8 > priv/set-300K-8.txt > perl priv/gen_set.pl 300 24 > priv/set-300K-24.txt > perl priv/gen_set.pl 300 40 > priv/set-300K-40.txt > or via make example if you are fun of GNU build like I am > > 2. I have made number of tests / examples to validate a performance. Essentially we are talking about ETL processes here: > ?* Extract data from csv file > ?* Transform csv lines into some other format > ?* Load data to some storage for processing > > The following test cases were implemented: > ?* extract data from CSV, this operation just parses a file and does nothing with data. The objective here is to validate a speed of the parser as such > ?* extract/transform, it parses a CSV file and calculates a rolling hash agains data > ?* extract/transform, it parses a CSV file and converts list into tuple. The tuple is not stored anywhere. The objective here is to validate speed of parser + transform operation > ?* extract/transform/load, it parses a CSV file, convers it to tuple and stores to some in-memory storage. > > You can find those cases at priv/csv_example.erl and run them by csv_example:run(80) > > 3. The results what I got are following (they are also available at README) > > ? E/Parse ? ? ? ? Size (MB) ? Read (ms) ? Handle (ms) ? ?Per Line (us) > ? ------------------------------------------------------------------- > ? 300K, ?8 flds ? ? 23.41 ? ? ? 91.722 ? ? 350.000 ? ? ? ? 1.16 > ? 300K, 24 flds ? ? 50.42 ? ? ?489.303 ? ? 697.739 ? ? ? ? 2.33 > ? 300K, 40 flds ? ? 77.43 ? ? ?780.296 ? ? 946.003 ? ? ? ? 3.15 > > > ? ET/hash ? ? ? ? Size (MB) ? Read (ms) ? Handle (ms) ? ?Per Line (us) > ? ------------------------------------------------------------------- > ? 300K, ?8 flds ? ? 23.41 ? ? ? 91.722 ? ? 384.598 ? ? ? ? 1.28 > ? 300K, 24 flds ? ? 50.42 ? ? ?489.303 ? ? 761.414 ? ? ? ? 2.54 > ? 300K, 40 flds ? ? 77.43 ? ? ?780.296 ? ?1047.329 ? ? ? ? 3.49 > > > ? ET/tuple ? ? ? ? Size (MB) ? Read (ms) ? Handle (ms) ? ?Per Line (us) > ? ------------------------------------------------------------------- > ? 300K, ?8 flds ? ? 23.41 ? ? ? 91.722 ? ? 228.306 ? ? ? ? 0.76 > ? 300K, 24 flds ? ? 50.42 ? ? ?489.303 ? ? 601.025 ? ? ? ? 2.00 > ? 300K, 40 flds ? ? 77.43 ? ? ?780.296 ? ? 984.676 ? ? ? ? 3.28 > > ? ETL/ets ? ? ? ? ?Size (MB) ? Read (ms) ? Handle (ms) ? ?Per Line (us) > ? ------------------------------------------------------------------- > ? 300K, ?8 flds ? ? 23.41 ? ? ? 91.722 ? ?1489.543 ? ? ? ? 4.50 > ? 300K, 24 flds ? ? 50.42 ? ? ?489.303 ? ?2249.689 ? ? ? ? 7.50 > ? 300K, 40 flds ? ? 77.43 ? ? ?780.296 ? ?2519.401 ? ? ? ? 8.39 > > ? ETL/pts ? ? ? ? ?Size (MB) ? Read (ms) ? Handle (ms) ? ?Per Line (us) > ? ------------------------------------------------------------------- > ? 300K, ?8 flds ? ? 23.41 ? ? ? 91.722 ? ? 592.886 ? ? ? ? 1.98 > ? 300K, 24 flds ? ? 50.42 ? ? ?489.303 ? ?1190.745 ? ? ? ? 3.97 > ? 300K, 40 flds ? ? 77.43 ? ? ?780.296 ? ?1734.898 ? ? ? ? 5.78 > > > The biggest frustration came from ets table. It is was to slow to load data into ets. Now, I believe that your results are slow because you are doing ets load... In my final test, I swap ets with lightweight pts (process term storage), this is a process that holds data an addressable by key. You can see that Max's original file is parsed / transformed and loaded into in-memory process based storage pertty fast, just 5.78 us per line. My original file is even faster 1.98 us per line > > 4. If you wish to replicate the results on you HW then I propose for you to compile csv library, generate data sets and try to compile csv_example. Please let me know you you have a trouble on each of those phases. and keep in-ming that +native flag were used... > > > Regards, Dmitry > > > > On Mar 26, 2012, at 3:16 PM, Tim Watson wrote: > >> Hi Dmitry, >> >> On 25 March 2012 11:09, Dmitry Kolesnikov wrote: >>> Hello, >>> >>> 1. Yes, I'd like to admit that I've missed the example file from the beginning of thread. I've put my sets generators into the repository. You can use them like this: >>> sh ./priv/dset line-ex.txt 1000 > ./priv/set-1M-ex.txt >>> The first parameter is line, second is kilolines to generate. line-16/48 are my original examples, line-ex.txt is from original example. >>> >> >> What is line-ex.txt supposed to contain - is it a single line from the >> original example? - I can't find it in the repo. >> >>> 2. I am a bit of curious why we are getting results of different magnitude, especially with Robert's run. My HW config is pretty close. So, I've build Erlang R15B from sources with following config: >>> ./configure --prefix=/usr/local/otp-R15B --enable-threads --enable-smp-support --enable-kernel-poll --enable-sctp --enable-hipe --disable-dynamic-ssl-lib --enable-darwin-64bit --enable-m64-build --without-javac >> >> I'll rebuild that way, but essentially this is my build config looked >> something like this (from config.log): >> >> configure:4674: running /bin/sh >> '/Users/t4/.kerl/builds/r15b-64-hipe-smp/otp_src_R15B/lib/configure' >> --prefix=/usr/local ?'--cache-file=/dev/null' '--enable-hipe' >> '--enable-darwin-64bit' '--enable-threads' '--enable-smp' >> '--enable-kernel-poll' >> 'ERL_TOP=/Users/t4/.kerl/builds/r15b-64-hipe-smp/otp_src_R15B' >> --cache-file=/dev/null >> --srcdir=/Users/t4/.kerl/builds/r15b-64-hipe-smp/otp_src_R15B/lib >> >>> >>> 3. I've re-run cases again with/without +native flag results are very interesting, so we can parse 300K lines in less then second, less the second for 1M rows is challenging: >>> >>> Parse time of data set is 300K: >>> set-300K-16 ? ? 653 ms / 2.18 us per line >>> set-300K-48 ? 1832 ms / 6.11 us per line >>> set-300K-ex ? 2561 ms / 8.54 us per line >>> >>> Parse time of data set is 300K +native: >>> set-300K-16 ? ? 277 ms / 0.92 us per line >>> set-300K-48 ? ? 672 ms / 2.24 us per line >>> set-300K-ex ? ? 925 ms / 3.09 us per line >>> >> >> I can't replicate this even with +native turned on (for both the csv >> modules and the csv_example) and I'm bemused as you don't have 'that >> much more' horsepower in your 4 cores than my mbpro's 2. Here's what I >> get for parsing the original file - as you would expect, there is very >> little difference in setting the segments to any number higher than >> the # cpus: >> >> 5> csv_example:import("example.csv", 40). >> size (MB): 74.958283 >> read (ms): 109.798000 >> parse (ms): 3239.264000 >> 266257 >> 6> csv_example:import("example.csv", 80). >> size (MB): 74.958283 >> read (ms): 105.949000 >> parse (ms): 3231.541000 >> 270352 >> 7> csv_example:import("example.csv", 4). >> size (MB): 74.958283 >> read (ms): 102.250000 >> parse (ms): 3275.317000 >> 274450 >> 8> csv_example:parse("example.csv", 4). >> lines: 300001 >> size (MB): 74.958283 >> read (ms): 102.768000 >> parse (ms): 2737.098000 >> per line (us): 9.123630 >> ok >> 9> csv_example:parse("example.csv", 44). >> lines: 300001 >> size (MB): 74.958283 >> read (ms): 106.153000 >> parse (ms): 2775.781000 >> per line (us): 9.252572 >> ok >> 10> csv_example:parse("example.csv", 2). >> lines: 300001 >> size (MB): 74.958283 >> read (ms): 104.410000 >> parse (ms): 2758.367000 >> per line (us): 9.194526 >> ok >> 11> csv_example:import("example.csv", 2). >> size (MB): 74.958283 >> read (ms): 108.705000 >> parse (ms): 3390.453000 >> 278547 >> >> How did you build Max's example file? I'm really struggling to >> understand how I've got 2 seconds more processing time for such a >> similar setup. >> >>> Parse time of data set is 1M: >>> set-300K-16 ? 4406 ms / 2.20 us per line >>> set-300K-48 ? 6076 ms / 6.08 us per line >>> set-300K-ex ? 8670 ms / 8.67 us per line >>> >>> Parse time of data set is 1M +native: >>> set-300K-16 ? ?1908 ms / 0.95 us per line >>> set-300K-48 ? ?2293 ms / 2.29 us per line >>> set-300K-ex ? ?3119 ms / 3.12 us per line >>> >>> 4. It might be unfair to comment but I have a good feeling that Max is evaluating/challenging a type of trade system ;-) Otherwise, why you have such hard latency requirements and dataset looks like a price snapshot ;-) IMHO, 2 - 4 us parse time per row is acceptable for "normal" web app. >> >> I agree your numbers seem perfectly reasonable, but I'm not able to >> reproduce them. I'm going to try on a 2 core linux machine later on >> and see how I get on. >> >>> >>> Regards, Dmitry >>> >>> On Mar 25, 2012, at 5:57 AM, Tim Watson wrote: >>> >>>> On 25 Mar 2012, at 03:54, Tim Watson wrote: >>>>> Please also do run this code on some other (faster) machines to see how much of the slowness is specific to my machine. >>>> >>>> Oh and can you please pass on the set and set2 text files so I can test them on my machine to baseline the comparative differences between our environments? >>>> >>>> Cheers! >>>> >>>> Tim >>> > From francesco@REDACTED Mon Mar 26 22:55:24 2012 From: francesco@REDACTED (Francesco Cesarini) Date: Mon, 26 Mar 2012 21:55:24 +0100 Subject: [erlang-questions] Erlang Jobs? In-Reply-To: <4F6B8A60.7050006@mansionfamily.plus.com> References: <4F6B8A60.7050006@mansionfamily.plus.com> Message-ID: <4F70D7BC.1040306@erlang-solutions.com> We post our Erlang Solutions internal jobs and open posts of some of our clients on our jobs page: http://www.erlang-solutions.com/jobs/listing Those who are registered receive an email every time a new job is posted. We are in the process of upgrading the site and will have a new API. Regards, Francesco On 22/03/2012 20:24, james wrote: > Are there any? > > I guess to be fair (since there aren't many of any sort around right > now) - are there normally any (in London)? > > Weird that there seem to be none on TotalJobs. Even for small > software houses trying it on. > > James > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Erlang Solutions Ltd. http://www.erlang-solutions.com From tuncer.ayaz@REDACTED Mon Mar 26 23:41:36 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 26 Mar 2012 23:41:36 +0200 Subject: [erlang-questions] [ANN] reltool app-level lib_dir setting Message-ID: Thanks to Siri, reltool in R15B01 includes a really useful feature. Many Erlang projects are using the following structure: ./src/foo.app.src ./src/foo.erl ./rel/reltool.config When you wanted to reference the application foo in rel/reltool.config, you previously had three options: * project-local libs/ or apps/ directory * unsafe {lib_dirs, "../.."} reltool.config setting * rebar_reltool_link fake_lib_dir plugin Starting with R15B01 all you have to do now is specify {app, foo, [{incl_cond, include}, {lib_dir, ".."}]} in reltool.config for applications not in lib_dirs. Although R15B01 is still in development, this feature is complete and can be used in the otp.git maint branch. From toby@REDACTED Tue Mar 27 01:36:10 2012 From: toby@REDACTED (Toby Thain) Date: Mon, 26 Mar 2012 19:36:10 -0400 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <201203260944.58509.clist@uah.es> Message-ID: <4F70FD6A.9070001@telegraphics.com.au> On 26/03/12 6:36 AM, Michael Turner wrote: >> I really can't understand why should parsing be slower than reading from HDD =) > > Are you converting the ASCII-coded floating point numbers to actual > floating point? That's actually quite a lot more overhead per > character than ... well, anything else I can think of in processing a > CSV file. > > And what do these numbers look like? Do they repeat? Are they short? > Or are they high-precision and varying wildly in order of magnitude, > and widely distributed statistically? I see where you're heading ... by the look of those 4-dig-digit numbers the FP conversion could be done by a lookup in a 10,000 element array - assuming this is cheaper than the straightforward conversion. --Toby > > -michael turner > > > > On Mon, Mar 26, 2012 at 5:37 PM, Max Lapshin wrote: >> On Mon, Mar 26, 2012 at 12:33 PM, Robert Melton wrote: >>> >>> Agreed. Do we have any baseline implementation in pure C or (insert >>> fastest language/implementation you are aware of)? I am working on >>> speeding this up (and having a lot of fun!), but I have no idea the >>> theory-craft maximum process speed (with proper escaping, etc) on my >>> hardware. >>> >> >> I really can't understand why should parsing be slower than reading from HDD =) >> >> However, it is slower. Currently I have 950 ms for 300K line CSV with >> 40 float columns when read on cold system and 820 ms when read from >> disk cache. >> >> Copying from kernel cache and byte-by-byte reading all data while >> searching '\n' takes 100 ms (it is time of wc -l), so it takes about >> 700 ms for erlang to parse + create all proper objects. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mjtruog@REDACTED Tue Mar 27 08:39:19 2012 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 26 Mar 2012 23:39:19 -0700 Subject: [erlang-questions] [ANN] CloudI 0.2.0 Released Message-ID: <4F716097.9010607@gmail.com> Download 0.2.0rc1 from http://sourceforge.net/projects/cloudi/files/latest/download CloudI (http://cloudi.org) is now beta and version 0.2.0 will be used for testing. After sufficient testing, the release will become 1.0.0 to begin semantic versioning (http://semver.org). The FAQ will soon be updated to reflect the changes for 0.2.0. The highlights for this new release are: * support for Erlang R15 * added support for using service name patterns (i.e., names that include the "*" wildcard character) * added priority queueing to Erlang services * added timeout handling to expire queued requests and now update the request timeout based on the time spent queued * added log output redirection with automatic partition tolerance (i.e., redirect local log output to a remote node and if contact with the remote node is lost, log locally until the connection is reestablished) * added floating point usage for configuring process and thread counts (so that it represents a ceil for numbers > 1.0 or a round for numbers < 1.0 when using it as a multiple of the number of cores (i.e., Erlang VM scheduler threads)) * added service options to the CloudI configuration so that typical defaults are easily modified Anyone with questions, comments, ideas, suggestions, criticisms, or concerns is welcome to send email. Thanks, Michael From dmkolesnikov@REDACTED Tue Mar 27 08:58:56 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Tue, 27 Mar 2012 09:58:56 +0300 Subject: [erlang-questions] Record metadata Message-ID: <-9083630128156938984@unknownmsgid> Hello, Let say I do have a record definition: -record(my, { a :: atom(), b :: binary() }). Are there any possibility to extract a metadata of record elements during run-time? E.g call_some_magic_fun that says R#my.a is atom, R#my.b is binary... Thanks for any advice! Best Regards, Dmitry >-|-|-*> From zabrane3@REDACTED Tue Mar 27 09:27:45 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 27 Mar 2012 09:27:45 +0200 Subject: [erlang-questions] [ANN] CloudI 0.2.0 Released In-Reply-To: <4F716097.9010607@gmail.com> References: <4F716097.9010607@gmail.com> Message-ID: <65BB1835-4BA3-42E8-8F56-3FA72378F5C2@gmail.com> Congratulations Michael Regards, Zabrane On Mar 27, 2012, at 8:39 AM, Michael Truog wrote: > Download 0.2.0rc1 from http://sourceforge.net/projects/cloudi/files/latest/download > > CloudI (http://cloudi.org) is now beta and version 0.2.0 will be used for testing. After sufficient testing, the release will become 1.0.0 to begin semantic versioning (http://semver.org). The FAQ will soon be updated to reflect the changes for 0.2.0. The highlights for this new release are: > * support for Erlang R15 > * added support for using service name patterns > (i.e., names that include the "*" wildcard character) > * added priority queueing to Erlang services > * added timeout handling to expire queued requests and now update the request > timeout based on the time spent queued > * added log output redirection with automatic partition tolerance > (i.e., redirect local log output to a remote node and if contact with the > remote node is lost, log locally until the connection is reestablished) > * added floating point usage for configuring process and thread counts > (so that it represents a ceil for numbers > 1.0 or > a round for numbers < 1.0 when using it as a multiple of the > number of cores (i.e., Erlang VM scheduler threads)) > * added service options to the CloudI configuration so that typical > defaults are easily modified > > Anyone with questions, comments, ideas, suggestions, criticisms, or concerns is welcome to send email. > > Thanks, > Michael > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From aleksandr.vin@REDACTED Tue Mar 27 11:01:38 2012 From: aleksandr.vin@REDACTED (Aleksandr Vinokurov) Date: Tue, 27 Mar 2012 13:01:38 +0400 Subject: [erlang-questions] Sources for the erlc Message-ID: Hello all, Can smb. point me to the sources of erlc? I've failed to find it for 15 mins and suspect that it's not worth of it -- smb. must know the answer. Sincerely yours, Aleksandr Vinokurov -- ????????? ????????? +7 (921) 982-21-43 @aleksandrvin -------------- next part -------------- An HTML attachment was scrubbed... URL: From onlyafly@REDACTED Tue Mar 27 11:05:08 2012 From: onlyafly@REDACTED (Kevin Albrecht) Date: Tue, 27 Mar 2012 11:05:08 +0200 Subject: [erlang-questions] Style enforcer for Erlang? Message-ID: Is there any tool that checks the coding style of Erlang programs? If not, does anyone have good ideas for how to accomplish it using some combination of other tools? Regards, Kevin Albrecht From ponton@REDACTED Tue Mar 27 11:08:19 2012 From: ponton@REDACTED (Tomasz Maciejewski) Date: Tue, 27 Mar 2012 11:08:19 +0200 Subject: [erlang-questions] Sources for the erlc In-Reply-To: References: Message-ID: https://github.com/erlang/otp/blob/master/erts/etc/common/erlc.c -- Tomasz Maciejewski W dniu 27 marca 2012 11:01 u?ytkownik Aleksandr Vinokurov napisa?: > > > Hello all, > > Can smb. point me to the sources of erlc? I've failed to find it for 15 mins > and suspect that it's not worth of it -- smb. must know the answer. > > Sincerely yours, > Aleksandr Vinokurov > > -- > ????????? ????????? > +7 (921) 982-21-43 > @aleksandrvin > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From bengt.kleberg@REDACTED Tue Mar 27 11:18:48 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 27 Mar 2012 11:18:48 +0200 Subject: [erlang-questions] Style enforcer for Erlang? In-Reply-To: References: Message-ID: <1332839928.5255.40.camel@seasc1137> Greetings, Is this something like what you want: http://www.erlang-factory.com/upload/presentations/121/KostisSagonas-ErlangFactoryLondon2009-CleaningupErlangcodeisadirtyjobbutsomebodysgottadoit.pdf bengt On Tue, 2012-03-27 at 11:05 +0200, Kevin Albrecht wrote: > Is there any tool that checks the coding style of Erlang programs? If > not, does anyone have good ideas for how to accomplish it using some > combination of other tools? > > Regards, > Kevin Albrecht > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From kaven.wong@REDACTED Tue Mar 27 11:18:54 2012 From: kaven.wong@REDACTED (Kevin Wang) Date: Tue, 27 Mar 2012 17:18:54 +0800 Subject: [erlang-questions] Style enforcer for Erlang? In-Reply-To: References: Message-ID: I just write code by using Emacs, which keeps the style of code with erlang mode. --Kevin On Tue, Mar 27, 2012 at 5:05 PM, Kevin Albrecht wrote: > Is there any tool that checks the coding style of Erlang programs? If > not, does anyone have good ideas for how to accomplish it using some > combination of other tools? > > Regards, > Kevin Albrecht > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From onlyafly@REDACTED Tue Mar 27 11:39:31 2012 From: onlyafly@REDACTED (Kevin Albrecht) Date: Tue, 27 Mar 2012 11:39:31 +0200 Subject: [erlang-questions] Style enforcer for Erlang? In-Reply-To: <1332839928.5255.40.camel@seasc1137> References: <1332839928.5255.40.camel@seasc1137> Message-ID: > Is this something like what you want: > http://www.erlang-factory.com/upload/presentations/121/KostisSagonas-ErlangFactoryLondon2009-CleaningupErlangcodeisadirtyjobbutsomebodysgottadoit.pdf Tidier is an interesting tool, but I am looking for something which does more surface-level checking (indentation, spacing, list and tuple layout, etc.) > I just write code by using Emacs, which keeps the style of code with erlang > mode. This works fine for yourself, but I want to validate the code of many developers. --Kevin Albrecht From max.lapshin@REDACTED Tue Mar 27 11:44:46 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 27 Mar 2012 13:44:46 +0400 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: <4F70FD6A.9070001@telegraphics.com.au> References: <201203260944.58509.clist@uah.es> <4F70FD6A.9070001@telegraphics.com.au> Message-ID: I'm experimenting more and found very strange thing sscanf((char *)start, "%4d%02d%02d", &year, &month, &day); this adds 3 microseconds per line. Why?? I've changed it to macros: #define D2(s) (((s)[0] - '0')*10 + ((s)[1] - '0')) and this macros works for several nanoseconds. Is sscanf really so slow?? And also I'll really have to cache somehow timestamps, because timegm takes more than 200 microseconds per line. It is impossible to call it on each line. From zabrane3@REDACTED Tue Mar 27 11:50:25 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 27 Mar 2012 11:50:25 +0200 Subject: [erlang-questions] Style enforcer for Erlang? In-Reply-To: <1332839928.5255.40.camel@seasc1137> References: <1332839928.5255.40.camel@seasc1137> Message-ID: Hi guys, Hi Kostis, I'm wondering if you keep the erlang source files (on your server) when using tidier's web interface? Can we be sure that you don't keep them? Regards, Zabrane On Mar 27, 2012, at 11:18 AM, Bengt Kleberg wrote: > http://www.erlang-factory.com/upload/presentations/121/KostisSagonas-ErlangFactoryLondon2009-CleaningupErlangcodeisadirtyjobbutsomebodysgottadoit.pdf -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Mar 27 12:08:58 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 27 Mar 2012 14:08:58 +0400 Subject: [erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second In-Reply-To: References: <201203260944.58509.clist@uah.es> <4F70FD6A.9070001@telegraphics.com.au> Message-ID: Yes. I have to use hand-made macros to create fixed-width string to integer and static caching for avoiding useless calls to timegm From kostis@REDACTED Tue Mar 27 12:11:45 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 27 Mar 2012 12:11:45 +0200 Subject: [erlang-questions] Style enforcer for Erlang? In-Reply-To: References: <1332839928.5255.40.camel@seasc1137> Message-ID: <4F719261.2010709@cs.ntua.gr> On 03/27/2012 11:50 AM, Zabrane Mickael wrote: > Hi guys, Hi Kostis, > > I'm wondering if you keep the erlang source files (on your server) when > using tidier's web interface? > Can we be sure that you don't keep them? Have you read the "important notice and disclaimer" which is right at the top of tidier's webpage?? http://tidier.softlab.ntua.gr I think it's pretty explicit about what's happening. Kostis From vances@REDACTED Tue Mar 27 12:14:37 2012 From: vances@REDACTED (Vance Shipley) Date: Tue, 27 Mar 2012 15:44:37 +0530 Subject: [erlang-questions] Style enforcer for Erlang? In-Reply-To: References: <1332839928.5255.40.camel@seasc1137> Message-ID: <20120327101436.GB356@aluminum.wavenet.lk> On Tue, Mar 27, 2012 at 11:39:31AM +0200, Kevin Albrecht wrote: } Tidier is an interesting tool, but I am looking for something which } does more surface-level checking (indentation, spacing, list and tuple } layout, etc.) Look at the syntax_tools application: http://www.erlang.org/doc/apps/syntax_tools/index.html Specifically erl_tidy:file/1: http://www.erlang.org/doc/man/erl_tidy.html#file-1 -- -Vance From zerthurd@REDACTED Tue Mar 27 12:33:09 2012 From: zerthurd@REDACTED (Maxim Treskin) Date: Tue, 27 Mar 2012 17:33:09 +0700 Subject: [erlang-questions] The largest number of CPUs utilized by one node Message-ID: Hello It seems, Erlang/OTP has better support for SMP now, among other languages/platforms. Does anyone know how much the largest number of processors or cores used by one Erlang node in production? Thank you -- Maxim Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Tue Mar 27 12:49:56 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 27 Mar 2012 11:49:56 +0100 Subject: [erlang-questions] Record metadata In-Reply-To: <-9083630128156938984@unknownmsgid> References: <-9083630128156938984@unknownmsgid> Message-ID: You'd need the abstract code available and then you can call dialyzer_utils:get_record_and_type_info/1 which returns a dict containing the type information. You can then pass this dict to various functions in the erl_types module to get back information about type specs. If you don't want to rely on dialyzer at runtime, you'll need to copy-paste the code from dialyzer_utils AFAICT. Also the erl_types module is part of hipe. On 27 March 2012 07:58, dmitry kolesnikov wrote: > Hello, > > Let say I do have a record definition: > -record(my, { > ?a :: atom(), > ?b :: binary() > }). > > Are there any possibility to extract a metadata of record elements > during run-time? E.g call_some_magic_fun that says R#my.a is atom, > R#my.b is binary... > > Thanks for any advice! > > Best Regards, > Dmitry >-|-|-*> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lukasp.p017@REDACTED Tue Mar 27 13:51:20 2012 From: lukasp.p017@REDACTED (Lukas P) Date: Tue, 27 Mar 2012 13:51:20 +0200 Subject: [erlang-questions] ssl timing issue in R15B? In-Reply-To: References: Message-ID: Many thanks for your help, Ingela! It works now reliably. Best regards, Lukas Dne 26. b?ezna 2012 15:16 Ingela Andin napsal(a): > Hi! > > The problem is that your server and client needs to agree to upgrade > to SSL/TLS and your client may not start the handshake until the > server is ready to receive it. If you have a erlang-server this will > be when you have set your tcp-socket in passive mode. > When your server has set its socket in passive mode, you can ack to > the client that it is ready to start and then call ssl:ssl_accept with > the socket. > Otherwhise a quick clients ?handshake may arrive to the inet driver > and be processed with socket options that the ssl-application has no > controll over. This may, depending on timing, result in that the > ssl-connection process recives data on an unexpected format. It could > also lead to that the some other process receives the handshake data > and if you are lucky it may work as expected ;) > > > Regards Ingela Erlang/OTP team - Ericsson AB > > > 2012/3/26 Lukas P : >> Dne 23. b?ezna 2012 18:16 Attila Rajmund Nohl >> napsal(a): >>> 2012/3/23 Lukas P : >>>> Hello. >>>> >>>> I think that I have been hit by a SSL timing issue in R15B. My >>>> functional test (= SSL/TCP client) crahes my server unless the test >>>> contains a sleep. >>>> >>>> The tested server implements a TCP based, line oriented protocol. >>>> Switch to SSL can be started with a "STARTTLS\r\n" command. >>>> >>>> The functional test starts a TCP connection and upgrades it to SSL: >>>> >>>> upgrade_connection_to_ssl(Socket) -> >>>> ? ?SendResult = gen_tcp:send(Socket, "STARTTLS\r\n"), >>>> ? ?%timer:sleep(10), % workaround for a SSL timing bug >>>> ? ?SslConnectResult = case SendResult of >>>> ? ? ? ?ok -> >>>> ? ? ? ? ? ?ssl:connect(Socket, [{active, false}, {packet, line}, list], 2000); >>>> ? ? ? ?{error, Reason1} -> >>>> ? ? ? ? ? ?{error, Reason1} >>>> ? ?end, >>>> ? ?... >>>> >>>> After the server receives "STARTTLS\r\n", it performs ssl:ssl_accept >>>> on the socket: >>>> >>>> ? ?inet:setopts(Socket, [{active, false}]), >>> >>> Don't you need a {reuseaddr, true} option here? The default is false. >> >> Unfortunately not, the tcp socket is already connected. >> >> When you look at the walkback, you can see that there is definitely a >> bug in ssl_record:get_tls_records_aux/2 (erlang:size/1 must not be >> used on a list). >> >> Lukas >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From mark.fine@REDACTED Mon Mar 26 23:50:20 2012 From: mark.fine@REDACTED (Mark Fine) Date: Mon, 26 Mar 2012 14:50:20 -0700 Subject: [erlang-questions] say no to aws dynamodb boilerplate and welcome dsl! In-Reply-To: References: Message-ID: Ask AWS to ship the models they generate their SDKs from. Mark On Mon, Mar 26, 2012 at 5:42 AM, Joel Reymont wrote: > Here's a DSL I whipped up to simplify working with Amazon DynamoDB by > automatically generating Erlang code. With a bit of thinking it could > be extended to SimpleDB, Python, etc. > > I don't have a name for this language so I'm calling it DSL for > "Declarative Schema Language" or maybe "Dynamo Schema Language". Feel > free to propose a better name. > > Here's an example schema. More comments further down. > > --- > record Publisher > ?id ? ? ? ?string (auto primary key) # auto-generated hash key > ?stacks ? ?[Stack.id] # set of stack ids, starts empty > ?sub-count count (Subscription.stack.publisher.id) > end > > record Stack > ?id ? ? ? ? ? ?string (auto primary key) > ?publisher ? ? Publisher.id > ?subscriptions [Subscription.id] > end > > #enum Frequency > # ?disabled > # ?immediate > # ?daily > # ?weekly > # ?monthly > #end > > record Subscription > ?id ? ? ? ? ? ?string (auto primary key) > ?user ? ? ? ? ?User.id > ?stack ? ? ? ? Stack.id > # notify-freq ? Frequency > ?notify-freq ? string > ?language ? ? ?string > ?last-notified date > end > > record NewDocs > ?stack ? ? Stack.id (primary key) > ?add-date ?date (secondary key) > ?doc-id ? ?string > end > > record User > ?id ? ? ? ? ? ?string (primary key) # Issuu id? > ?subscriptions [Subscription.id] > end > > record UnconfirmedEmails > ?id ? ?string (auto primary key) > ?email string > end > --- > > This generates Erlang code using http://github.com/wagerlabs/ddb that > looks like this > > --- > -module(publisher). > > -export([setup/0, teardown/0, > ? ? ? ?put/1, get/1, delete/1]). > > setup() -> > ? ddb:create_table(<<"Publisher">>, > ? ? ? ? ? ? ? ? ? ddb:key_type(<<"id">>, 'string'), > ? ? ? ? ? ? ? ? ? 10, 10). > > teardown() -> > ? ddb:remove_table(<<"Publisher">>). > --- > > and this > > --- > -module(newdocs). > > -export([setup/0, teardown/0, > ? ? ? ?put/3, get/3, delete/3]). > > setup() -> > ? ddb:create_table(<<"NewDocs">>, > ? ? ? ? ? ? ? ? ? ddb:key_type(<<"stack">>, 'string', > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?<<"add_date">>, 'string'), > ? ? ? ? ? ? ? ? ? 10, 10). > > teardown() -> > ? ddb:remove_table(<<"NewDocs">>). > --- > > A bunch of code is missing but you should be able to auto-magically > update Publisher.stacks with Stack.id when a new stack is created > where Stack.publisher matches Publisher.id. > > It's gets even better and you should be able to bump > Publisher.sub-count (subscription count) when a new subscription is > added to a publisher's stack! > > Of course you should also be able to have put and get functions to > create, read and update the items, as well as functions to manipulate > the sets, e.g. Publisher.stacks, Stack.subscriptions and > User.subscriptions. > > What do you think? > > Do you think this can be accomplished using the SQL syntax? How would > you update the Publisher.sub-count counter then? > > ? Thanks, Joel > > P.S. I wrote the compiler in OCaml and it has a proper lexer, parser, > AST and code generator for Erlang. > > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michal.ptaszek@REDACTED Tue Mar 27 15:25:22 2012 From: michal.ptaszek@REDACTED (=?utf-8?Q?Micha=C5=82?= Ptaszek) Date: Tue, 27 Mar 2012 14:25:22 +0100 (BST) Subject: [erlang-questions] current xml parsers In-Reply-To: Message-ID: <2956113b-1784-4c2c-bba1-f46a98437faf@knuth> There was a bug in exml that has been fixed a couple of weeks ago (buffer overflow in the NIF function), I tried it on WURFL today and the results are as follows: {ok, XML} = file:read_file("/tmp/wurfl.xml"). timer:tc(exml, parse, [XML]). {3796645, {ok,{xmlelement,<<"wurfl">>,[], [{xmlcdata,<<"\n ">>}, {xmlelement,<<"version">>,[], [{xmlcdata,<<"\n ">>}, {xmlelement,<<"ver">>,[], [{xmlcdata,<<"2.3, db.scientiamobile.c"...>>}]}, {xmlcdata,<<"\n ">>}, {xmlelement,<<"last_updated">>,[], [{xmlcdata,<<"Thu Nov 17 18:01"...>>}]}, {xmlcdata,<<"\n ">>}, {xmlelement,<<"official_url">>,[], [{xmlcdata,<<"http://w"...>>}]}, {xmlcdata,<<"\n\t ">>}, {xmlelement,<<"maintainers">>,[], [{xmlcdata,<<...>>},{xmlelement,...},{...}|...]}, {xmlcdata,<<"\n\t ">>}, ... Best regards, Michal Ptaszek ----- Original Message ----- > I think I tried that lib once, because I had the requirement to parse > WURFL > http://wurfl.sourceforge.net/ > It is a giant XML that has info about mobile devices. It worked fine > with > normal sized XMLs, but WURFL crashed it (and erlang along with it). > > > Sergej > > On Mon, Mar 26, 2012 at 3:34 PM, Micha? Ptaszek < > michal.ptaszek@REDACTED> wrote: > > > Hi Roberto, > > > > you might want to be interested in looking at exml: > > it's a very simple NIF-based parser built around > > expat library: > > https://github.com/paulgray/exml > > > > Basing on some of the simple benchmarks of my own it > > should be 2-3 times faster than xmerl. > > > > Best regards, > > Michal Ptaszek > > > > ----- Original Message ----- > > > Dear list, > > > > > > does someone have recent considerations on xml parsers in terms > > > of > > > memory > > > footprint, parsing speed and stability? > > > > > > The ones I'm aware of are xmerl, erlsom [1] and the driver used > > > in > > > ejabberd > > > (which unfortunately is GPL). > > > > > > I don't care about DTD validation. > > > > > > Thank you, > > > > > > r. > > > > > > [1] http://erlsom.sourceforge.net/erlsom.htm > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > From fhunleth@REDACTED Tue Mar 27 15:37:13 2012 From: fhunleth@REDACTED (Frank Hunleth) Date: Tue, 27 Mar 2012 09:37:13 -0400 Subject: [erlang-questions] erl using 99% CPU while idle Message-ID: I was wondering if anyone had seen erl, specifically the erts-5.9/bin/beam process, run at 99% CPU while at the shell prompt. All that I do is run erl without any command line arguments and let it sit. This is a cross-compiled Erlang R15B running on an ARM processor, so it could be the case that I messed up an option somewhere along the line. FWIW, the processing must be trivial since the system feels completely responsive and Erlang applications appear to run fine. I ran strace to see what was going on and the following lines repeat: poll([{fd=3, events=POLLIN|POLLRDNORM}, {fd=0, events=POLLIN|POLLRDNORM}], 2, 0) = 0 (Timeout) clock_gettime(CLOCK_MONOTONIC, {799, 545800597}) = 0 clock_gettime(CLOCK_MONOTONIC, {799, 546350139}) = 0 I haven't gotten far enough through the source to figure out why poll would be repeatedly called with a 0 timeout. Does anyone have any clues? Thanks, Frank From ponton@REDACTED Tue Mar 27 15:45:35 2012 From: ponton@REDACTED (Tomasz Maciejewski) Date: Tue, 27 Mar 2012 15:45:35 +0200 Subject: [erlang-questions] erl using 99% CPU while idle In-Reply-To: References: Message-ID: W dniu 27 marca 2012 15:37 u?ytkownik Frank Hunleth napisa?: > I was wondering if anyone had seen erl, specifically the > erts-5.9/bin/beam process, run at 99% CPU while at the shell prompt. > All that I do is run erl without any command line arguments and let it > sit. This is a cross-compiled Erlang R15B running on an ARM processor, > so it could be the case that I messed up an option somewhere along the > line. FWIW, the processing must be trivial since the system feels > completely responsive and Erlang applications appear to run fine. I > ran strace to see what was going on and the following lines repeat: > > poll([{fd=3, events=POLLIN|POLLRDNORM}, {fd=0, > events=POLLIN|POLLRDNORM}], 2, 0) = 0 (Timeout) > clock_gettime(CLOCK_MONOTONIC, {799, 545800597}) = 0 > clock_gettime(CLOCK_MONOTONIC, {799, 546350139}) = 0 > > I haven't gotten far enough through the source to figure out why poll > would be repeatedly called with a 0 timeout. Does anyone have any > clues? > > Thanks, > Frank I don't know if it's even related to the problem, but have tried using kernel polling (erl +K true)? -- Tomasz Maciejewski From clist@REDACTED Tue Mar 27 15:57:17 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Tue, 27 Mar 2012 15:57:17 +0200 Subject: [erlang-questions] escript's beam in in distributed environment In-Reply-To: References: Message-ID: <201203271557.17348.clist@uah.es> Sorry for so long delay, im reading older unread messages in my box... You can try this to distribute code onto remotre nodes.(assume distribute_app(Modules) -> %% extraer todos los nodos excepto el Master RemoteNodes = [X || X <- pool:get_nodes(), X =/= node()], %% Transferir el c?digo lists:foreach(fun(Node) -> transfer_code(Node, Modules) end, RemoteNodes). transfer_code(Node, Modules) -> [transfer_module(Node, Module) || Module <- Modules]. transfer_module(Node, Module) -> {_Module, Binary, FileName} = code:get_object_code(Module), rpc:call(Node, code, load_binary, [Module, FileName, Binary]). Or you can user "--inet loader" slave:start("192.168.X,Y","remote_node_test", ["-host 192.168.my.IP -loader inet "++"-setcookie "++erlang:get_cookie()] ). and also youll need to pass same options when use pool:start(Name,Args) for getting a bunch of nodes. Note that you need additional setup on your code server to aloow reuqest from those new remote nodes, just check erl -man erl_boot_server Whenever some remote node need to load a new beam it asks your local boot_server server instead of going to the FS so avoids to distribute, or share your beams via NFS and also without the need to extract them form the script... Ive tested this (no production yet) and seems to work... (well there are some trouble in "slave" when you spawn a lot of nodes but this is another story..) regards, Angel On Martes, 7 de Febrero de 2012 11:45:53 karol skocik escribi?: > Hi, > I want to have an escript, which starts some slave nodes making > requests to some target host - basically a simple load generator. > On a slave nodes, running functions from escript directly fails, since > slaves can't find those escript functions, which is understandable. > For that purpose, the escript contains this function: > > make_beam() -> > Name = escript:script_name(), > AbsName = filename:absname(Name), > {ok, Sections} = escript:extract(AbsName, [compile_source]), > {source, BeamCode} = lists:keyfind(source, 1, Sections), > BeamFile = filename:join("/tmp", atom_to_list(?MODULE) ++ ".beam"), > file:write_file(BeamFile, BeamCode), > BeamFile. > > The plan was: > 1.) save the escript as a beam to the "/tmp" directory (with a funky > name from ?MODULE - like 'gcf-tester__escript__1328__608633__210866') > 2.) add "/tmp" to the code path on master, before starting slaves > 3.) let the slaves load module (named ?MODULE in escript) using code > load mechanism thru master > > Now the problem: > > When the saved escript's beam is loaded in the Erlang shell, it seems > broken: > > (gcfc@REDACTED)5> code:add_patha("/tmp"). > true > (gcfc@REDACTED)6> l('gcf-tester__escript__1328__608402__188688'). > {error,badfile} > (gcfc@REDACTED)7> > =ERROR REPORT==== 7-Feb-2012::10:54:09 === > Loading of /tmp/gcf-tester__escript__1328__608402__188688.beam failed: > badfile > > =ERROR REPORT==== 7-Feb-2012::10:54:09 === > beam/beam_load.c(1084): Error loading module > 'gcf-tester__escript__1328__608402__188688': > module name in object code is gcf-tester__escript__1328__608402__542812 > > I am using: > Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] on: > Gentoo Linux 2.6.38-tuxonice-r2-mars #9 SMP PREEMPT x86_64 Intel(R) > Core(TM)2 CPU T7400 @ 2.16GHz GenuineIntel GNU/Linux > > Is this a bug? (Maybe fixed in R15?). > If this is an expected behavior, how do I know the final module name > at the time of creating the escript's beam? > And finally, what's the easiest way to "patch" the beam's binary to > supply the correct module name? > > Thank you for your advice, > Karol Skocik > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From fhunleth@REDACTED Tue Mar 27 15:58:58 2012 From: fhunleth@REDACTED (Frank Hunleth) Date: Tue, 27 Mar 2012 09:58:58 -0400 Subject: [erlang-questions] erl using 99% CPU while idle In-Reply-To: References: Message-ID: On Tue, Mar 27, 2012 at 9:45 AM, Tomasz Maciejewski wrote: >> [snip] > > I don't know if it's even related to the problem, but have tried using > kernel polling (erl +K true)? I just tried it. The CPU is still at 99%. When running strace, the calls to poll() are now calls to epoll_wait(). The timeout parameter on epoll_wait() is 0. Thanks, Frank From vances@REDACTED Tue Mar 27 16:01:23 2012 From: vances@REDACTED (Vance Shipley) Date: Tue, 27 Mar 2012 19:31:23 +0530 Subject: [erlang-questions] Distributed Applications: Stop Before Start? Message-ID: <20120327140122.GD356@aluminum.wavenet.lk> Is there a clever way to get "stop first" semantics out of dist_ac? A distributed application in OTP may migrate from a failed node to another and it may migrate back when the original node returns. You can initiate a transfer with application:permit(foo, false). The order of the callbacks is: a@REDACTED b@REDACTED --------------------- ------------------- foo_app:start({takeover, a@REDACTED}, []) foo_app:start_phase(phase1, {takeover, a@REDACTED}, []) foo_app:start_phase(phase2, {takeover, a@REDACTED}, []) foo_app:prep_stop([]) foo_app:stop([]) Only the start/2 and stop/1 callbacks are required, the others are optional. I'm currently trying to distribute OTP's snmp application between two nodes on the same host but it fails in the situation above because it tries to bind to the address already in use: ** Configuration error: failed starting net-if: {udp_open,161,eaddrinuse} So in this case I'd like to have the application stopped on the old node before it is started on the new node. -- -Vance From g@REDACTED Tue Mar 27 16:01:37 2012 From: g@REDACTED (Garrett Smith) Date: Tue, 27 Mar 2012 09:01:37 -0500 Subject: [erlang-questions] Please correct my leeway on my first Erlang/OTP application In-Reply-To: References: Message-ID: Hi Aleksandr, On Wed, Mar 21, 2012 at 2:05 AM, Aleksandr Vinokurov wrote: > > Hello all, > > I'm quite a newbee in Erlang/OTP and will be very appreciated if you can > point any wrong style or misunderstanding of the OTP in my first Erlang/OTP > application. > > I've put it here?https://github.com/aleksandr-vin/clustmea?and will be glad > to here any of your feedback. I didn't look at everything -- that'd take quite a bit of time. But a couple quick observations: - (as Ladisklav mentioned) your project structure is not canonical -- I suggest using "rebar" to generate your project structure and build your project (rebar isn't perfect, but it will serve you well for this) - Variables named "State2" or "Config2" are a sign that you should think more carefully about your function Here's an example from your project: handle_call(#new{}, _From, State) -> {state, Configs1} = State, Cid = make_ref(), Configs2 = dict:store(Cid, [], Configs1), NewState = State#state{configs=Configs2}, Reply = {ok, Cid}, {reply, Reply, NewState}. When I saw this I thought, "okay, this code needs to be cleaned up -- but what does it *do*?". I then spent several seconds squinting at the code, sorting it out. This version makes it obvious what's going on -- and you don't need to enumerate different variable versions: handle_call(#new{}, _From, State) -> Cid = make_ref(), {reply, {ok, Cid}, store_cid(Cid, State)}. store_cid(Cid, #state{config=C}=State) -> State#state{config=dict:store(Cid, [], C). "Single variable assignment" is a great feature! When you're tempted to write using an imperative style -- long lists of instructions -- take a moment (or several moments) to think carefully about what you're actually doing, and use functions to spell it out clearly. Garrett From ulf@REDACTED Tue Mar 27 16:29:03 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 27 Mar 2012 07:29:03 -0700 Subject: [erlang-questions] Distributed Applications: Stop Before Start? In-Reply-To: <20120327140122.GD356@aluminum.wavenet.lk> References: <20120327140122.GD356@aluminum.wavenet.lk> Message-ID: <6951179A-8889-418C-8C53-D3E7BD1FE138@feuerlabs.com> In the start_phase() call, you can make a gen_server call to the foo_app instance running on a and ask it to close its connections. If that's what you're already doing, perhaps I misunderstood the question? BR, Ulf W On 27 Mar 2012, at 07:01, Vance Shipley wrote: > Is there a clever way to get "stop first" semantics out of dist_ac? > > A distributed application in OTP may migrate from a failed node > to another and it may migrate back when the original node returns. > > You can initiate a transfer with application:permit(foo, false). > The order of the callbacks is: > > a@REDACTED b@REDACTED > --------------------- ------------------- > foo_app:start({takeover, a@REDACTED}, []) > foo_app:start_phase(phase1, {takeover, a@REDACTED}, []) > foo_app:start_phase(phase2, {takeover, a@REDACTED}, []) > foo_app:prep_stop([]) > foo_app:stop([]) > > Only the start/2 and stop/1 callbacks are required, the others are optional. > > I'm currently trying to distribute OTP's snmp application between two > nodes on the same host but it fails in the situation above because it > tries to bind to the address already in use: > > ** Configuration error: failed starting net-if: > {udp_open,161,eaddrinuse} > > So in this case I'd like to have the application stopped on the old > node before it is started on the new node. > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vances@REDACTED Tue Mar 27 16:55:58 2012 From: vances@REDACTED (Vance Shipley) Date: Tue, 27 Mar 2012 20:25:58 +0530 Subject: [erlang-questions] Distributed Applications: Stop Before Start? Message-ID: My current problem is that it's not my application, it's OTP's. It's snmp. I tried using rpc:call/4 in my application's start/2 callback to call application:permit/2 but that just deadlocked. Ulf Wiger wrote: > >In the start_phase() call, you can make a gen_server call to the foo_app >instance running on a and ask it to close its connections. > >If that's what you're already doing, perhaps I misunderstood the question? > >BR, >Ulf W > >On 27 Mar 2012, at 07:01, Vance Shipley wrote: > >> Is there a clever way to get "stop first" semantics out of dist_ac? >> >> A distributed application in OTP may migrate from a failed node >> to another and it may migrate back when the original node returns. >> >> You can initiate a transfer with application:permit(foo, false). >> The order of the callbacks is: >> >> a@REDACTED b@REDACTED >> --------------------- ------------------- >> foo_app:start({takeover, a@REDACTED}, []) >> foo_app:start_phase(phase1, {takeover, a@REDACTED}, []) >> foo_app:start_phase(phase2, {takeover, a@REDACTED}, []) >> foo_app:prep_stop([]) >> foo_app:stop([]) >> >> Only the start/2 and stop/1 callbacks are required, the others are optional. >> >> I'm currently trying to distribute OTP's snmp application between two >> nodes on the same host but it fails in the situation above because it >> tries to bind to the address already in use: >> >> ** Configuration error: failed starting net-if: >> {udp_open,161,eaddrinuse} >> >> So in this case I'd like to have the application stopped on the old >> node before it is started on the new node. >> >> -- >> -Vance >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From ulf@REDACTED Tue Mar 27 17:15:17 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 27 Mar 2012 08:15:17 -0700 Subject: [erlang-questions] Distributed Applications: Stop Before Start? In-Reply-To: References: Message-ID: Ok, then one thing that can be done is to create a 'wrapper application' that starts snmp as an included app. Your start function would have to call the appropriate function in snmp, since included apps are not automatically started by OTP. We used to do this at AXD 301, in order to ensure that applications using snmp did a takeover in synch with snmp itself. BR, Ulf W On 27 Mar 2012, at 07:55, Vance Shipley wrote: > My current problem is that it's not my application, it's OTP's. It's snmp. I tried using rpc:call/4 in my application's start/2 callback to call application:permit/2 but that just deadlocked. > > > Ulf Wiger wrote: > >> >> In the start_phase() call, you can make a gen_server call to the foo_app >> instance running on a and ask it to close its connections. >> >> If that's what you're already doing, perhaps I misunderstood the question? >> >> BR, >> Ulf W >> >> On 27 Mar 2012, at 07:01, Vance Shipley wrote: >> >>> Is there a clever way to get "stop first" semantics out of dist_ac? >>> >>> A distributed application in OTP may migrate from a failed node >>> to another and it may migrate back when the original node returns. >>> >>> You can initiate a transfer with application:permit(foo, false). >>> The order of the callbacks is: >>> >>> a@REDACTED b@REDACTED >>> --------------------- ------------------- >>> foo_app:start({takeover, a@REDACTED}, []) >>> foo_app:start_phase(phase1, {takeover, a@REDACTED}, []) >>> foo_app:start_phase(phase2, {takeover, a@REDACTED}, []) >>> foo_app:prep_stop([]) >>> foo_app:stop([]) >>> >>> Only the start/2 and stop/1 callbacks are required, the others are optional. >>> >>> I'm currently trying to distribute OTP's snmp application between two >>> nodes on the same host but it fails in the situation above because it >>> tries to bind to the address already in use: >>> >>> ** Configuration error: failed starting net-if: >>> {udp_open,161,eaddrinuse} >>> >>> So in this case I'd like to have the application stopped on the old >>> node before it is started on the new node. >>> >>> -- >>> -Vance >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> From fhunleth@REDACTED Tue Mar 27 19:19:06 2012 From: fhunleth@REDACTED (Frank Hunleth) Date: Tue, 27 Mar 2012 13:19:06 -0400 Subject: [erlang-questions] erl using 99% CPU while idle In-Reply-To: References: Message-ID: On Tue, Mar 27, 2012 at 9:37 AM, Frank Hunleth wrote: > I was wondering if anyone had seen erl, specifically the > erts-5.9/bin/beam process, run at 99% CPU while at the shell prompt. Finally found the issue! This was caused by passing the "--disable-threads" option to ./configure when compiling erlang. Removing the option (the default is --enable-threads) and recompiling fixes the problem. One last question: Besides fixing the above issue, I'm a little confused on what --enable-threads does for me. I see "[async-threads: 0]", so does that mean that even though they're enabled, they won't be used by default? Thanks, Frank From james@REDACTED Tue Mar 27 21:20:59 2012 From: james@REDACTED (james) Date: Tue, 27 Mar 2012 20:20:59 +0100 Subject: [erlang-questions] Erlang Jobs? In-Reply-To: <4F70D7BC.1040306@erlang-solutions.com> References: <4F6B8A60.7050006@mansionfamily.plus.com> <4F70D7BC.1040306@erlang-solutions.com> Message-ID: <4F72131B.1090305@mansionfamily.plus.com> Well, I'm still waiting to see if anyone is going to reply to my email offering to wander over and buy some beer near Liverpool Street. Some 'of us' (a nebulous group, bu basically grumpy old C++ contractors each with a couple of decades experience writing tradng systems) are interested in understanding whether there is any fun to be had with Erlang. That's not really the same as being interested in working for you, or any other of the consultancies or media-oriented companies, so I guess we'd be buying. Its more just fact finding to see if its worth investing in Erlang professionally. There is some functional stuff happening for pricing and risk, for example, but there's only so many times you can write message routing and process monitoring subsystems that seem to come out of the box with Erlang. James From dmkolesnikov@REDACTED Tue Mar 27 21:29:31 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Tue, 27 Mar 2012 22:29:31 +0300 Subject: [erlang-questions] Record metadata In-Reply-To: References: <-9083630128156938984@unknownmsgid> Message-ID: <171ACD12-0FFB-455D-9076-1A269FD7E86F@gmail.com> Thanks again for hit! I found the following solution very promising for my need, just need to hack it for record type. http://blogtrader.net/blog/recbird_an_erlang_dynamic_record Regards, Dmitry On Mar 27, 2012, at 1:49 PM, Tim Watson wrote: > You'd need the abstract code available and then you can call > dialyzer_utils:get_record_and_type_info/1 which returns a dict > containing the type information. You can then pass this dict to > various functions in the erl_types module to get back information > about type specs. > > If you don't want to rely on dialyzer at runtime, you'll need to > copy-paste the code from dialyzer_utils AFAICT. Also the erl_types > module is part of hipe. > > On 27 March 2012 07:58, dmitry kolesnikov wrote: >> Hello, >> >> Let say I do have a record definition: >> -record(my, { >> a :: atom(), >> b :: binary() >> }). >> >> Are there any possibility to extract a metadata of record elements >> during run-time? E.g call_some_magic_fun that says R#my.a is atom, >> R#my.b is binary... >> >> Thanks for any advice! >> >> Best Regards, >> Dmitry >-|-|-*> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Tue Mar 27 21:41:17 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 27 Mar 2012 20:41:17 +0100 Subject: [erlang-questions] Record metadata In-Reply-To: <171ACD12-0FFB-455D-9076-1A269FD7E86F@gmail.com> References: <-9083630128156938984@unknownmsgid> <171ACD12-0FFB-455D-9076-1A269FD7E86F@gmail.com> Message-ID: Looks interesting. That code would be quite a lot shorter I think, if it used http://github.com/esl/parse_trans rather than doing it by hand. On 27 March 2012 20:29, Dmitry Kolesnikov wrote: > Thanks again for hit! > > I found the following solution very promising for my need, just need to hack it for record type. > http://blogtrader.net/blog/recbird_an_erlang_dynamic_record > > Regards, Dmitry > > On Mar 27, 2012, at 1:49 PM, Tim Watson wrote: > >> You'd need the abstract code available and then you can call >> dialyzer_utils:get_record_and_type_info/1 which returns a dict >> containing the type information. You can then pass this dict to >> various functions in the erl_types module to get back information >> about type specs. >> >> If you don't want to rely on dialyzer at runtime, you'll need to >> copy-paste the code from dialyzer_utils AFAICT. Also the erl_types >> module is part of hipe. >> >> On 27 March 2012 07:58, dmitry kolesnikov wrote: >>> Hello, >>> >>> Let say I do have a record definition: >>> -record(my, { >>> ?a :: atom(), >>> ?b :: binary() >>> }). >>> >>> Are there any possibility to extract a metadata of record elements >>> during run-time? E.g call_some_magic_fun that says R#my.a is atom, >>> R#my.b is binary... >>> >>> Thanks for any advice! >>> >>> Best Regards, >>> Dmitry >-|-|-*> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > From daniel.goertzen@REDACTED Wed Mar 28 05:22:20 2012 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Tue, 27 Mar 2012 22:22:20 -0500 Subject: [erlang-questions] [ANN] nifpp: C++11 wrappers for NIF API Message-ID: I'm pleased to announce nifpp, C++11 wrappers for the Erlang NIF API. nifpp provides... - Conversions to/from C++ container types, namely tuple, vector, list, deque, set, and multiset. - A safe resource wrapper so that any type can be created as a resource. Source and documentation here: https://github.com/goertzenator/nifpp I need to polish, test and document binary support, but the rest of it is in good shape so I'm sharing it now. Below are links to C and C++ versions of an identical tuple manipulation operation. The C++ version is [imho] easier to understand, and it actually benchmarks slightly faster than the C version on most of my runs. https://github.com/goertzenator/nifpp/blob/master/examples/tuple_twiddle_c.c https://github.com/goertzenator/nifpp/blob/master/examples/tuple_twiddle_cpp.cpp And here is an example of resource wrapping. The Boost class "mapped_file_source" which provides portable file memory mapping is instantiated as an Erlang resource. https://github.com/goertzenator/nifpp/blob/master/examples/mmap_binary.cpp All feedback appreciated. Thanks, Dan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rickard@REDACTED Wed Mar 28 10:01:47 2012 From: rickard@REDACTED (Rickard Green) Date: Wed, 28 Mar 2012 10:01:47 +0200 Subject: [erlang-questions] erl using 99% CPU while idle In-Reply-To: References: Message-ID: <4F72C56B.2050207@erlang.org> > On Tue, Mar 27, 2012 at 9:37 AM, Frank Hunleth > wrote: >> I was wondering if anyone had seen erl, specifically the >> erts-5.9/bin/beam process, run at 99% CPU while at the shell prompt. > > Finally found the issue! This was caused by passing the > "--disable-threads" option to ./configure when compiling erlang. > Removing the option (the default is --enable-threads) and recompiling > fixes the problem. > The following fix for this has successfully passed one daily build, and will probably appear as it is in R15B01. As you noted, this bug does not appear unless you disable thread support. This bug was first introduced in R15B. diff --git a/erts/emulator/sys/common/erl_poll.c b/erts/emulator/sys/common/erl_poll.c index 3817b1e..2685805 100644 --- a/erts/emulator/sys/common/erl_poll.c +++ b/erts/emulator/sys/common/erl_poll.c @@ -2143,7 +2143,7 @@ ERTS_POLL_EXPORT(erts_poll_wait)(ErtsPollSet ps, void ERTS_POLL_EXPORT(erts_poll_interrupt)(ErtsPollSet ps, int set) { -#if defined(USE_THREADS) +#if defined(USE_THREADS) || ERTS_POLL_ASYNC_INTERRUPT_SUPPORT if (!set) reset_wakeup_state(ps); else > One last question: Besides fixing the above issue, I'm a little > confused on what --enable-threads does for me. This only enable thread support for the runtime system (builds runtime system with -D_REENTRANT etc). > I see "[async-threads: > 0]", so does that mean that even though they're enabled, they won't be > used by default? > No they will not be enabled by default. You enable them by passing +A as a command line argument to erl. > Thanks, > Frank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Regards, Rickard Green, Erlang/OTP, Ericsson AB. From hirotnkg@REDACTED Wed Mar 28 10:51:56 2012 From: hirotnkg@REDACTED (Yoshihiro Tanaka) Date: Wed, 28 Mar 2012 01:51:56 -0700 Subject: [erlang-questions] How do funs work ? Message-ID: Hello, I have a function which returns fun as below: make_fun(_, [], F) -> F; make_fun(Operator, [Operand1, Operand2|Operands], F) -> F2 = case Operator of plus -> fun() -> Operand1 + Operand2 + F() end; minus -> fun() -> Operand1 - Operand2 + F() end end, make_fun(Operator, Operands, F2). When I call it as: F1 = make_fun(plus, lists:seq(1,10), fun() -> 0 end). Is F1 same as F2 below ? F2 = fun() -> 1 + 2 + fun() -> 3 + 4 + fun() -> 5 + 6 + fun() -> 7 + 8 + fun() -> 9 + 10 + fun() -> 0 end() end() end() end() end() end. Also, is there any difference between funs that are defined at runtime and funs that are defined at compile time in terms of how they are executed ? Thank you Yoshihiro From potapovsanya@REDACTED Wed Mar 28 12:17:11 2012 From: potapovsanya@REDACTED (Sanya Potapov) Date: Wed, 28 Mar 2012 14:17:11 +0400 Subject: [erlang-questions] The largest number of CPUs utilized by one node Message-ID: Here is the paper which "presents a study on the scalability of the Erlang VM on a many-core processor with 64 cores, TILEPro64." http://kth.diva-portal.org/smash/get/diva2:392243/FULLTEXT01 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hynek@REDACTED Wed Mar 28 13:01:26 2012 From: hynek@REDACTED (Hynek Vychodil) Date: Wed, 28 Mar 2012 13:01:26 +0200 Subject: [erlang-questions] How do funs work ? In-Reply-To: References: Message-ID: I think it should be F2 = fun() -> 9 + 10 + ? ? ? fun() -> 7 + 8 + ? ? ? ? fun() -> 5 + 6 + ? ? ? ? ? fun() -> 3 + 4 + ? ? ? ? ? ? fun() -> 1 + 2 + ? ? ? ? ? ? ? fun() -> 0 end() ? ? ? ? ? ? end() ? ? ? ? ? end() ? ? ? ? end() ? ? ? end() ? ? end. On Wed, Mar 28, 2012 at 10:51 AM, Yoshihiro Tanaka wrote: > Hello, > > I have a function which returns fun as below: > > make_fun(_, [], F) -> F; > make_fun(Operator, [Operand1, Operand2|Operands], F) -> > ?F2 = case Operator of > ? ? ? ? plus ?-> fun() -> Operand1 + Operand2 + F() end; > ? ? ? ? minus -> fun() -> Operand1 - Operand2 + F() end > ? ? ? end, > ?make_fun(Operator, Operands, F2). > > When I call it as: > F1 = make_fun(plus, lists:seq(1,10), fun() -> 0 end). > > Is F1 same as F2 below ? > F2 = fun() -> 1 + 2 + > ? ? ? fun() -> 3 + 4 + > ? ? ? ? fun() -> 5 + 6 + > ? ? ? ? ? fun() -> 7 + 8 + > ? ? ? ? ? ? fun() -> 9 + 10 + > ? ? ? ? ? ? ? fun() -> 0 end() > ? ? ? ? ? ? end() > ? ? ? ? ? end() > ? ? ? ? end() > ? ? ? end() > ? ? end. > > > Also, is there any difference between funs that are defined at runtime > and funs that are defined at compile time in terms of how they are > executed ? > > Thank you > Yoshihiro > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hynek Vychodil BI consultant GoodData n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole Office: ? +420 530 50 7704 E-mail: ?hynek@REDACTED Web: ? ? www.gooddata.com From vances@REDACTED Wed Mar 28 13:40:31 2012 From: vances@REDACTED (Vance Shipley) Date: Wed, 28 Mar 2012 17:10:31 +0530 Subject: [erlang-questions] Distributed Applications: Stop Before Start? In-Reply-To: References: Message-ID: <20120328114011.GA10364@aluminum.wavenet.lk> On Tue, Mar 27, 2012 at 08:15:17AM -0700, Ulf Wiger wrote: } Ok, then one thing that can be done is to create a 'wrapper application' } that starts snmp as an included app. } } Your start function would have to call the appropriate function in snmp, } since included apps are not automatically started by OTP. } } We used to do this at AXD 301, in order to ensure that applications using } snmp did a takeover in synch with snmp itself. Ulf, A couple things about this concern me; /Users/otpuser/lib/erlang/doc/design_principles/included_applications.html "the top supervisor of the included application must be started by a supervisor in the including application" The snmp:start/0,1 snmp:start_agent/0,1 and application:start/1,2 functions don't behave as a supervisor requires. There is an appropriate start function in snmpa_supervisor:start_master_sup/1 however then the snmp application is not listed with application:which_applications/0. "This means that when running, an included application is in fact part of the primary application and a process in an included application will consider itself belonging to the primary application." So if a process in the snmp application would call application:get_all_env/0 it wouldn't get what it expects. It does look like all the calls to the application module do include the application name (snmp) however. -- -Vance From erlang@REDACTED Wed Mar 28 14:32:37 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 28 Mar 2012 14:32:37 +0200 Subject: [erlang-questions] Browser quest anyone? Message-ID: Has anybody taken a peep inside Browser Quest? https://hacks.mozilla.org/2012/03/browserquest/ It's HTML5/websocket game - client in browser server in nodejs Might be fun to implement the server in Erlang and compare with nodejs. This would give all sorts of interesting metrics - lines of code/performance etc. Anybody interested ? /Joe From watson.timothy@REDACTED Wed Mar 28 15:04:10 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 28 Mar 2012 14:04:10 +0100 Subject: [erlang-questions] Distributed Applications: Stop Before Start? In-Reply-To: <20120328114011.GA10364@aluminum.wavenet.lk> References: <20120328114011.GA10364@aluminum.wavenet.lk> Message-ID: On 28 March 2012 12:40, Vance Shipley wrote: > On Tue, Mar 27, 2012 at 08:15:17AM -0700, Ulf Wiger wrote: > } ?Ok, then one thing that can be done is to create a 'wrapper application' > } ?that starts snmp as an included app. > } > } ?Your start function would have to call the appropriate function in snmp, > } ?since included apps are not automatically started by OTP. > } > } ?We used to do this at AXD 301, in order to ensure that applications using > } ?snmp did a takeover in synch with snmp itself. > > Ulf, > > A couple things about this concern me; > > ? ? ? ?/Users/otpuser/lib/erlang/doc/design_principles/included_applications.html > > ?"the top supervisor of the included application must be started by a > ? supervisor in the including application" > > The snmp:start/0,1 snmp:start_agent/0,1 and application:start/1,2 functions > don't behave as a supervisor requires. ?There is an appropriate start > function in snmpa_supervisor:start_master_sup/1 however then the snmp > application is not listed with application:which_applications/0. > > ?"This means that when running, an included application is in fact part > ? of the primary application and a process in an included application > ? will consider itself belonging to the primary application." > > So if a process in the snmp application would call application:get_all_env/0 > it wouldn't get what it expects. ?It does look like all the calls to the > application module do include the application name (snmp) however. > > Is calling application:load(snmp) prior to starting the top level supervisor the 'right way' to resolve this issue when working with included_applications? > -- > ? ? ? ?-Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mfidelman@REDACTED Wed Mar 28 15:05:30 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Wed, 28 Mar 2012 09:05:30 -0400 Subject: [erlang-questions] Browser quest anyone? In-Reply-To: References: Message-ID: <4F730C9A.1010100@meetinghouse.net> Joe Armstrong wrote: > Has anybody taken a peep inside Browser Quest? > > https://hacks.mozilla.org/2012/03/browserquest/ > > It's HTML5/websocket game - client in browser server in nodejs > > Might be fun to implement the server in Erlang and compare with nodejs. > This would give all sorts of interesting metrics - lines of > code/performance etc. > Certainly seems like it has some scaling and/or reliability issues - after initial screen setup, all I get is an infinite "waiting for server" hang. Repeatable in Seamonkey, Chrome, Safari - so it's not a browser issue. Might benefit from Erlang, indeed! Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From bourinov@REDACTED Wed Mar 28 15:38:17 2012 From: bourinov@REDACTED (Max Bourinov) Date: Wed, 28 Mar 2012 17:38:17 +0400 Subject: [erlang-questions] Browser quest anyone? In-Reply-To: <4F730C9A.1010100@meetinghouse.net> References: <4F730C9A.1010100@meetinghouse.net> Message-ID: Hi Joe, We do something similar but with different type of the game. We already outperformed Java implementation by more than 10 times in terms of performance. Cannot say anything about LoC, but having some Java experience I assume codebase is about 10 times smaller (including eunit and ct tests). All this was achieved by our concept prove app. Now we about to release real app (yes. we really re-write it from the scratch.) and we expect even better performance. If mail-list don't mind I will post a link and small story when the game is released. Best regards, Max On Wed, Mar 28, 2012 at 5:05 PM, Miles Fidelman wrote: > Joe Armstrong wrote: > >> Has anybody taken a peep inside Browser Quest? >> >> https://hacks.mozilla.org/**2012/03/browserquest/ >> >> It's HTML5/websocket game - client in browser server in nodejs >> >> Might be fun to implement the server in Erlang and compare with nodejs. >> This would give all sorts of interesting metrics - lines of >> code/performance etc. >> >> > Certainly seems like it has some scaling and/or reliability issues - after > initial screen setup, all I get is an infinite "waiting for server" hang. > Repeatable in Seamonkey, Chrome, Safari - so it's not a browser issue. > Might benefit from Erlang, indeed! > > Miles Fidelman > > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed Mar 28 15:49:46 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 28 Mar 2012 14:49:46 +0100 Subject: [erlang-questions] Browser quest anyone? In-Reply-To: References: <4F730C9A.1010100@meetinghouse.net> Message-ID: On 28 March 2012 14:38, Max Bourinov wrote: > Hi Joe, > > We do?something?similar but with different type of the game. We already > outperformed Java implementation by more than 10 times in terms of > performance. Cannot say anything about LoC, but having some > Java?experience?I assume codebase is about 10 times smaller (including eunit > and ct tests). All this was achieved by our concept prove app. > > Now we about to release real app (yes. we really re-write it from the > scratch.) and we expect even better performance. > > If mail-list don't mind I will post a link and small story when the game is > released. > > Best regards, > Max Yes please Max, I'd love to read about this! From erlang@REDACTED Wed Mar 28 16:02:54 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 28 Mar 2012 16:02:54 +0200 Subject: [erlang-questions] Browser quest anyone? In-Reply-To: References: Message-ID: I had a quick peep at the internals It's web sockets with JSON encoded messages - so the transport layer looks pretty easy From: http://arstechnica.com/business/news/2012/03/mozilla-launches-multiplayer-browser-adventure-to-showcase-html5-gaming.ars?utm_source=rss&utm_medium=rss&utm_campaign=rss The game's remote backend, which enables the real-time multiplayer gameplay, was coded in JavaScript and runs on top of Node.js. The load is balanced across multiple Node.js instances on three separate severs. At the time this story was written, the backend was successfully handling over 1,900 simultaneous players. The status of the BrowserQuest backend can be monitored through the game's real-time dashboard interface. (I guess this mean 650 ish players/machine) It would be very interesting to do a drop-in replacement for the server in erlang - measure performance and #lines of code I'll have to see if I can get it running on my machine :-) /Joe On Wed, Mar 28, 2012 at 2:32 PM, Joe Armstrong wrote: > Has anybody taken a peep inside Browser Quest? > > https://hacks.mozilla.org/2012/03/browserquest/ > > It's HTML5/websocket game - client in browser server in nodejs > > Might be fun to implement the server in Erlang and compare with nodejs. > This would give all sorts of interesting metrics - lines of > code/performance etc. > > Anybody interested ? > > /Joe From attila.r.nohl@REDACTED Wed Mar 28 16:25:37 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 28 Mar 2012 15:25:37 +0100 Subject: [erlang-questions] Performance SNMP counters for the erlang VM itself In-Reply-To: <20120326151424.GA356@aluminum.wavenet.lk> References: <20120326151424.GA356@aluminum.wavenet.lk> Message-ID: Hello! I haven't checked the most obvious place :-) Thanks, this will be good starting point. 2012/3/26 Vance Shipley : > Attila, > > OTP has great SNMP support. ?The otp_mibs application makes VM > performance data available: > > ? ? ? ?http://www.erlang.org/doc/apps/otp_mibs/introduction.html > > The os_mon application handles operating system stuff: > > ? ? ? ?http://www.erlang.org/doc/man/os_mon_app.html > > An example is here: > > ? http://www.trapexit.org/SNMP_Quick_Start#Walking_the_OTP_MIB > > > On Mon, Mar 26, 2012 at 02:21:07PM +0200, Attila Rajmund Nohl wrote: > } ?I would like to get "relevant for performance" data out of an Erlang > } ?VM via SNMP. What do you think, which data is important? Number of > } ?processes? Length of messages queues? Number of reductions? Is there > } ?already an interface like this? This would be used for performance > } ?monitoring, so these counters could be queried about every 1 or 5 or > } ?15 minutes and it would be good if these could indicate that there > } ?will be problems with performance. > } ?_______________________________________________ > } ?erlang-questions mailing list > } ?erlang-questions@REDACTED > } ?http://erlang.org/mailman/listinfo/erlang-questions > > -- > ? ? ? ?-Vance From hirotnkg@REDACTED Wed Mar 28 19:55:52 2012 From: hirotnkg@REDACTED (Yoshihiro Tanaka) Date: Wed, 28 Mar 2012 10:55:52 -0700 Subject: [erlang-questions] How do funs work ? In-Reply-To: References: Message-ID: Yes, you are right. But my point is, in this fun, each part of 'fun() -> something end()' will be executed during F2 is created ? or that part will be executed when I call F2 like F2() ? I'm not sure how it works. I'm also curious about when funs are created at runtime, is there any difference from hard coded funs? It seems like that I can not extract abstract code from funs that are created at runtime using 'erlang:fun_info/2'. I'd appreciate if someone could give me a pointer. Thank you Yoshihiro On Wed, Mar 28, 2012 at 4:01 AM, Hynek Vychodil wrote: > I think it should be > > F2 = fun() -> 9 + 10 + > ? ? ? fun() -> 7 + 8 + > ? ? ? ? fun() -> 5 + 6 + > ? ? ? ? ? fun() -> 3 + 4 + > ? ? ? ? ? ? fun() -> 1 + 2 + > ? ? ? ? ? ? ? fun() -> 0 end() > ? ? ? ? ? ? end() > ? ? ? ? ? end() > ? ? ? ? end() > ? ? ? end() > ? ? end. > > > On Wed, Mar 28, 2012 at 10:51 AM, Yoshihiro Tanaka wrote: >> Hello, >> >> I have a function which returns fun as below: >> >> make_fun(_, [], F) -> F; >> make_fun(Operator, [Operand1, Operand2|Operands], F) -> >> ?F2 = case Operator of >> ? ? ? ? plus ?-> fun() -> Operand1 + Operand2 + F() end; >> ? ? ? ? minus -> fun() -> Operand1 - Operand2 + F() end >> ? ? ? end, >> ?make_fun(Operator, Operands, F2). >> >> When I call it as: >> F1 = make_fun(plus, lists:seq(1,10), fun() -> 0 end). >> >> Is F1 same as F2 below ? >> F2 = fun() -> 1 + 2 + >> ? ? ? fun() -> 3 + 4 + >> ? ? ? ? fun() -> 5 + 6 + >> ? ? ? ? ? fun() -> 7 + 8 + >> ? ? ? ? ? ? fun() -> 9 + 10 + >> ? ? ? ? ? ? ? fun() -> 0 end() >> ? ? ? ? ? ? end() >> ? ? ? ? ? end() >> ? ? ? ? end() >> ? ? ? end() >> ? ? end. >> >> >> Also, is there any difference between funs that are defined at runtime >> and funs that are defined at compile time in terms of how they are >> executed ? >> >> Thank you >> Yoshihiro >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Hynek Vychodil > BI consultant > > GoodData > n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole > Office: ? +420 530 50 7704 > E-mail: ?hynek@REDACTED > Web: ? ? www.gooddata.com From dmercer@REDACTED Wed Mar 28 20:08:31 2012 From: dmercer@REDACTED (David Mercer) Date: Wed, 28 Mar 2012 13:08:31 -0500 Subject: [erlang-questions] How do funs work ? In-Reply-To: References: Message-ID: <005501cd0d0d$ca08e950$5e1abbf0$@com> On Wednesday, March 28, 2012, Yoshihiro Tanaka wrote: > Yes, you are right. But my point is, in this fun, each part of 'fun() > -> something end()' will be executed during F2 is created ? > or that part will be executed when I call F2 like F2() ? I'm not sure > how it works. If I recall correctly, a new function is not created, just the bindings. For instance, if you take a fun and send it in a message, the whole fun isn't being sent, just a reference to the function in the module and the variable bindings. For example: -module(for_example). -export([example/4]). example(X, Y, Z, P) -> F = fun(W) -> (X + Y * Z) / W end, P ! F. When example(1, 2, 3, Pid) is called, a new fun is not created (F(W) -> (1 + 2 * 3) / W end) and sent to Pid. Instead, what's sent to Pid is a reference to the function definition in module for_example with the bindings X = 1, Y = 2, Z = 3. If Pid is on another node with a different version of the for_example module, then it will fail. Not sure exactly what will happen, whether the node will crash, the process crash, or just the message get discarded, but you could probably experiment to figure it out. Someone correct me if I am wrong. This is partly based on memory and partly based on how I figure it would be implemented. Cheers, DBM From yoursurrogategod@REDACTED Wed Mar 28 20:48:32 2012 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Wed, 28 Mar 2012 14:48:32 -0400 Subject: [erlang-questions] "function password_generator/0 undefined" <--- Having a n00b moment. Message-ID: I'm trying to get back into Erlang and figured I'd try this example, just for my own amusement and scant benefit. This is my code and -- frankly -- I don't understand why I'm getting this error. http://bin.cakephp.org/view/439045988 Why am I getting this error? -------------- next part -------------- An HTML attachment was scrubbed... URL: From yoursurrogategod@REDACTED Wed Mar 28 20:52:59 2012 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Wed, 28 Mar 2012 14:52:59 -0400 Subject: [erlang-questions] Erlang jobs Message-ID: Hi all, I'm just curious if there are any Erlang employment opportunities out there. I have a job at the moment, but I'm not optimistic as to how much I can grow here. I do PHP web-development and would like to do more non-web-development, but wouldn't balk at working with Yaws and Erlang (it would be fun in its own way). What are your thoughts and inputs? -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre@REDACTED Wed Mar 28 20:54:54 2012 From: andre@REDACTED (=?ISO-8859-1?Q?Andr=E9_Graf?=) Date: Wed, 28 Mar 2012 20:54:54 +0200 Subject: [erlang-questions] "function password_generator/0 undefined" <--- Having a n00b moment. In-Reply-To: References: Message-ID: Hello The two password_generator functions are of different arity (0 and 5), so you cannot terminate the arity 0 function with the semicolon. Use the dot instead. Cheers Andre On 28 March 2012 20:48, Yves S. Garret wrote: > I'm trying to get back into Erlang and figured I'd try this example, just > for my own amusement and scant benefit. > > This is my code and -- frankly -- I don't understand why I'm getting this > error. > http://bin.cakephp.org/view/439045988 > > Why am I getting this error? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From gleber.p@REDACTED Wed Mar 28 20:54:54 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 28 Mar 2012 20:54:54 +0200 Subject: [erlang-questions] "function password_generator/0 undefined" <--- Having a n00b moment. In-Reply-To: References: Message-ID: On Wed, Mar 28, 2012 at 20:48, Yves S. Garret wrote: > I'm trying to get back into Erlang and figured I'd try this example, just > for my own amusement and scant benefit. Take a look at a semicolon at the end of the first clause. There should be a dot instead From gumm@REDACTED Wed Mar 28 20:58:10 2012 From: gumm@REDACTED (Jesse Gumm) Date: Wed, 28 Mar 2012 13:58:10 -0500 Subject: [erlang-questions] Erlang jobs In-Reply-To: References: Message-ID: There's actually another thread about this topic from last week. http://erlang.org/pipermail/erlang-questions/2012-March/065363.html Lots of good info in there. -Jesse On Wed, Mar 28, 2012 at 1:52 PM, Yves S. Garret wrote: > Hi all, I'm just curious if there are any Erlang employment opportunities > out there.? I have a job at the moment, but I'm not optimistic as to how > much I can grow here.? I do PHP web-development and would like to do more > non-web-development, but wouldn't balk at working with Yaws and Erlang (it > would be fun in its own way). > > What are your thoughts and inputs? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm From yoursurrogategod@REDACTED Wed Mar 28 21:15:55 2012 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Wed, 28 Mar 2012 15:15:55 -0400 Subject: [erlang-questions] "function password_generator/0 undefined" <--- Having a n00b moment. In-Reply-To: References: Message-ID: I thought that one of the things that I can do with methods in Erlang is use the ';' in order to cycle through a number of different input types for methods. I think I might have forgotten some parts of this works :-) . *starts digging up his Erlang book* On Wed, Mar 28, 2012 at 2:54 PM, Gleb Peregud wrote: > On Wed, Mar 28, 2012 at 20:48, Yves S. Garret > wrote: > > I'm trying to get back into Erlang and figured I'd try this example, just > > for my own amusement and scant benefit. > > Take a look at a semicolon at the end of the first clause. There > should be a dot instead > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Mar 28 22:15:21 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 29 Mar 2012 00:15:21 +0400 Subject: [erlang-questions] Erlang jobs In-Reply-To: References: Message-ID: I think, that when we speak about "erlang programming" there is no plain relations "look for a job on monster.com". If there is a company that has such business requirements that can be solved with erlang, they need engineer of such qualification than can solve this problem in any language. But Erlang will be the best way to solve it. From yoursurrogategod@REDACTED Wed Mar 28 23:05:20 2012 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Wed, 28 Mar 2012 17:05:20 -0400 Subject: [erlang-questions] Erlang jobs In-Reply-To: References: Message-ID: Since I could still use a little bit of experience, how would you guys recommend I gain some experience working in Erlang? I could go through my book and try everything that they have in it. An open-source projects? A list of problems I could solve just for the heck of it? On Wed, Mar 28, 2012 at 4:15 PM, Max Lapshin wrote: > I think, that when we speak about "erlang programming" there is no > plain relations "look for a job on monster.com". > > If there is a company that has such business requirements that can be > solved with erlang, they need engineer of such qualification than can > solve this problem in any language. But Erlang will be the best way to > solve it. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sam@REDACTED Wed Mar 28 23:46:20 2012 From: sam@REDACTED (Samuel Elliott) Date: Wed, 28 Mar 2012 23:46:20 +0200 Subject: [erlang-questions] "function password_generator/0 undefined" <--- Having a n00b moment. In-Reply-To: References: Message-ID: On Wed, Mar 28, 2012 at 9:15 PM, Yves S. Garret wrote: > I thought that one of the things that I can do with methods in Erlang is use > the ';' in order to? cycle through a number of different input types for > methods. You're correct, however two functions are different functions if their arity is different. You'd use a semicolon between the different inputs to foo/2, but that's a completely different function to foo/1 so you use a dot between the two functions themselves. I hope that's clear, if it's not, I'm sure someone else on the list will have a far clearer explanation than I can manage. Sam > > I think I might have forgotten some parts of this works :-) . > > *starts digging up his Erlang book* > > > On Wed, Mar 28, 2012 at 2:54 PM, Gleb Peregud wrote: >> >> On Wed, Mar 28, 2012 at 20:48, Yves S. Garret >> wrote: >> > I'm trying to get back into Erlang and figured I'd try this example, >> > just >> > for my own amusement and scant benefit. >> >> Take a look at a semicolon at the end of the first clause. There >> should be a dot instead > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Samuel Elliott sam@REDACTED http://lenary.co.uk/ +44 (0)7891 993 664 From zerthurd@REDACTED Thu Mar 29 05:21:33 2012 From: zerthurd@REDACTED (Maxim Treskin) Date: Thu, 29 Mar 2012 10:21:33 +0700 Subject: [erlang-questions] The largest number of CPUs utilized by one node In-Reply-To: References: Message-ID: Yes, I know about this paper, but in theory there is no difference between practice and theory, but in practice there is. So, will be more interesting to hear about the real situation. On 28 March 2012 17:17, Sanya Potapov wrote: > Here is the paper which "presents a study on the scalability of the > Erlang VM on a many-core processor with 64 cores, TILEPro64." > > http://kth.diva-portal.org/smash/get/diva2:392243/FULLTEXT01 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Maxim Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Thu Mar 29 06:50:52 2012 From: vances@REDACTED (Vance Shipley) Date: Thu, 29 Mar 2012 10:20:52 +0530 Subject: [erlang-questions] Distributed Applications: Stop Before Start? In-Reply-To: References: <20120328114011.GA10364@aluminum.wavenet.lk> Message-ID: <20120329045035.GA14994@aluminum.wavenet.lk> On Wed, Mar 28, 2012 at 02:04:10PM +0100, Tim Watson wrote: } On 28 March 2012 12:40, Vance Shipley wrote: } > /Users/otpuser/lib/erlang/doc/design_principles/included_applications.html } > } > ?"This means that when running, an included application is in fact part } > ? of the primary application and a process in an included application } > ? will consider itself belonging to the primary application." } > } > So if a process in the snmp application would call application:get_all_env/0 } > it wouldn't get what it expects. ?It does look like all the calls to the } > application module do include the application name (snmp) however. } } Is calling application:load(snmp) prior to starting the top level } supervisor the 'right way' to resolve this issue when working with } included_applications? No, loading the application doesn't help. It's loaded anyway by virtue of being listed in the included_applications key of the application resource file. -- -Vance From vances@REDACTED Thu Mar 29 08:28:13 2012 From: vances@REDACTED (Vance Shipley) Date: Thu, 29 Mar 2012 11:58:13 +0530 Subject: [erlang-questions] Distributed Applications: Stop Before Start? In-Reply-To: <20120328114011.GA10364@aluminum.wavenet.lk> References: <20120328114011.GA10364@aluminum.wavenet.lk> Message-ID: <20120329062812.GB14994@aluminum.wavenet.lk> On Wed, Mar 28, 2012 at 05:10:31PM +0530, Vance Shipley wrote: } The snmp:start/0,1 snmp:start_agent/0,1 and application:start/1,2 functions } don't behave as a supervisor requires. There is an appropriate start } function in snmpa_supervisor:start_master_sup/1 however then the snmp } application is not listed with application:which_applications/0. I did implement this, and it seems to work well. I am listing snmp in my application's included_applications key and starting the snmpa_supervisor from it's top level supervisor. When Appname:start({takeover, Node}, []) is called I use supervisor:terminate_child({AppName, Node}, AppSupName, Id) to stop the snmp agent at the old node. -- -Vance From matthias@REDACTED Thu Mar 29 09:04:17 2012 From: matthias@REDACTED (Matthias Lang) Date: Thu, 29 Mar 2012 09:04:17 +0200 Subject: [erlang-questions] erl using 99% CPU while idle In-Reply-To: References: Message-ID: <20120329070416.GA2480@corelatus.se> On Tuesday, March 27, Frank Hunleth wrote: > I was wondering if anyone had seen erl, specifically the > erts-5.9/bin/beam process, run at 99% CPU while at the shell prompt. > All that I do is run erl without any command line arguments and let it > sit. This is a cross-compiled Erlang R15B running on an ARM processor, I don't see this. Erlang R15B (erts-5.9) [source] [async-threads:0] [kernel-poll:false] Linux gth 2.6.36-07959-g6f9f1ac #1 Mon Nov 14 00:29:39 CET 2011 armv5tejl GNU/Linux gth3 >arm-linux-gcc -v Reading specs from /usr/local/eldk42/usr/bin/../lib/gcc/arm-linux-gnueabi/4.2.2/specs Target: arm-linux-gnueabi Configured with: /opt/eldk/build/arm-2008-11-24/work/usr/src/denx/BUILD/crosstool-0.43/build/gcc-4.2.2-glibc-20070515T2025-eldk/arm-linux-gnueabi/gcc-4.2.2/configure --target=arm-linux-gnueabi --host=i686-host_pc-linux-gnu --prefix=/var/tmp/eldk.ywMqKk/usr/crosstool/gcc-4.2.2-glibc-20070515T2025-eldk/arm-linux-gnueabi --disable-hosted-libstdcxx --with-headers=/var/tmp/eldk.ywMqKk/usr/crosstool/gcc-4.2.2-glibc-20070515T2025-eldk/arm-linux-gnueabi/arm-linux-gnueabi/include --with-local-prefix=/var/tmp/eldk.ywMqKk/usr/crosstool/gcc-4.2.2-glibc-20070515T2025-eldk/arm-linux-gnueabi/arm-linux-gnueabi --disable-nls --enable-threads=posix --enable-symvers=gnu --enable-__cxa_atexit --enable-languages=c,c++,java --enable-shared --enable-c99 --enable-long-long --without-x Thread model: posix gcc version 4.2.2 Mail me off-list if you want a copy of my binary. Matt From alessandro.sivieri@REDACTED Thu Mar 29 09:27:21 2012 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Thu, 29 Mar 2012 09:27:21 +0200 Subject: [erlang-questions] erl using 99% CPU while idle In-Reply-To: References: Message-ID: On Tue, Mar 27, 2012 at 3:37 PM, Frank Hunleth < fhunleth@REDACTED> wrote: > I was wondering if anyone had seen erl, specifically the > erts-5.9/bin/beam process, run at 99% CPU while at the shell prompt. > All that I do is run erl without any command line arguments and let it > sit. This is a cross-compiled Erlang R15B running on an ARM processor, > so it could be the case that I messed up an option somewhere along the > line. FWIW, the processing must be trivial since the system feels > completely responsive and Erlang applications appear to run fine. I > ran strace to see what was going on and the following lines repeat: > > The same thing happen on MIPS microprocessors, and it is solved by not specifying the --disable-threads option. Thanks for having solved this. -- Sivieri Alessandro alessandro.sivieri@REDACTED http://sivieri.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Mar 29 09:32:33 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 29 Mar 2012 20:32:33 +1300 Subject: [erlang-questions] Fear and Loathing in Programming La La Land Message-ID: I'm to the Psychology of Programming Interest Group mailing list. In a discussion of naming, reference was made to a body of work containing Usability Implications of Requiring Parameters in Objects' Constructors by Jeffrey Stylos of CMU and Steven Clarke of Microsoft at http://www.cs.cmu.edu/~NatProg/papers/Stylos2007CreateSetCall.pdf The abstract reads The usability of APIs is increasingly important to programmer productivity. Based on experience with usability studies of specific APIs, techniques were explored for studying the usability of design choices common to many APIs. A comparative study was performed to assess how professional programmers use APIs with required parameters in objects? constructors as opposed to parameterless ?default? constructors. It was hypothesized that required parameters would create more usable and self-documenting APIs by guiding programmers toward the correct use of objects and preventing errors. However, in the study, it was found that, contrary to expectations, programmers strongly preferred and were more effective with APIs that did not require constructor parameters. Participants? behavior was analyzed using the cognitive dimensions framework, and revealing that required constructor parameters interfere with common learning strategies, causing undesirable premature commitment. The study was done in 2005. We're talking about the difference between fs = new FileReader("foo/bar.txt", Sharing.NONE); ln = fs.ReadLine(); ... and fs = new FileReader(); fs.SetFileName("foo/bar.txt"); fs.SetSharing(Sharing.NONE); ln = fs.ReadLine(); ... I think this is worth bringing up here because if functional programming is about anything at all, it's certainly about NOT setting up a value one field at a time! Their sample was carefully chosen to include three kinds of programmers: * OPPORTUNISITC programmers are more concerned with productivity than control or understanding. For these programmers objects that required constructor parameters were unfamiliar and unexpected, and even after repeated exposure these programmers had difficulty with these objects. That is, they just didn't "get" the idea of constructors having parameters. * PRAGMATIC programmers balance productivity with control and understanding. These programmers also did not expect objects with required constructors, and while pragmatic programmers were more effective than opportunistic programmers at using these objects, the objects still provided a minor stumbling block and these programmers preferred the flexibility offered by objects that used the create-set-call pattern. Remember, this was all about .NET. Failing to expect constructors with parameters in C# is like failing to expect assignment statements in C. * SYSTEMATIC programmers program defensively and these are the programmers for whom low-level APIs are targeted. These programmers were effective at using all of the objects; however, they preferred create-set-call because of the finer granularity of control it offered by allowing objects to be initialized one piece at a time. The purpose of the study was to provide guidelines for API designers at Microsoft: apparently they now recommend create-set-call. Remember, that's the idea where you create an object without saying *anything* about what you want it to be and then successively kick it into shape. They later say [Systematic programmers] want not just to get their code working, but to understand why it works, what assumptions it makes and when it might fail. They are rare, and prefer languages that give them the most detailed control such as C++, C and assembly. I'd like to think of myself as a systematic programmer. I certainly like to understand all those things. But if I preferred assembly I would not be writing in the Erlang mailing list! The basic conclusion seems to be that you should not design APIs for such rare animals. I made what I thought were three obvious points: (1) There is a confounding factor in the study: the create-set-call style lets you *name* the information going into an object, while at that time the full-initialisation style did not. There are constructors for System.IO.FileStream with six arguments; maybe more. It seemed at least *possible* that if the subjects had been given a third choice, constructors with *named* parameters, they might have preferred that. Because the study didn't include such a choice, we certainly cannot use it to argue *against* that style. (2) C# 4.0 has named (and optional) parameters, so the study is no longer adequate to tell us about good API design in C#. (3) If there are programmers out there who *don't* care much about understanding what they are doing, I certainly don't want them writing anything that might in any way affect me or anyone I care about. If they just don't "get" constructors with parameters, that's really scary. A lengthy discussion has followed in which I've felt rather lonely. I'm being beaten about the head for not noticing things in the study that I did notice, and for being rather elitist. One quote: We don?t have the luxury of dismissing these types of programmers. While it might strike you with terror that these programmers exist, It doesn't frighten me that they _exist_, it frightens me that they are _programming_. they are successfully building applications in many different domains. They may work differently to you and many other programmers but that doesn?t necessarily mean that the code they create is worthless. Within the Visual Studio team at Microsoft we?ve devoted efforts to attempting to make them successful by adapting to their workstyles when appropriate. I find myself wondering just how "successful" their applications really are. Oh no! I hope they're not writing climate modelling code! That would explain so much... (Yes, I've looked at the code in the ClimateGate dump.) Frankly, I've turned here for a bit of comfort. If anyone likes completely initialising things rather than smacking one field at a time, surely I will find such people here. Am I wrong? Would *you* be happy opening a file in C by doing FILE *f = malloc(sizeof f); set_title(f, "foo/bar.txt"); set_access(f, "r"); gets(f, buffer); From vladdu55@REDACTED Thu Mar 29 09:42:55 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 29 Mar 2012 09:42:55 +0200 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: References: Message-ID: Hi Richard, On Thu, Mar 29, 2012 at 09:32, Richard O'Keefe wrote: <...snip...> > Frankly, I've turned here for a bit of comfort. ?If anyone likes completely > initialising things rather than smacking one field at a time, surely I will > find such people here. > > Am I wrong? ?Would *you* be happy opening a file in C by doing > > ? ? ? ?FILE *f = malloc(sizeof f); > ? ? ? ?set_title(f, "foo/bar.txt"); > ? ? ? ?set_access(f, "r"); > ? ? ? ?gets(f, buffer); No, you are not wrong nor alone :-) I'm doing a lot of Java these days and the "ideology" that I'm following is to try to get *all* initialization in the constructor. If there are many constructor parameters (especially with the same types, when they are difficult to tell apart), then I use a builder object that can construct my desired target one piece at a time and has sensible defaults. Something like (made up example) File f = new FileBuilder().withTitle("foo").withAccess("r").build(); best regards, Vlad From steve@REDACTED Thu Mar 29 09:49:22 2012 From: steve@REDACTED (Steve Strong) Date: Thu, 29 Mar 2012 09:49:22 +0200 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: References: Message-ID: <4F1DDB056DD742D3A07BC6ABE7804DFE@srstrong.com> You're certainly not alone :) The idea of create-set-call is, imho, madness. It means that you never really know if an object is in a sane state to be used - what properties need to be set to make the object "usable"? What happens if some properties are set more than once? What if I set some properties, start using the object, and then change some properties? I much, much prefer passing arguments to a constructor and knowing that when it returns to me I have an object in a sane state to use. That said, some of the C# objects have way to many overloaded constructors - named parameters no doubt help here, as would better adherence to the SRP. The level of some of the programmers out there is scary - if it were the medical or legal profession, they'd be struck off - who'd want an 'opportunistic' doctor, who's measure of success was how quickly he could get through patients, with scant disregard to how many of them later end up at the mortuary... Love the C example :) -- Steve. Strong @srstrong Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Thursday, 29 March 2012 at 09:32, Richard O'Keefe wrote: > I'm to the Psychology of Programming Interest Group mailing list. > In a discussion of naming, reference was made to a body of work > containing > Usability Implications of Requiring Parameters > in Objects' Constructors > by Jeffrey Stylos of CMU > and Steven Clarke of Microsoft > at http://www.cs.cmu.edu/~NatProg/papers/Stylos2007CreateSetCall.pdf > > The abstract reads > The usability of APIs is increasingly important to programmer > productivity. Based on experience with usability studies of > specific APIs, techniques were explored for studying the usability > of design choices common to many APIs. A comparative study was > performed to assess how professional programmers use APIs with > required parameters in objects? constructors as opposed to > parameterless ?default? constructors. It was hypothesized that > required parameters would create more usable and self-documenting > APIs by guiding programmers toward the correct use of objects and > preventing errors. However, in the study, it was found that, > contrary to expectations, programmers strongly preferred and > were more effective with APIs that did not require constructor > parameters. Participants? behavior was analyzed using the > cognitive dimensions framework, and revealing that required > constructor parameters interfere with common learning strategies, > causing undesirable premature commitment. > > The study was done in 2005. > We're talking about the difference between > > fs = new FileReader("foo/bar.txt", Sharing.NONE); > ln = fs.ReadLine(); > ... > > and > > fs = new FileReader(); > fs.SetFileName("foo/bar.txt"); > fs.SetSharing(Sharing.NONE); > ln = fs.ReadLine(); > ... > > I think this is worth bringing up here because if functional programming > is about anything at all, it's certainly about NOT setting up a value one > field at a time! > > Their sample was carefully chosen to include three kinds of programmers: > > * OPPORTUNISITC programmers are more concerned with productivity > than control or understanding. For these programmers objects > that required constructor parameters were unfamiliar and > unexpected, and even after repeated exposure these programmers > had difficulty with these objects. > > That is, they just didn't "get" the idea of constructors having > parameters. > > * PRAGMATIC programmers balance productivity with control and > understanding. These programmers also did not expect objects > with required constructors, and while pragmatic programmers > were more effective than opportunistic programmers at using > these objects, the objects still provided a minor stumbling > block and these programmers preferred the flexibility offered > by objects that used the create-set-call pattern. > > Remember, this was all about .NET. Failing to expect > constructors with parameters in C# is like failing to expect > assignment statements in C. > > * SYSTEMATIC programmers program defensively and these are the > programmers for whom low-level APIs are targeted. These programmers > were effective at using all of the objects; however, they preferred > create-set-call because of the finer granularity of control it > offered by allowing objects to be initialized one piece at a time. > > The purpose of the study was to provide guidelines for API designers at > Microsoft: apparently they now recommend create-set-call. Remember, > that's the idea where you create an object without saying *anything* about > what you want it to be and then successively kick it into shape. > > They later say > > [Systematic programmers] want not just to get their code working, > but to understand why it works, what assumptions it makes > and when it might fail. They are rare, and prefer languages that > give them the most detailed control such as C++, C and assembly. > > I'd like to think of myself as a systematic programmer. I certainly like > to understand all those things. But if I preferred assembly I would not > be writing in the Erlang mailing list! The basic conclusion seems to be > that you should not design APIs for such rare animals. > > I made what I thought were three obvious points: > > (1) There is a confounding factor in the study: the create-set-call style > lets you *name* the information going into an object, while at that > time the full-initialisation style did not. There are constructors > for System.IO.FileStream with six arguments; maybe more. It seemed at > least *possible* that if the subjects had been given a third choice, > constructors with *named* parameters, they might have preferred that. > Because the study didn't include such a choice, we certainly cannot > use it to argue *against* that style. > > (2) C# 4.0 has named (and optional) parameters, so the study is no longer > adequate to tell us about good API design in C#. > > (3) If there are programmers out there who *don't* care much about > understanding what they are doing, I certainly don't want them writing > anything that might in any way affect me or anyone I care about. > If they just don't "get" constructors with parameters, that's really > scary. > > A lengthy discussion has followed in which I've felt rather lonely. > I'm being beaten about the head for not noticing things in the study > that I did notice, and for being rather elitist. One quote: > > We don?t have the luxury of dismissing these types of programmers. > While it might strike you with terror that these programmers exist, > > It doesn't frighten me that they _exist_, > it frightens me that they are _programming_. > > they are successfully building applications in many different domains. > They may work differently to you and many other programmers but that > doesn?t necessarily mean that the code they create is worthless. > Within the Visual Studio team at Microsoft we?ve devoted efforts to > attempting to make them successful by adapting to their workstyles > when appropriate. > > I find myself wondering just how "successful" their applications really are. > Oh no! I hope they're not writing climate modelling code! That would > explain so much... (Yes, I've looked at the code in the ClimateGate dump.) > > Frankly, I've turned here for a bit of comfort. If anyone likes completely > initialising things rather than smacking one field at a time, surely I will > find such people here. > > Am I wrong? Would *you* be happy opening a file in C by doing > > FILE *f = malloc(sizeof f); > set_title(f, "foo/bar.txt"); > set_access(f, "r"); > gets(f, buffer); > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hirotnkg@REDACTED Thu Mar 29 09:53:00 2012 From: hirotnkg@REDACTED (Yoshihiro Tanaka) Date: Thu, 29 Mar 2012 00:53:00 -0700 Subject: [erlang-questions] How do funs work ? In-Reply-To: <005501cd0d0d$ca08e950$5e1abbf0$@com> References: <005501cd0d0d$ca08e950$5e1abbf0$@com> Message-ID: Thank you David for making it clear. It all makes sense. Yoshihiro On Wed, Mar 28, 2012 at 11:08 AM, David Mercer wrote: > On Wednesday, March 28, 2012, Yoshihiro Tanaka wrote: > >> Yes, you are right. But my point is, in this fun, each part of 'fun() >> -> something end()' will be executed during F2 is created ? >> or that part will be executed when I call F2 like F2() ? I'm not sure >> how it works. > > If I recall correctly, a new function is not created, just the bindings. ?For instance, if you take a fun and send it in a message, the whole fun isn't being sent, just a reference to the function in the module and the variable bindings. ?For example: > > -module(for_example). > -export([example/4]). > > example(X, Y, Z, P) -> > ? ? ? ?F = fun(W) -> (X + Y * Z) / W end, > ? ? ? ?P ! F. > > When example(1, 2, 3, Pid) is called, a new fun is not created (F(W) -> (1 + 2 * 3) / W end) and sent to Pid. ?Instead, what's sent to Pid is a reference to the function definition in module for_example with the bindings X = 1, Y = 2, Z = 3. > > If Pid is on another node with a different version of the for_example module, then it will fail. ?Not sure exactly what will happen, whether the node will crash, the process crash, or just the message get discarded, but you could probably experiment to figure it out. > > Someone correct me if I am wrong. ?This is partly based on memory and partly based on how I figure it would be implemented. > > Cheers, > > DBM > From masklinn@REDACTED Thu Mar 29 10:04:22 2012 From: masklinn@REDACTED (Masklinn) Date: Thu, 29 Mar 2012 10:04:22 +0200 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: References: Message-ID: <19485CC0-B4D1-44F2-A0F2-BE3C7C182823@masklinn.net> On 2012-03-29, at 09:32 , Richard O'Keefe wrote: > > Am I wrong? Would *you* be happy opening a file in C by doing > > FILE *f = malloc(sizeof f); > set_title(f, "foo/bar.txt"); > set_access(f, "r"); > gets(f, buffer); I'm not aware of very many people who'd be: even back when I used Java, most people I worked with were not fond of this style at all (and IOC containers, which were gaining in popularity, tended to promote constructor injection ? where all values are set in the constructor ? more than property injection ? where the constructor takes no argument and all properties are set via setters). Going through constructors/atomically building objects has issues mostly in two situations: a language without keyword arguments ? which can be neatly side-stepped by using "builders"; and objects with far too much configurable state ? which should be solved by breaking things into smaller and more self-contained pieces. To answer the question, no, I can't say I would be happy. From lenartlad@REDACTED Thu Mar 29 11:00:13 2012 From: lenartlad@REDACTED (Ladislav Lenart) Date: Thu, 29 Mar 2012 11:00:13 +0200 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: References: Message-ID: <4F74249D.5000703@volny.cz> Hello. I completely agree with you Richard. That is, I dislike the create- -set-call style and mindset. From my personal experience in Smalltalk which has keyword messages, it is a breeze to create and set an object in one go. Plus you have a potential to create a value (=stateless) object, i.e. one that does NOT change in time. And that IS the property we all here seek, appreaciate and enjoy very much :-) You simply cannot achieve this with the create-set-call style EVEN IF YOU OTHERWISE COULD. This alone should be enough reason to abandon it completely. Also, perhaps it is a little bit of a chicken-egg problem. If the programmers are accustomed to some predominant programming style, it is very likely they prefer it simply because they know it even though it is suboptimal. Ladislav Lenart On 29.3.2012 09:32, Richard O'Keefe wrote: > I'm to the Psychology of Programming Interest Group mailing list. > In a discussion of naming, reference was made to a body of work > containing > Usability Implications of Requiring Parameters > in Objects' Constructors > by Jeffrey Stylos of CMU > and Steven Clarke of Microsoft > at http://www.cs.cmu.edu/~NatProg/papers/Stylos2007CreateSetCall.pdf > > The abstract reads > The usability of APIs is increasingly important to programmer > productivity. Based on experience with usability studies of > specific APIs, techniques were explored for studying the usability > of design choices common to many APIs. A comparative study was > performed to assess how professional programmers use APIs with > required parameters in objects? constructors as opposed to > parameterless ?default? constructors. It was hypothesized that > required parameters would create more usable and self-documenting > APIs by guiding programmers toward the correct use of objects and > preventing errors. However, in the study, it was found that, > contrary to expectations, programmers strongly preferred and > were more effective with APIs that did not require constructor > parameters. Participants? behavior was analyzed using the > cognitive dimensions framework, and revealing that required > constructor parameters interfere with common learning strategies, > causing undesirable premature commitment. > > The study was done in 2005. > We're talking about the difference between > > fs = new FileReader("foo/bar.txt", Sharing.NONE); > ln = fs.ReadLine(); > ... > > and > > fs = new FileReader(); > fs.SetFileName("foo/bar.txt"); > fs.SetSharing(Sharing.NONE); > ln = fs.ReadLine(); > ... > > I think this is worth bringing up here because if functional programming > is about anything at all, it's certainly about NOT setting up a value one > field at a time! > > Their sample was carefully chosen to include three kinds of programmers: > > * OPPORTUNISITC programmers are more concerned with productivity > than control or understanding. For these programmers objects > that required constructor parameters were unfamiliar and > unexpected, and even after repeated exposure these programmers > had difficulty with these objects. > > That is, they just didn't "get" the idea of constructors having > parameters. > > * PRAGMATIC programmers balance productivity with control and > understanding. These programmers also did not expect objects > with required constructors, and while pragmatic programmers > were more effective than opportunistic programmers at using > these objects, the objects still provided a minor stumbling > block and these programmers preferred the flexibility offered > by objects that used the create-set-call pattern. > > Remember, this was all about .NET. Failing to expect > constructors with parameters in C# is like failing to expect > assignment statements in C. > > * SYSTEMATIC programmers program defensively and these are the > programmers for whom low-level APIs are targeted. These programmers > were effective at using all of the objects; however, they preferred > create-set-call because of the finer granularity of control it > offered by allowing objects to be initialized one piece at a time. > > The purpose of the study was to provide guidelines for API designers at > Microsoft: apparently they now recommend create-set-call. Remember, > that's the idea where you create an object without saying *anything* about > what you want it to be and then successively kick it into shape. > > They later say > > [Systematic programmers] want not just to get their code working, > but to understand why it works, what assumptions it makes > and when it might fail. They are rare, and prefer languages that > give them the most detailed control such as C++, C and assembly. > > I'd like to think of myself as a systematic programmer. I certainly like > to understand all those things. But if I preferred assembly I would not > be writing in the Erlang mailing list! The basic conclusion seems to be > that you should not design APIs for such rare animals. > > I made what I thought were three obvious points: > > (1) There is a confounding factor in the study: the create-set-call style > lets you *name* the information going into an object, while at that > time the full-initialisation style did not. There are constructors > for System.IO.FileStream with six arguments; maybe more. It seemed at > least *possible* that if the subjects had been given a third choice, > constructors with *named* parameters, they might have preferred that. > Because the study didn't include such a choice, we certainly cannot > use it to argue *against* that style. > > (2) C# 4.0 has named (and optional) parameters, so the study is no longer > adequate to tell us about good API design in C#. > > (3) If there are programmers out there who *don't* care much about > understanding what they are doing, I certainly don't want them writing > anything that might in any way affect me or anyone I care about. > If they just don't "get" constructors with parameters, that's really > scary. > > A lengthy discussion has followed in which I've felt rather lonely. > I'm being beaten about the head for not noticing things in the study > that I did notice, and for being rather elitist. One quote: > > We don?t have the luxury of dismissing these types of programmers. > While it might strike you with terror that these programmers exist, > > It doesn't frighten me that they _exist_, > it frightens me that they are _programming_. > > they are successfully building applications in many different domains. > They may work differently to you and many other programmers but that > doesn?t necessarily mean that the code they create is worthless. > Within the Visual Studio team at Microsoft we?ve devoted efforts to > attempting to make them successful by adapting to their workstyles > when appropriate. > > I find myself wondering just how "successful" their applications really are. > Oh no! I hope they're not writing climate modelling code! That would > explain so much... (Yes, I've looked at the code in the ClimateGate dump.) > > Frankly, I've turned here for a bit of comfort. If anyone likes completely > initialising things rather than smacking one field at a time, surely I will > find such people here. > > Am I wrong? Would *you* be happy opening a file in C by doing > > FILE *f = malloc(sizeof f); > set_title(f, "foo/bar.txt"); > set_access(f, "r"); > gets(f, buffer); > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From prof3ta@REDACTED Thu Mar 29 11:00:55 2012 From: prof3ta@REDACTED (Roberto Aloi) Date: Thu, 29 Mar 2012 11:00:55 +0200 Subject: [erlang-questions] How do funs work ? In-Reply-To: References: <005501cd0d0d$ca08e950$5e1abbf0$@com> Message-ID: Just as a reference, some time ago Kresten wrote a really interested article about funs and transferring funs between Erlang nodes. Might be of interest: http://www.javalimit.com/2010/05/passing-funs-to-other-erlang-nodes.html Cheers, Roberto Aloi @robertoaloi On Thu, Mar 29, 2012 at 9:53 AM, Yoshihiro Tanaka wrote: > Thank you David for making it clear. > It all makes sense. > > Yoshihiro > > > On Wed, Mar 28, 2012 at 11:08 AM, David Mercer wrote: >> On Wednesday, March 28, 2012, Yoshihiro Tanaka wrote: >> >>> Yes, you are right. But my point is, in this fun, each part of 'fun() >>> -> something end()' will be executed during F2 is created ? >>> or that part will be executed when I call F2 like F2() ? I'm not sure >>> how it works. >> >> If I recall correctly, a new function is not created, just the bindings. ?For instance, if you take a fun and send it in a message, the whole fun isn't being sent, just a reference to the function in the module and the variable bindings. ?For example: >> >> -module(for_example). >> -export([example/4]). >> >> example(X, Y, Z, P) -> >> ? ? ? ?F = fun(W) -> (X + Y * Z) / W end, >> ? ? ? ?P ! F. >> >> When example(1, 2, 3, Pid) is called, a new fun is not created (F(W) -> (1 + 2 * 3) / W end) and sent to Pid. ?Instead, what's sent to Pid is a reference to the function definition in module for_example with the bindings X = 1, Y = 2, Z = 3. >> >> If Pid is on another node with a different version of the for_example module, then it will fail. ?Not sure exactly what will happen, whether the node will crash, the process crash, or just the message get discarded, but you could probably experiment to figure it out. >> >> Someone correct me if I am wrong. ?This is partly based on memory and partly based on how I figure it would be implemented. >> >> Cheers, >> >> DBM >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Roberto Aloi --- Website: http://roberto-aloi.com Twitter: @robertoaloi From janburse@REDACTED Thu Mar 29 11:31:35 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 29 Mar 2012 11:31:35 +0200 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: References: Message-ID: <4F742BF7.2010004@fastmail.fm> Richard O'Keefe schrieb: > fs = new FileReader(); > fs.SetFileName("foo/bar.txt"); > fs.SetSharing(Sharing.NONE); > ln = fs.ReadLine(); > ... The above programming pattern has a couple of advantages. It is not suitable to keep the original file reader semantics, where the constructor would for example fail when no file is found, and thus no object reference would be returned in the first place. But otherwise it is very suitable for large scale software development. It allows some neat object evolution. The basic idea is to start with a small set of classes and a small set of attributes. In the above example this would for example mean to only have: class: FileReader attribute: FileName Then over the time it can happen that it becomes necessary to support additional attributes, i.e. a refinement of the classes can happen. In the above example this could for example mean to then have: class: FileReader attribute: FileName attribute: Sharing The idea is then that the legacy code, or any code that doesn't need the refinement, would simply not set the sharing attribute. If no setter is called then the object will have the default value that the constructor assigns to it. So that legacy code or any code that doesn't need the refinement will not break. The above technique is especially useful when writing concurrent programs. Classes will then be used to represent moment-intervals. The basic idea is to have represented tasks and their parameters by objects. The programming pattern is then: Task task = new Task(); task.setParameter1(); task.setParameter2(); task.performTask(); The performTask() method need not be synchronized, since the object task is created on the fly by the thread that needs the task. So it will not be invoked by other threads. In the inner working of the implementation of the perform task method the code will then have easy access to the parameter 1 and parameter 2. The defaulting can then again be applied. For example if the task is "buy milk", I can default the shop location. So if I do: BuyMilk buyMilk = new BuyMilk(); buyMilk.performBuyMilk(); This will go to the default shop location, i.e. the farmer around the corner. If I want that the milk is bought from a non default location, I can use the corresponding parameter: BuyMilk buyMilk = new BuyMilk(); buyMilk.setShop("shopping mal"); buyMilk.performBuyMilk(); Interestingly the programming pattern can also be used to compose tasks. This has been favored Peter Coad's 'modeling in color' technique. See for example: http://www.step-10.com/SoftwareDesign/ModellingInColour/Moment-Interval.html Unfortunately the above approach is not found in large shops. For example where MDA is used in bigger companies the approach is effectively to have services of the form: serviceName( in parameter 1 .. in parameter 2 out parameter 1 .. out parameter 2) The MDA generated code, in one incarnation that I saw it, is a class for the in/out parameter block and a service entry point which takes this parameter argument, since Java does not have out parameters: class ServiceNameParameter { in parameter 1 .. in parameter 2 out parameter 1 .. out parameter 2) } serviceName(ServiceNameParameter p); Setter are then used to set the out parameters (sic!). The resulting code is not only bloathed by the many parameter blocks, but also the defaulting is done via service names. Namely we might then end up with: buyMilkAtFarmer(...) buyMilkFromShop(...) Erlang would be found somewhere in the MDA camp. Where a functional service view is adopted. The out parameters would not be passed as arguments, but returned in an agregate result, right? With the same disadvantage of service verb name proliferation. I think one has a choice. Having attributes even in services is as old as the internet. Look for example at the SMTP protocol, it is nothing else than a setter sequence before a perform instruction happens. http://de.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol In Erlang I would do it properly functional with the record type. So the Java programming pattern can be modelled as follows: - record(buymilk, {shop='next farmer'}} BuyMilk = #buymilk{}, pid ! BuyMilk BuyMilk1 = #buymilk{}, BuyMilk2 = BuyMilk1#buymilk{shop='shopping mal'} pid ! BuyMilk2 But I guess more elaborate solutions are also possible in Erlang. Bye From janburse@REDACTED Thu Mar 29 11:45:26 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 29 Mar 2012 11:45:26 +0200 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: <4F742BF7.2010004@fastmail.fm> References: <4F742BF7.2010004@fastmail.fm> Message-ID: <4F742F36.50508@fastmail.fm> Jan Burse schrieb: > In Erlang I would do it properly functional > with the record type. So the Java programming pattern > can be modelled as follows: > > - record(buymilk, {shop='next farmer'}} > > BuyMilk = #buymilk{}, > pid ! BuyMilk > > BuyMilk1 = #buymilk{}, > BuyMilk2 = BuyMilk1#buymilk{shop='shopping mal'} > pid ! BuyMilk2 > > But I guess more elaborate solutions are also > possible in Erlang. > > Bye It would be fun if Erlang systems would have means to optimize the above. Not sure what happens under the hood. But from SMTP it can easily be seen that the set attribute don't eat any bandwidth, at least during the process communication. Bye From janburse@REDACTED Thu Mar 29 11:47:12 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 29 Mar 2012 11:47:12 +0200 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: <4F742F36.50508@fastmail.fm> References: <4F742BF7.2010004@fastmail.fm> <4F742F36.50508@fastmail.fm> Message-ID: <4F742FA0.8050802@fastmail.fm> Jan Burse schrieb: > But from SMTP it can easily be seen that the > set attribute don't eat any bandwidth, at > least during the process communication. Corr.: But from SMTP it can easily be seen that the missing set attributes don't eat any bandwidth, at least during the process communication. From michael.eugene.turner@REDACTED Thu Mar 29 12:15:17 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 29 Mar 2012 19:15:17 +0900 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: References: Message-ID: > It doesn't frighten me that [such programmers] _exist_, > it frightens me that they are _programming_. LOL. I don't have that much against create-set-call, provided that (1) What I get from creation is either (1a) ... something in a consistent and usable state, or (1b) ... something that will throw a useful exception in informative ways if it can't (yet) be used ... and .... (2) Expression of the "set" part is reasonably concise and clear, and always yields a situation like (1ab) above. But it does seem to lend itself to small-minded purism. Of which we've had plenty. And from Microsoft, too. (Remember Hungarian Notation?) I mean, how about this? P = new Point P.set_x(0) P.set_y(0) It's just dumb to *require* doing it that way. The test for me isn't purity, it's idiomaticity. I'm not saying Point wouldn't have setters and getters. Just that I shouldn't be limited to those. On the other side of the debate: I once worked in a C++ shop with an API that required up to 22 parameters for construction of certain objects. "It's not so bad," a long-timer tried to assure me, "You just copy-paste a call from someone else's code then make the changes you need ...." I almost hit the ceiling. -michael turner On Thu, Mar 29, 2012 at 4:32 PM, Richard O'Keefe wrote: > I'm to the Psychology of Programming Interest Group mailing list. > In a discussion of naming, reference was made to a body of work > containing > ? ? ? ?Usability Implications of Requiring Parameters > ? ? ? ?in Objects' Constructors > ? ? ? ?by Jeffrey Stylos of CMU > ? ? ? ?and Steven Clarke of Microsoft > at ? ? ?http://www.cs.cmu.edu/~NatProg/papers/Stylos2007CreateSetCall.pdf > > The abstract reads > ? ? ? ?The usability of APIs is increasingly important to programmer > ? ? ? ?productivity. ?Based on experience with usability studies of > ? ? ? ?specific APIs, techniques were explored for studying the usability > ? ? ? ?of design choices common to many APIs. ?A comparative study was > ? ? ? ?performed to assess how professional programmers use APIs with > ? ? ? ?required parameters in objects? constructors as opposed to > ? ? ? ?parameterless ?default? constructors. ?It was hypothesized that > ? ? ? ?required parameters would create more usable and self-documenting > ? ? ? ?APIs by guiding programmers toward the correct use of objects and > ? ? ? ?preventing errors. ?However, in the study, it was found that, > ? ? ? ?contrary to expectations, programmers strongly preferred and > ? ? ? ?were more effective with APIs that did not require constructor > ? ? ? ?parameters. ?Participants? behavior was analyzed using the > ? ? ? ?cognitive dimensions framework, and revealing that required > ? ? ? ?constructor parameters interfere with common learning strategies, > ? ? ? ?causing undesirable premature commitment. > > The study was done in 2005. > We're talking about the difference between > > ? ? ? ?fs = new FileReader("foo/bar.txt", Sharing.NONE); > ? ? ? ?ln = fs.ReadLine(); > ? ? ? ?... > > and > > ? ? ? ?fs = new FileReader(); > ? ? ? ?fs.SetFileName("foo/bar.txt"); > ? ? ? ?fs.SetSharing(Sharing.NONE); > ? ? ? ?ln = fs.ReadLine(); > ? ? ? ?... > > I think this is worth bringing up here because if functional programming > is about anything at all, it's certainly about NOT setting up a value one > field at a time! > > Their sample was carefully chosen to include three kinds of programmers: > > ?* ? ? OPPORTUNISITC programmers are more concerned with productivity > ? ? ? ?than control or understanding. ?For these programmers objects > ? ? ? ?that required constructor parameters were unfamiliar and > ? ? ? ?unexpected, and even after repeated exposure these programmers > ? ? ? ?had difficulty with these objects. > > ? ? ? ?That is, they just didn't "get" the idea of constructors having > ? ? ? ?parameters. > > ?* ? ? PRAGMATIC programmers balance productivity with control and > ? ? ? ?understanding. ?These programmers also did not expect objects > ? ? ? ?with required constructors, and while pragmatic programmers > ? ? ? ?were more effective than opportunistic programmers at using > ? ? ? ?these objects, the objects still provided a minor stumbling > ? ? ? ?block and these programmers preferred the flexibility offered > ? ? ? ?by objects that used the create-set-call pattern. > > ? ? ? ?Remember, this was all about .NET. ?Failing to expect > ? ? ? ?constructors with parameters in C# is like failing to expect > ? ? ? ?assignment statements in C. > > ?* ? ? SYSTEMATIC programmers program defensively and these are the > ? ? ? ?programmers for whom low-level APIs are targeted. ?These programmers > ? ? ? ?were effective at using all of the objects; however, they preferred > ? ? ? ?create-set-call because of the finer granularity of control it > ? ? ? ?offered by allowing objects to be initialized one piece at a time. > > The purpose of the study was to provide guidelines for API designers at > Microsoft: ?apparently they now recommend create-set-call. ?Remember, > that's the idea where you create an object without saying *anything* about > what you want it to be and then successively kick it into shape. > > They later say > > ? ? ? ?[Systematic programmers] want not just to get their code working, > ? ? ? ?but to understand why it works, what assumptions it makes > ? ? ? ?and when it might fail. ?They are rare, and prefer languages that > ? ? ? ?give them the most detailed control such as C++, C and assembly. > > I'd like to think of myself as a systematic programmer. ?I certainly like > to understand all those things. ?But if I preferred assembly I would not > be writing in the Erlang mailing list! ?The basic conclusion seems to be > that you should not design APIs for such rare animals. > > I made what I thought were three obvious points: > > (1) There is a confounding factor in the study: ?the create-set-call style > ? ?lets you *name* the information going into an object, while at that > ? ?time the full-initialisation style did not. ?There are constructors > ? ?for System.IO.FileStream with six arguments; maybe more. ?It seemed at > ? ?least *possible* that if the subjects had been given a third choice, > ? ?constructors with *named* parameters, they might have preferred that. > ? ?Because the study didn't include such a choice, we certainly cannot > ? ?use it to argue *against* that style. > > (2) C# 4.0 has named (and optional) parameters, so the study is no longer > ? ?adequate to tell us about good API design in C#. > > (3) If there are programmers out there who *don't* care much about > ? ?understanding what they are doing, I certainly don't want them writing > ? ?anything that might in any way affect me or anyone I care about. > ? ?If they just don't "get" constructors with parameters, that's really > ? ?scary. > > A lengthy discussion has followed in which I've felt rather lonely. > I'm being beaten about the head for not noticing things in the study > that I did notice, and for being rather elitist. ?One quote: > > ? ? ? ?We don?t have the luxury of dismissing these types of programmers. > ? ? ? ?While it might strike you with terror that these programmers exist, > > It doesn't frighten me that they _exist_, > it frightens me that they are _programming_. > > ? ? ? ?they are successfully building applications in many different domains. > ? ? ? ?They may work differently to you and many other programmers but that > ? ? ? ?doesn?t necessarily mean that the code they create is worthless. > ? ? ? ?Within the Visual Studio team at Microsoft we?ve devoted efforts to > ? ? ? ?attempting to make them successful by adapting to their workstyles > ? ? ? ?when appropriate. > > I find myself wondering just how "successful" their applications really are. > Oh no! ?I hope they're not writing climate modelling code! ?That would > explain so much... ?(Yes, I've looked at the code in the ClimateGate dump.) > > Frankly, I've turned here for a bit of comfort. ?If anyone likes completely > initialising things rather than smacking one field at a time, surely I will > find such people here. > > Am I wrong? ?Would *you* be happy opening a file in C by doing > > ? ? ? ?FILE *f = malloc(sizeof f); > ? ? ? ?set_title(f, "foo/bar.txt"); > ? ? ? ?set_access(f, "r"); > ? ? ? ?gets(f, buffer); > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From gustav.simonsson@REDACTED Thu Mar 29 12:33:20 2012 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Thu, 29 Mar 2012 11:33:20 +0100 (BST) Subject: [erlang-questions] Erlang jobs In-Reply-To: Message-ID: <525f23d2-56ff-4acf-8a15-682eb47b6d09@knuth> A fun and challenging way to become more comfortable with the language itself (and the ubiquitous lists module) is to try out the euler problems in Erlang: http://projecteuler.net/ Then try to either find a open-source project or create one yourself that makes use of multiple processes executing in parallel, possibly on multiple nodes as well :) That should expose you to the gen_* behaviours, supervisors and the distribution. // Gustav Sent from my PC ----- Original Message ----- > From: "Yves S. Garret" > To: "Max Lapshin" > Cc: erlang-questions@REDACTED, "Jesse Gumm" > Sent: Wednesday, 28 March, 2012 11:05:20 PM > Subject: Re: [erlang-questions] Erlang jobs > > > Since I could still use a little bit of experience, how would you > guys recommend I gain some experience working in Erlang? I could go > through my book and try everything that they have in it. An > open-source projects? A list of problems I could solve just for the > heck of it? > > > On Wed, Mar 28, 2012 at 4:15 PM, Max Lapshin < max.lapshin@REDACTED > > wrote: > > > I think, that when we speak about "erlang programming" there is no > plain relations "look for a job on monster.com ". > > If there is a company that has such business requirements that can be > solved with erlang, they need engineer of such qualification than can > solve this problem in any language. But Erlang will be the best way > to > solve it. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From fhunleth@REDACTED Thu Mar 29 14:43:02 2012 From: fhunleth@REDACTED (Frank Hunleth) Date: Thu, 29 Mar 2012 08:43:02 -0400 Subject: [erlang-questions] erl using 99% CPU while idle In-Reply-To: <4F72C56B.2050207@erlang.org> References: <4F72C56B.2050207@erlang.org> Message-ID: Hi Rickard, On Wed, Mar 28, 2012 at 4:01 AM, Rickard Green wrote: > The following fix for this has successfully passed one daily build, and will > probably appear as it is in R15B01. As you noted, this bug does not appear > unless you disable thread support. This bug was first introduced in R15B. > > diff --git a/erts/emulator/sys/common/erl_poll.c Thanks for looking into the problem and posting the patch. I tried it out, and it works for me too. Thanks, Frank From emmiller@REDACTED Thu Mar 29 15:50:04 2012 From: emmiller@REDACTED (Evan Miller) Date: Thu, 29 Mar 2012 08:50:04 -0500 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: References: Message-ID: Programmer "types" aside, ease of writing code is not the only desideratum. Something missing from your discussion is program readability and maintainability. Requiring parameters in the constructor packs much more information per line of code than the create-set-call convention, and so I am able to read it more quickly. Initialize-everything also reduces the potential for bugs, because with create-set-call there become N! ways to initialize an object, instead of just 1. It is short-sighted to consider only programmers' stated preferences when designing an API. I am sure programmers in the study would also have preferred not to write code comments or unit tests if given the option to leave early for lunch. I am reminded of a quotation that appeared in my high school's writing center: "What is written without effort is in general read without pleasure." --Samuel Johnson Evan On Thu, 29 Mar 2012 02:32:33 -0500, Richard O'Keefe wrote: > I'm to the Psychology of Programming Interest Group mailing list. > In a discussion of naming, reference was made to a body of work > containing > Usability Implications of Requiring Parameters > in Objects' Constructors > by Jeffrey Stylos of CMU > and Steven Clarke of Microsoft > at http://www.cs.cmu.edu/~NatProg/papers/Stylos2007CreateSetCall.pdf > > The abstract reads > The usability of APIs is increasingly important to programmer > productivity. Based on experience with usability studies of > specific APIs, techniques were explored for studying the usability > of design choices common to many APIs. A comparative study was > performed to assess how professional programmers use APIs with > required parameters in objects? constructors as opposed to > parameterless ?default? constructors. It was hypothesized that > required parameters would create more usable and self-documenting > APIs by guiding programmers toward the correct use of objects and > preventing errors. However, in the study, it was found that, > contrary to expectations, programmers strongly preferred and > were more effective with APIs that did not require constructor > parameters. Participants? behavior was analyzed using the > cognitive dimensions framework, and revealing that required > constructor parameters interfere with common learning strategies, > causing undesirable premature commitment. > > The study was done in 2005. > We're talking about the difference between > > fs = new FileReader("foo/bar.txt", Sharing.NONE); > ln = fs.ReadLine(); > ... > > and > > fs = new FileReader(); > fs.SetFileName("foo/bar.txt"); > fs.SetSharing(Sharing.NONE); > ln = fs.ReadLine(); > ... > > I think this is worth bringing up here because if functional programming > is about anything at all, it's certainly about NOT setting up a value one > field at a time! > > Their sample was carefully chosen to include three kinds of programmers: > > * OPPORTUNISITC programmers are more concerned with productivity > than control or understanding. For these programmers objects > that required constructor parameters were unfamiliar and > unexpected, and even after repeated exposure these programmers > had difficulty with these objects. > > That is, they just didn't "get" the idea of constructors having > parameters. > > * PRAGMATIC programmers balance productivity with control and > understanding. These programmers also did not expect objects > with required constructors, and while pragmatic programmers > were more effective than opportunistic programmers at using > these objects, the objects still provided a minor stumbling > block and these programmers preferred the flexibility offered > by objects that used the create-set-call pattern. > > Remember, this was all about .NET. Failing to expect > constructors with parameters in C# is like failing to expect > assignment statements in C. > > * SYSTEMATIC programmers program defensively and these are the > programmers for whom low-level APIs are targeted. These programmers > were effective at using all of the objects; however, they preferred > create-set-call because of the finer granularity of control it > offered by allowing objects to be initialized one piece at a time. > > The purpose of the study was to provide guidelines for API designers at > Microsoft: apparently they now recommend create-set-call. Remember, > that's the idea where you create an object without saying *anything* > about > what you want it to be and then successively kick it into shape. > > They later say > > [Systematic programmers] want not just to get their code working, > but to understand why it works, what assumptions it makes > and when it might fail. They are rare, and prefer languages that > give them the most detailed control such as C++, C and assembly. > > I'd like to think of myself as a systematic programmer. I certainly like > to understand all those things. But if I preferred assembly I would not > be writing in the Erlang mailing list! The basic conclusion seems to be > that you should not design APIs for such rare animals. > > I made what I thought were three obvious points: > > (1) There is a confounding factor in the study: the create-set-call > style > lets you *name* the information going into an object, while at that > time the full-initialisation style did not. There are constructors > for System.IO.FileStream with six arguments; maybe more. It seemed > at > least *possible* that if the subjects had been given a third choice, > constructors with *named* parameters, they might have preferred that. > Because the study didn't include such a choice, we certainly cannot > use it to argue *against* that style. > > (2) C# 4.0 has named (and optional) parameters, so the study is no longer > adequate to tell us about good API design in C#. > > (3) If there are programmers out there who *don't* care much about > understanding what they are doing, I certainly don't want them > writing > anything that might in any way affect me or anyone I care about. > If they just don't "get" constructors with parameters, that's really > scary. > > A lengthy discussion has followed in which I've felt rather lonely. > I'm being beaten about the head for not noticing things in the study > that I did notice, and for being rather elitist. One quote: > > We don?t have the luxury of dismissing these types of programmers. > While it might strike you with terror that these programmers exist, > > It doesn't frighten me that they _exist_, > it frightens me that they are _programming_. > > they are successfully building applications in many different domains. > They may work differently to you and many other programmers but that > doesn?t necessarily mean that the code they create is worthless. > Within the Visual Studio team at Microsoft we?ve devoted efforts to > attempting to make them successful by adapting to their workstyles > when appropriate. > > I find myself wondering just how "successful" their applications really > are. > Oh no! I hope they're not writing climate modelling code! That would > explain so much... (Yes, I've looked at the code in the ClimateGate > dump.) > > Frankly, I've turned here for a bit of comfort. If anyone likes > completely > initialising things rather than smacking one field at a time, surely I > will > find such people here. > > Am I wrong? Would *you* be happy opening a file in C by doing > > FILE *f = malloc(sizeof f); > set_title(f, "foo/bar.txt"); > set_access(f, "r"); > gets(f, buffer); > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g9414002.pccu.edu.tw@REDACTED Thu Mar 29 16:18:12 2012 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Thu, 29 Mar 2012 22:18:12 +0800 Subject: [erlang-questions] How do funs work ? In-Reply-To: References: Message-ID: On Wed, Mar 28, 2012 at 4:51 PM, Yoshihiro Tanaka wrote: > Hello, > > I have a function which returns fun as below: > > make_fun(_, [], F) -> F; > make_fun(Operator, [Operand1, Operand2|Operands], F) -> > F2 = case Operator of > plus -> fun() -> Operand1 + Operand2 + F() end; > minus -> fun() -> Operand1 - Operand2 + F() end > end, > make_fun(Operator, Operands, F2). > > When I call it as: > F1 = make_fun(plus, lists:seq(1,10), fun() -> 0 end). > > Is F1 same as F2 below ? > F2 = fun() -> 1 + 2 + > fun() -> 3 + 4 + > fun() -> 5 + 6 + > fun() -> 7 + 8 + > fun() -> 9 + 10 + > fun() -> 0 end() > end() > end() > end() > end() > end. > No, F1 is not the same as F2. A function F is a function definition, then F() is a function application. The F1 shell be expand as (fun() -> 9 + 10 + (fun() -> 7 + 8 + (fun() -> 5 + 6 + (fun() -> 3 + 4 + (fun() -> 1 + 2 + (fun() -> 0 end)() end)() end)() end)() end)() end) All F() evaluates itself, but F doesn't. Functions applied can be reduced, and F1 is reduced as a form (fun() -> 9 + 10 + (7 + 8 + (5 + 6 + (3 + 4 + (1 + 2 + 0) ) ) ) end) Note that it cannot be a function with any binary operator + accompanied by a function definition. For example, try to write a program with you make_fun/3 definition, and make the F1 function in Erlang shell. And give following instructions: F3 = fun()-> 1 + 2 + F1 end, F3(). You will get a bad argument error in arithmetic expression. Obviously in your make_fun/3, every time it make a fun() -> end expression, it evaluates a preceding function and puts the result as an operand of +, and the preceding function is not a function definition any more. > Also, is there any difference between funs that are defined at runtime > and funs that are defined at compile time in terms of how they are > executed ? > > Thank you > Yoshihiro > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best Regards. --- Y-H. H. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave@REDACTED Thu Mar 29 16:27:57 2012 From: dave@REDACTED (David Goehrig) Date: Thu, 29 Mar 2012 10:27:57 -0400 Subject: [erlang-questions] Erlang jobs In-Reply-To: References: Message-ID: <541B3180-27E3-4D68-8502-9460CC7DD34F@nexttolast.com> You don't get a job programming Erlang. You get a job solving problems and just happen to implement the solutions in Erlang. :) That said I'm taking resumes. Dave -=-=- dave@REDACTED -=-=- On Mar 28, 2012, at 2:52 PM, "Yves S. Garret" wrote: > Hi all, I'm just curious if there are any Erlang employment opportunities out there. I have a job at the moment, but I'm not optimistic as to how much I can grow here. I do PHP web-development and would like to do more non-web-development, but wouldn't balk at working with Yaws and Erlang (it would be fun in its own way). > > What are your thoughts and inputs? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From straightflush@REDACTED Thu Mar 29 18:32:01 2012 From: straightflush@REDACTED (AD) Date: Thu, 29 Mar 2012 12:32:01 -0400 Subject: [erlang-questions] Erlang jobs In-Reply-To: <541B3180-27E3-4D68-8502-9460CC7DD34F@nexttolast.com> References: <541B3180-27E3-4D68-8502-9460CC7DD34F@nexttolast.com> Message-ID: love that answer. On Thu, Mar 29, 2012 at 10:27 AM, David Goehrig wrote: > You don't get a job programming Erlang. You get a job solving problems and > just happen to implement the solutions in Erlang. :) > > That said I'm taking resumes. > > Dave > > -=-=- dave@REDACTED -=-=- > > On Mar 28, 2012, at 2:52 PM, "Yves S. Garret" > wrote: > > > Hi all, I'm just curious if there are any Erlang employment > opportunities out there. I have a job at the moment, but I'm not > optimistic as to how much I can grow here. I do PHP web-development and > would like to do more non-web-development, but wouldn't balk at working > with Yaws and Erlang (it would be fun in its own way). > > > > What are your thoughts and inputs? > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hirotnkg@REDACTED Thu Mar 29 18:55:20 2012 From: hirotnkg@REDACTED (Yoshihiro Tanaka) Date: Thu, 29 Mar 2012 09:55:20 -0700 Subject: [erlang-questions] How do funs work ? In-Reply-To: References: Message-ID: 2012/3/29 ??? (Yau-Hsien Huang) : > On Wed, Mar 28, 2012 at 4:51 PM, Yoshihiro Tanaka > wrote: >> >> Hello, >> >> I have a function which returns fun as below: >> >> make_fun(_, [], F) -> F; >> make_fun(Operator, [Operand1, Operand2|Operands], F) -> >> ?F2 = case Operator of >> ? ? ? ? plus ?-> fun() -> Operand1 + Operand2 + F() end; >> ? ? ? ? minus -> fun() -> Operand1 - Operand2 + F() end >> ? ? ? end, >> ?make_fun(Operator, Operands, F2). >> >> When I call it as: >> F1 = make_fun(plus, lists:seq(1,10), fun() -> 0 end). >> >> Is F1 same as F2 below ? >> F2 = fun() -> 1 + 2 + >> ? ? ? fun() -> 3 + 4 + >> ? ? ? ? fun() -> 5 + 6 + >> ? ? ? ? ? fun() -> 7 + 8 + >> ? ? ? ? ? ? fun() -> 9 + 10 + >> ? ? ? ? ? ? ? fun() -> 0 end() >> ? ? ? ? ? ? end() >> ? ? ? ? ? end() >> ? ? ? ? end() >> ? ? ? end() >> ? ? end. > > > No, F1 is not the same as F2.?A function F is a function definition, then > F() is a function application. The?F1 shell be expand as > > (fun() -> > ? ? 9 + 10 + > ? ?(fun() -> > ? ? ? ? 7 + 8 + > ? ? ? ?(fun() -> > ? ? ? ? ? ?5 + 6 + > ? ? ? ? ? (fun() -> > ? ? ? ? ? ? ? 3 + 4 + > ? ? ? ? ? ? ?(fun() -> > ? ? ? ? ? ? ? ? 1 + 2 + (fun() -> 0 end)() > ? ? ? ? ? ? ? end)() > ? ? ? ? ? end)() > ? ? ? end)() > ? end)() > end) > > All F() evaluates itself, but F doesn't. Functions applied?can be reduced, > and F1 is reduced as a form > > (fun() -> > ? ? 9 + 10 + > ? ? ?(7 + 8 + > ? ? ? ? (5 + 6 + > ? ? ? ? ? ?(3 + 4 + > ? ? ? ? ? ? ? (1 + 2 + 0) > ? ? ?) ?) ?) > end) > > Note that it cannot be a function with any binary operator + accompanied > by a function definition. For example, try to write a program with you > make_fun/3 definition, and make the F1 function in Erlang shell. And > give following instructions: > > ? ? ? ? ? F3 = fun()-> 1 + 2 + F1 end, F3(). > > You will get a bad argument error in arithmetic expression. Obviously > in your make_fun/3, every time it make a fun() -> end expression, > it evaluates a preceding function and puts the result as an operand of +, > and the preceding function is not a function definition any more. Hi Yau-Hsien, It seems like not. For example, below example indicates F is not evaluated when constructing F2. 1> F = fun() -> blah end. #Fun 2> F2 = fun() -> 1 + 2 + F() end. #Fun 3> F2(). ** exception error: bad argument in an arithmetic expression in operator +/2 called as 3 + blah I also checked core file of sample code, using 'to_core' option, and the code: 18 fun() -> 9 + 10 + 19 fun() -> 0 end() 20 end() is translated to: %% Line 18 ( fun () -> let <_cor0> = %% Line 19 ( fun () -> 0 -| [{'id',{2,100470014,'-make_fun2/0-fun-0-'}}] ) in let <_cor1> = %% Line 19 apply _cor0 () in call 'erlang':'+' (19, _cor1) -| [{'id',{3,119737161,'-make_fun2/0-fun-1-'}}] ) It defines <_cor0>, as fun() -> 0 end, then doing 'apply _cor0'. So It's calling fun _cor0. Yoshihiro > >> >> Also, is there any difference between funs that are defined at runtime >> and funs that are defined at compile time in terms of how they are >> executed ? >> >> Thank you >> Yoshihiro >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > > Best Regards. > > --- Y-H. H. > > From jared.kofron@REDACTED Thu Mar 29 20:34:16 2012 From: jared.kofron@REDACTED (Jared Kofron) Date: Thu, 29 Mar 2012 11:34:16 -0700 Subject: [erlang-questions] systools problem? Message-ID: <6D456DF1-BFE3-403B-AB44-A08CBF0AE1FC@gmail.com> When I do ./rebar generate-upgrade previous_release=v0p1, I get "ERROR: Systools [systools:make_relup/4] aborted with", and then a huge list of errors of the form: {error_reading, {mochiweb, {{bad_param, start_phases}, {application, mochiweb, [{description, "MochiMedia Web Server"}, {vsn,"2.3.1"}, {id,[]}, {modules, [mochifmt, mochifmt_records, mochifmt_std, mochiglobal, mochihex, mochijson, mochijson2, mochilists, mochilogfile2, mochinum, mochitemp, mochiutf8, mochiweb, mochiweb_acceptor, mochiweb_charref, mochiweb_cookies, mochiweb_cover, mochiweb_echo, mochiweb_headers, mochiweb_html, mochiweb_http, mochiweb_io, mochiweb_mime, mochiweb_multipart, mochiweb_request, mochiweb_request_tests, mochiweb_response, mochiweb_socket, mochiweb_socket_server, mochiweb_util, reloader]}, {registered,[]}, {applications, [kernel,stdlib, crypto,inets, ssl,xmerl, compiler, syntax_tools]}, {included_applications, []}, {env,[]}, {start_phases, undefined}, {maxT,infinity}, {maxP, infinity}]}}}} Digging in a little bit it seems like systools is expecting an empty list instead of 'undefined' for start_phases. Has anybody else run in to this? From anders.nygren@REDACTED Thu Mar 29 20:48:07 2012 From: anders.nygren@REDACTED (Anders Nygren) Date: Thu, 29 Mar 2012 12:48:07 -0600 Subject: [erlang-questions] systools problem? In-Reply-To: <6D456DF1-BFE3-403B-AB44-A08CBF0AE1FC@gmail.com> References: <6D456DF1-BFE3-403B-AB44-A08CBF0AE1FC@gmail.com> Message-ID: Yes, this was discussed some time ago, check the archives for details. /Anders On Thu, Mar 29, 2012 at 12:34 PM, Jared Kofron wrote: > When I do ./rebar generate-upgrade previous_release=v0p1, I get "ERROR: Systools [systools:make_relup/4] aborted with", and then a huge > list of errors of the form: > > {error_reading, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{mochiweb, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {{bad_param, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? start_phases}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{application, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? mochiweb, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [{description, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "MochiMedia Web Server"}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{vsn,"2.3.1"}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{id,[]}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{modules, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [mochifmt, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochifmt_records, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochifmt_std, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiglobal, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochihex, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochijson, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochijson2, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochilists, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochilogfile2, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochinum, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochitemp, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiutf8, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_acceptor, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_charref, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_cookies, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_cover, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_echo, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_headers, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_html, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_http, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_io, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_mime, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_multipart, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_request, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_request_tests, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_response, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_socket, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_socket_server, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?mochiweb_util, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?reloader]}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{registered,[]}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{applications, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [kernel,stdlib, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?crypto,inets, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ssl,xmerl, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?compiler, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?syntax_tools]}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{included_applications, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? []}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{env,[]}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{start_phases, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{maxT,infinity}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{maxP, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? infinity}]}}}} > > Digging in a little bit it seems like systools is expecting an empty list instead of 'undefined' for start_phases. ?Has anybody else run in to this? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andy.richards.iit@REDACTED Thu Mar 29 22:54:09 2012 From: andy.richards.iit@REDACTED (Andy Richards) Date: Thu, 29 Mar 2012 21:54:09 +0100 Subject: [erlang-questions] systools problem? In-Reply-To: <6D456DF1-BFE3-403B-AB44-A08CBF0AE1FC@gmail.com> References: <6D456DF1-BFE3-403B-AB44-A08CBF0AE1FC@gmail.com> Message-ID: <995833CF-23F7-40D1-B8F2-5B8940EDBE3B@googlemail.com> Hi Jared, yes I ran into this problem. There is a long old thread about it here.. http://erlang.org/pipermail/erlang-patches/2012-January/002563.html I have a shell script from which I call rebar to generate my upgrades and releases and from this script I grep the .app files and replace the undefined tuple with a empty list which solved my problem in a simple application when generating upgrades. I'm not sure if this is a suitable work a round for a more complex application. Andy. Sent from my iPad On 29 Mar 2012, at 19:34, Jared Kofron wrote: > When I do ./rebar generate-upgrade previous_release=v0p1, I get "ERROR: Systools [systools:make_relup/4] aborted with", and then a huge > list of errors of the form: > > {error_reading, > {mochiweb, > {{bad_param, > start_phases}, > {application, > mochiweb, > [{description, > "MochiMedia Web Server"}, > {vsn,"2.3.1"}, > {id,[]}, > {modules, > [mochifmt, > mochifmt_records, > mochifmt_std, > mochiglobal, > mochihex, > mochijson, > mochijson2, > mochilists, > mochilogfile2, > mochinum, > mochitemp, > mochiutf8, > mochiweb, > mochiweb_acceptor, > mochiweb_charref, > mochiweb_cookies, > mochiweb_cover, > mochiweb_echo, > mochiweb_headers, > mochiweb_html, > mochiweb_http, > mochiweb_io, > mochiweb_mime, > mochiweb_multipart, > mochiweb_request, > mochiweb_request_tests, > mochiweb_response, > mochiweb_socket, > mochiweb_socket_server, > mochiweb_util, > reloader]}, > {registered,[]}, > {applications, > [kernel,stdlib, > crypto,inets, > ssl,xmerl, > compiler, > syntax_tools]}, > {included_applications, > []}, > {env,[]}, > {start_phases, > undefined}, > {maxT,infinity}, > {maxP, > infinity}]}}}} > > Digging in a little bit it seems like systools is expecting an empty list instead of 'undefined' for start_phases. Has anybody else run in to this? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bourinov@REDACTED Thu Mar 29 22:56:49 2012 From: bourinov@REDACTED (Max Bourinov) Date: Fri, 30 Mar 2012 00:56:49 +0400 Subject: [erlang-questions] Erlang jobs In-Reply-To: References: <541B3180-27E3-4D68-8502-9460CC7DD34F@nexttolast.com> Message-ID: <4709741893507721794@unknownmsgid> +1 Good that Erlang is right for my needs. It is pleasure to work with Erlang! Sent from my iPhone On 29.03.2012, at 20:32, AD wrote: love that answer. On Thu, Mar 29, 2012 at 10:27 AM, David Goehrig wrote: > You don't get a job programming Erlang. You get a job solving problems and > just happen to implement the solutions in Erlang. :) > > That said I'm taking resumes. > > Dave > > -=-=- dave@REDACTED -=-=- > > On Mar 28, 2012, at 2:52 PM, "Yves S. Garret" > wrote: > > > Hi all, I'm just curious if there are any Erlang employment > opportunities out there. I have a job at the moment, but I'm not > optimistic as to how much I can grow here. I do PHP web-development and > would like to do more non-web-development, but wouldn't balk at working > with Yaws and Erlang (it would be fun in its own way). > > > > What are your thoughts and inputs? > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From yoursurrogategod@REDACTED Fri Mar 30 01:20:26 2012 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Thu, 29 Mar 2012 19:20:26 -0400 Subject: [erlang-questions] Erlang jobs In-Reply-To: <4709741893507721794@unknownmsgid> References: <541B3180-27E3-4D68-8502-9460CC7DD34F@nexttolast.com> <4709741893507721794@unknownmsgid> Message-ID: Hmm, good answer indeed :-) . But to be honest -- and I will go off the rails for a sec -- I love the Erlang language. The syntax is quirky, but of all the programming languages that I've tried and messed with, this is the only one that has grabbed my attention and that is Erlang. I just have fun making and breaking Erlang apps. The last time I've had this much fun was when I discovered VB .NET (hold the flaming, the process of writing the code was actually very pleasant). I don't have much time to do Erlang coding at the moment (although I am beginning to make more time for it), I just wake up and run to my PC when I know I'll be doing Erlang coding. Call me a fan boy, but it's fun hacking away code in Erlang :-) . On Thu, Mar 29, 2012 at 4:56 PM, Max Bourinov wrote: > +1 > > Good that Erlang is right for my needs. It is pleasure to work with Erlang! > > Sent from my iPhone > > On 29.03.2012, at 20:32, AD wrote: > > love that answer. > > On Thu, Mar 29, 2012 at 10:27 AM, David Goehrig wrote: > >> You don't get a job programming Erlang. You get a job solving problems >> and just happen to implement the solutions in Erlang. :) >> >> That said I'm taking resumes. >> >> Dave >> >> -=-=- dave@REDACTED -=-=- >> >> On Mar 28, 2012, at 2:52 PM, "Yves S. Garret" >> wrote: >> >> > Hi all, I'm just curious if there are any Erlang employment >> opportunities out there. I have a job at the moment, but I'm not >> optimistic as to how much I can grow here. I do PHP web-development and >> would like to do more non-web-development, but wouldn't balk at working >> with Yaws and Erlang (it would be fun in its own way). >> > >> > What are your thoughts and inputs? >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g9414002.pccu.edu.tw@REDACTED Fri Mar 30 02:15:55 2012 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Fri, 30 Mar 2012 08:15:55 +0800 Subject: [erlang-questions] Fwd: How do funs work ? In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: ??? (Yau-Hsien Huang) Date: 2012/3/30 Subject: Re: [erlang-questions] How do funs work ? To: Yoshihiro Tanaka 2012/3/30 Yoshihiro Tanaka > Hi Yau-Hsien, > > It seems like not. For example, below example indicates F is not > evaluated when constructing F2. > > 1> F = fun() -> blah end. > #Fun > 2> F2 = fun() -> 1 + 2 + F() end. > No. Here you type F() which means F is evaluated when F2 is evaluating. I said the word "evaluated" but "reduced." If you expand the symbol 'F' to the function, in syntax, it must be F2 = fun() -> 1 + 2 + (fun -> 0 end)() end. but F2 = fun() -> 1 + 2 + (fun() -> 0 end) end. Later one is depicted in your original letter, then I point the difference on parenthesis after. Maybe it's ill-wording, that it ought to be function application for symbol 'F()' and function-taking for symbol 'F'. #Fun > 3> F2(). > ** exception error: bad argument in an arithmetic expression > in operator +/2 > called as 3 + blah > > I also checked core file of sample code, using 'to_core' option, and the > code: > 18 fun() -> 9 + 10 + > 19 fun() -> 0 end() > 20 end() > The part of the core file seems as what I mentioned. :) > is translated to: > %% Line 18 > ( fun () -> > let <_cor0> = > %% Line 19 > ( fun () -> > 0 > -| [{'id',{2,100470014,'-make_fun2/0-fun-0-'}}] ) > in let <_cor1> = > %% Line 19 > apply _cor0 > () > in call 'erlang':'+' > (19, _cor1) > -| [{'id',{3,119737161,'-make_fun2/0-fun-1-'}}] ) > > It defines <_cor0>, as fun() -> 0 end, then doing 'apply _cor0'. So > It's calling fun _cor0. > > > Yoshihiro > > > > > > >> > >> Also, is there any difference between funs that are defined at runtime > >> and funs that are defined at compile time in terms of how they are > >> executed ? > >> > >> Thank you > >> Yoshihiro > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > -- > > > > Best Regards. > > > > --- Y-H. H. > > > > > -- Best Regards. --- Y-H. H. -- Best Regards. --- Y-H. H. -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Fri Mar 30 04:47:22 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Fri, 30 Mar 2012 11:47:22 +0900 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: References: Message-ID: "Requiring parameters in the constructor packs much more information per line of code than the create-set-call convention, and so I am able to read it more quickly." Yes, IF you know the order of parameters and their meanings in advance, OR you have some way to look those up almost instantly. True in some cases, not in all. You can read any piece of code very fast. The challenge is more often in *understanding* what you're reading. If the name of the class/type is suggestive, e.g., Point, then P = Point(0,1) might seem pretty obvious: x=0, y=1. But how about this? R = Rect(a,n0,r,v) Do the parameters refer to the *ranges* [a,n0] and [r,v]? Or to *points*: (a,n0) the lower left, and (r,v) the upper right? The following (notional) syntax for construction and setting instance variables would make it clear: R = Rect().lowerleft(a,n0).upperright(r,v) and it doesn't exclude other equivalent constructions for the same type, e.g. R = Rect().x_range(a,r).y_range(n0,v) " Initialize-everything also reduces the potential for bugs, because with create-set-call there become N! ways to initialize an object, instead of just 1." N! overstates the *likely* combinations. Order of initialization is typically idiomatic (e.g., people set x before y, usually). Explicit setting of parameters that ordinarily default to reasonable values will tend to follow other initializations that are almost always set explicitly. Create-all-at-once has its pitfalls too. I mentioned a job where a co-worker said that constructors with 22 parameters were OK because you could just copy a working constructor call from some other code, then make any mods you felt necessary. I immediately pointed out to him that you might be copy-pasting bugs -- bugs that were latent in the code you were copying, and that would manifest themselves much more perfidiously in what you were copying to. If create-set-call is the dominant style, I think people should at least enforce rules about states reached by setting certain values, and exceptions thrown if operations are performed prematurely, on any not-quite-yet-initialized object. I don't think there's any single right way. Readability and conciseness will always trade off in complex ways, depending on people's preconceptions. I usually prefer a mix, when I write code. -michael turner On Thu, Mar 29, 2012 at 10:50 PM, Evan Miller wrote: > Programmer "types" aside, ease of writing code is not the only desideratum. > Something missing from your discussion is program readability and > maintainability. Requiring parameters in the constructor packs much more > information per line of code than the create-set-call convention, and so I > am able to read it more quickly. Initialize-everything also reduces the > potential for bugs, because with create-set-call there become N! ways to > initialize an object, instead of just 1. > > It is short-sighted to consider only programmers' stated preferences when > designing an API. I am sure programmers in the study would also have > preferred not to write code comments or unit tests if given the option to > leave early for lunch. I am reminded of a quotation that appeared in my high > school's writing center: > > "What is written without effort is in general read without pleasure." > --Samuel Johnson > > > Evan > > On Thu, 29 Mar 2012 02:32:33 -0500, Richard O'Keefe > wrote: > >> I'm to the Psychology of Programming Interest Group mailing list. >> In a discussion of naming, reference was made to a body of work >> containing >> ? ? ? ?Usability Implications of Requiring Parameters >> ? ? ? ?in Objects' Constructors >> ? ? ? ?by Jeffrey Stylos of CMU >> ? ? ? ?and Steven Clarke of Microsoft >> at ? ? ?http://www.cs.cmu.edu/~NatProg/papers/Stylos2007CreateSetCall.pdf >> >> The abstract reads >> ? ? ? ?The usability of APIs is increasingly important to programmer >> ? ? ? ?productivity. ?Based on experience with usability studies of >> ? ? ? ?specific APIs, techniques were explored for studying the usability >> ? ? ? ?of design choices common to many APIs. ?A comparative study was >> ? ? ? ?performed to assess how professional programmers use APIs with >> ? ? ? ?required parameters in objects? constructors as opposed to >> ? ? ? ?parameterless ?default? constructors. ?It was hypothesized that >> ? ? ? ?required parameters would create more usable and self-documenting >> ? ? ? ?APIs by guiding programmers toward the correct use of objects and >> ? ? ? ?preventing errors. ?However, in the study, it was found that, >> ? ? ? ?contrary to expectations, programmers strongly preferred and >> ? ? ? ?were more effective with APIs that did not require constructor >> ? ? ? ?parameters. ?Participants? behavior was analyzed using the >> ? ? ? ?cognitive dimensions framework, and revealing that required >> ? ? ? ?constructor parameters interfere with common learning strategies, >> ? ? ? ?causing undesirable premature commitment. >> >> The study was done in 2005. >> We're talking about the difference between >> >> ? ? ? ?fs = new FileReader("foo/bar.txt", Sharing.NONE); >> ? ? ? ?ln = fs.ReadLine(); >> ? ? ? ?... >> >> and >> >> ? ? ? ?fs = new FileReader(); >> ? ? ? ?fs.SetFileName("foo/bar.txt"); >> ? ? ? ?fs.SetSharing(Sharing.NONE); >> ? ? ? ?ln = fs.ReadLine(); >> ? ? ? ?... >> >> I think this is worth bringing up here because if functional programming >> is about anything at all, it's certainly about NOT setting up a value one >> field at a time! >> >> Their sample was carefully chosen to include three kinds of programmers: >> >> ?* ? ? OPPORTUNISITC programmers are more concerned with productivity >> ? ? ? ?than control or understanding. ?For these programmers objects >> ? ? ? ?that required constructor parameters were unfamiliar and >> ? ? ? ?unexpected, and even after repeated exposure these programmers >> ? ? ? ?had difficulty with these objects. >> >> ? ? ? ?That is, they just didn't "get" the idea of constructors having >> ? ? ? ?parameters. >> >> ?* ? ? PRAGMATIC programmers balance productivity with control and >> ? ? ? ?understanding. ?These programmers also did not expect objects >> ? ? ? ?with required constructors, and while pragmatic programmers >> ? ? ? ?were more effective than opportunistic programmers at using >> ? ? ? ?these objects, the objects still provided a minor stumbling >> ? ? ? ?block and these programmers preferred the flexibility offered >> ? ? ? ?by objects that used the create-set-call pattern. >> >> ? ? ? ?Remember, this was all about .NET. ?Failing to expect >> ? ? ? ?constructors with parameters in C# is like failing to expect >> ? ? ? ?assignment statements in C. >> >> ?* ? ? SYSTEMATIC programmers program defensively and these are the >> ? ? ? ?programmers for whom low-level APIs are targeted. ?These >> programmers >> ? ? ? ?were effective at using all of the objects; however, they preferred >> ? ? ? ?create-set-call because of the finer granularity of control it >> ? ? ? ?offered by allowing objects to be initialized one piece at a time. >> >> The purpose of the study was to provide guidelines for API designers at >> Microsoft: ?apparently they now recommend create-set-call. ?Remember, >> that's the idea where you create an object without saying *anything* about >> what you want it to be and then successively kick it into shape. >> >> They later say >> >> ? ? ? ?[Systematic programmers] want not just to get their code working, >> ? ? ? ?but to understand why it works, what assumptions it makes >> ? ? ? ?and when it might fail. ?They are rare, and prefer languages that >> ? ? ? ?give them the most detailed control such as C++, C and assembly. >> >> I'd like to think of myself as a systematic programmer. ?I certainly like >> to understand all those things. ?But if I preferred assembly I would not >> be writing in the Erlang mailing list! ?The basic conclusion seems to be >> that you should not design APIs for such rare animals. >> >> I made what I thought were three obvious points: >> >> (1) There is a confounding factor in the study: ?the create-set-call style >> ? ?lets you *name* the information going into an object, while at that >> ? ?time the full-initialisation style did not. ?There are constructors >> ? ?for System.IO.FileStream with six arguments; maybe more. ?It seemed at >> ? ?least *possible* that if the subjects had been given a third choice, >> ? ?constructors with *named* parameters, they might have preferred that. >> ? ?Because the study didn't include such a choice, we certainly cannot >> ? ?use it to argue *against* that style. >> >> (2) C# 4.0 has named (and optional) parameters, so the study is no longer >> ? ?adequate to tell us about good API design in C#. >> >> (3) If there are programmers out there who *don't* care much about >> ? ?understanding what they are doing, I certainly don't want them writing >> ? ?anything that might in any way affect me or anyone I care about. >> ? ?If they just don't "get" constructors with parameters, that's really >> ? ?scary. >> >> A lengthy discussion has followed in which I've felt rather lonely. >> I'm being beaten about the head for not noticing things in the study >> that I did notice, and for being rather elitist. ?One quote: >> >> ? ? ? ?We don?t have the luxury of dismissing these types of programmers. >> ? ? ? ?While it might strike you with terror that these programmers exist, >> >> It doesn't frighten me that they _exist_, >> it frightens me that they are _programming_. >> >> ? ? ? ?they are successfully building applications in many different >> domains. >> ? ? ? ?They may work differently to you and many other programmers but >> that >> ? ? ? ?doesn?t necessarily mean that the code they create is worthless. >> ? ? ? ?Within the Visual Studio team at Microsoft we?ve devoted efforts to >> ? ? ? ?attempting to make them successful by adapting to their workstyles >> ? ? ? ?when appropriate. >> >> I find myself wondering just how "successful" their applications really >> are. >> Oh no! ?I hope they're not writing climate modelling code! ?That would >> explain so much... ?(Yes, I've looked at the code in the ClimateGate >> dump.) >> >> Frankly, I've turned here for a bit of comfort. ?If anyone likes >> completely >> initialising things rather than smacking one field at a time, surely I >> will >> find such people here. >> >> Am I wrong? ?Would *you* be happy opening a file in C by doing >> >> ? ? ? ?FILE *f = malloc(sizeof f); >> ? ? ? ?set_title(f, "foo/bar.txt"); >> ? ? ? ?set_access(f, "r"); >> ? ? ? ?gets(f, buffer); >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dan@REDACTED Fri Mar 30 06:12:31 2012 From: dan@REDACTED (Daniel Dormont) Date: Fri, 30 Mar 2012 00:12:31 -0400 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: References: Message-ID: I think I'm mostly with Michael on this one. I do program Java for my day job for the most part, and I do use IOC, and I do find the _requirement_ to have/use empty constructors to be somewhat tedious. I've also found over the years that one advantage I have in programming is that I can keep more random crap in my head at once, or at least access it faster, than most people out there in the world. But I've never been able to keep the order of more than a few parameters to a function straight. So really long constructor argument lists are a dealbreaker for me. I haven't done enough Erlang for the "Erlangish" way to be intuitive for me yet. For example I can absolutely never remember what the order of arguments to lists:keyreplace is. Dan On Thu, Mar 29, 2012 at 6:15 AM, Michael Turner < michael.eugene.turner@REDACTED> wrote: > > It doesn't frighten me that [such programmers] _exist_, > > it frightens me that they are _programming_. > > LOL. > > I don't have that much against create-set-call, provided that > > (1) What I get from creation is either > (1a) ... something in a consistent and usable state, or > (1b) ... something that will throw a useful exception in informative > ways if it can't (yet) be used > ... and .... > (2) Expression of the "set" part is reasonably concise and clear, and > always yields a situation like (1ab) above. > > But it does seem to lend itself to small-minded purism. Of which we've > had plenty. And from Microsoft, too. (Remember Hungarian Notation?) > > I mean, how about this? > > P = new Point > P.set_x(0) > P.set_y(0) > > It's just dumb to *require* doing it that way. The test for me isn't > purity, it's idiomaticity. I'm not saying Point wouldn't have setters > and getters. Just that I shouldn't be limited to those. > > On the other side of the debate: I once worked in a C++ shop with an > API that required up to 22 parameters for construction of certain > objects. "It's not so bad," a long-timer tried to assure me, "You just > copy-paste a call from someone else's code then make the changes you > need ...." I almost hit the ceiling. > > -michael turner > > On Thu, Mar 29, 2012 at 4:32 PM, Richard O'Keefe > wrote: > > I'm to the Psychology of Programming Interest Group mailing list. > > In a discussion of naming, reference was made to a body of work > > containing > > Usability Implications of Requiring Parameters > > in Objects' Constructors > > by Jeffrey Stylos of CMU > > and Steven Clarke of Microsoft > > at > http://www.cs.cmu.edu/~NatProg/papers/Stylos2007CreateSetCall.pdf > > > > The abstract reads > > The usability of APIs is increasingly important to programmer > > productivity. Based on experience with usability studies of > > specific APIs, techniques were explored for studying the usability > > of design choices common to many APIs. A comparative study was > > performed to assess how professional programmers use APIs with > > required parameters in objects? constructors as opposed to > > parameterless ?default? constructors. It was hypothesized that > > required parameters would create more usable and self-documenting > > APIs by guiding programmers toward the correct use of objects and > > preventing errors. However, in the study, it was found that, > > contrary to expectations, programmers strongly preferred and > > were more effective with APIs that did not require constructor > > parameters. Participants? behavior was analyzed using the > > cognitive dimensions framework, and revealing that required > > constructor parameters interfere with common learning strategies, > > causing undesirable premature commitment. > > > > The study was done in 2005. > > We're talking about the difference between > > > > fs = new FileReader("foo/bar.txt", Sharing.NONE); > > ln = fs.ReadLine(); > > ... > > > > and > > > > fs = new FileReader(); > > fs.SetFileName("foo/bar.txt"); > > fs.SetSharing(Sharing.NONE); > > ln = fs.ReadLine(); > > ... > > > > I think this is worth bringing up here because if functional programming > > is about anything at all, it's certainly about NOT setting up a value one > > field at a time! > > > > Their sample was carefully chosen to include three kinds of programmers: > > > > * OPPORTUNISITC programmers are more concerned with productivity > > than control or understanding. For these programmers objects > > that required constructor parameters were unfamiliar and > > unexpected, and even after repeated exposure these programmers > > had difficulty with these objects. > > > > That is, they just didn't "get" the idea of constructors having > > parameters. > > > > * PRAGMATIC programmers balance productivity with control and > > understanding. These programmers also did not expect objects > > with required constructors, and while pragmatic programmers > > were more effective than opportunistic programmers at using > > these objects, the objects still provided a minor stumbling > > block and these programmers preferred the flexibility offered > > by objects that used the create-set-call pattern. > > > > Remember, this was all about .NET. Failing to expect > > constructors with parameters in C# is like failing to expect > > assignment statements in C. > > > > * SYSTEMATIC programmers program defensively and these are the > > programmers for whom low-level APIs are targeted. These > programmers > > were effective at using all of the objects; however, they > preferred > > create-set-call because of the finer granularity of control it > > offered by allowing objects to be initialized one piece at a time. > > > > The purpose of the study was to provide guidelines for API designers at > > Microsoft: apparently they now recommend create-set-call. Remember, > > that's the idea where you create an object without saying *anything* > about > > what you want it to be and then successively kick it into shape. > > > > They later say > > > > [Systematic programmers] want not just to get their code working, > > but to understand why it works, what assumptions it makes > > and when it might fail. They are rare, and prefer languages that > > give them the most detailed control such as C++, C and assembly. > > > > I'd like to think of myself as a systematic programmer. I certainly like > > to understand all those things. But if I preferred assembly I would not > > be writing in the Erlang mailing list! The basic conclusion seems to be > > that you should not design APIs for such rare animals. > > > > I made what I thought were three obvious points: > > > > (1) There is a confounding factor in the study: the create-set-call > style > > lets you *name* the information going into an object, while at that > > time the full-initialisation style did not. There are constructors > > for System.IO.FileStream with six arguments; maybe more. It seemed at > > least *possible* that if the subjects had been given a third choice, > > constructors with *named* parameters, they might have preferred that. > > Because the study didn't include such a choice, we certainly cannot > > use it to argue *against* that style. > > > > (2) C# 4.0 has named (and optional) parameters, so the study is no longer > > adequate to tell us about good API design in C#. > > > > (3) If there are programmers out there who *don't* care much about > > understanding what they are doing, I certainly don't want them writing > > anything that might in any way affect me or anyone I care about. > > If they just don't "get" constructors with parameters, that's really > > scary. > > > > A lengthy discussion has followed in which I've felt rather lonely. > > I'm being beaten about the head for not noticing things in the study > > that I did notice, and for being rather elitist. One quote: > > > > We don?t have the luxury of dismissing these types of programmers. > > While it might strike you with terror that these programmers > exist, > > > > It doesn't frighten me that they _exist_, > > it frightens me that they are _programming_. > > > > they are successfully building applications in many different > domains. > > They may work differently to you and many other programmers but > that > > doesn?t necessarily mean that the code they create is worthless. > > Within the Visual Studio team at Microsoft we?ve devoted efforts > to > > attempting to make them successful by adapting to their workstyles > > when appropriate. > > > > I find myself wondering just how "successful" their applications really > are. > > Oh no! I hope they're not writing climate modelling code! That would > > explain so much... (Yes, I've looked at the code in the ClimateGate > dump.) > > > > Frankly, I've turned here for a bit of comfort. If anyone likes > completely > > initialising things rather than smacking one field at a time, surely I > will > > find such people here. > > > > Am I wrong? Would *you* be happy opening a file in C by doing > > > > FILE *f = malloc(sizeof f); > > set_title(f, "foo/bar.txt"); > > set_access(f, "r"); > > gets(f, buffer); > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hirotnkg@REDACTED Fri Mar 30 07:05:32 2012 From: hirotnkg@REDACTED (Yoshihiro Tanaka) Date: Thu, 29 Mar 2012 22:05:32 -0700 Subject: [erlang-questions] How do funs work ? In-Reply-To: References: Message-ID: 2012/3/29 ??? (Yau-Hsien Huang) : > 2012/3/30 Yoshihiro Tanaka >> >> Hi Yau-Hsien, >> >> It seems like not. For example, below example indicates F is not >> evaluated when constructing F2. >> >> 1> F = fun() -> blah end. >> #Fun >> 2> F2 = fun() -> 1 + 2 + F() end. This example was not helpful to check if F is executed when constructing F2. I think I should have written as: 2> EvalCheckFun = fun() -> 1 + 2 + fun() -> io:format("blah~n"), blah end() end. #Fun 3> EvalCheckFun(). blah ** exception error: bad argument in an arithmetic expression in operator +/2 called as 3 + blah > > > No. Here you type F() which means F is evaluated when F2 is evaluating. > I said the word "evaluated" but "reduced." > > If you expand the symbol 'F' to the function, in syntax, it must be > ? ? F2 = fun() -> 1 + 2 + (fun -> 0 end)() end. > but > ? ? F2 = fun() -> 1 + 2 + (fun() -> 0 end) end. > Later one is depicted in your original letter, then I point the difference Maybe I'm not understanding something, but I can not find it. Could you point that line in my original post ? My current understanding is when I write like below, the part '(fun -> 0 end)()' is not evaluated/executed yet when F2 is bound, and it will be evaluated when F2 is evaluated as F2(). F2 = fun() -> 1 + 2 + (fun -> 0 end)() end. Is this understanding different from yours ? > on parenthesis after.?Maybe it's ill-wording, that it ought to?be function > application > for symbol 'F()' and function-taking for symbol 'F'. > > >> #Fun >> 3> F2(). >> ** exception error: bad argument in an arithmetic expression >> ? ? in operator ?+/2 >> ? ? ? ?called as 3 + blah >> >> I also checked core file of sample code, using 'to_core' option, and the >> code: >> 18 ? ? ? ? ? fun() -> 9 + 10 + >> 19 ? ? ? ? ? ? fun() -> 0 end() >> 20 ? ? ? ? ? end() > > > The part of the core file seems as what I mentioned. :) > >> >> is translated to: >> %% Line 18 >> ( fun () -> >> ? ? ?let <_cor0> = >> ? ?%% Line 19 >> ? ?( fun () -> >> ? ?0 >> ? ? ?-| [{'id',{2,100470014,'-make_fun2/0-fun-0-'}}] ) >> ? ? ?in ?let <_cor1> = >> ? ? ? ?%% Line 19 >> ? ? ? ?apply _cor0 >> ? ? ?() >> ? ?in ?call 'erlang':'+' >> ? ? ?(19, _cor1) >> ?-| [{'id',{3,119737161,'-make_fun2/0-fun-1-'}}] ) >> >> It defines <_cor0>, as fun() -> 0 end, then doing 'apply _cor0'. So >> It's calling fun _cor0. >> >> >> Yoshihiro >> >> >> >> > >> >> >> >> Also, is there any difference between funs that are defined at runtime >> >> and funs that are defined at compile time in terms of how they are >> >> executed ? >> >> >> >> Thank you >> >> Yoshihiro >> >> >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > >> > >> > -- >> > >> > Best Regards. >> > >> > --- Y-H. H. >> > >> > > > > > > -- > > Best Regards. > > --- Y-H. H. > > From erlangsiri@REDACTED Fri Mar 30 09:05:44 2012 From: erlangsiri@REDACTED (Siri Hansen) Date: Fri, 30 Mar 2012 09:05:44 +0200 Subject: [erlang-questions] systools problem? In-Reply-To: <995833CF-23F7-40D1-B8F2-5B8940EDBE3B@googlemail.com> References: <6D456DF1-BFE3-403B-AB44-A08CBF0AE1FC@gmail.com> <995833CF-23F7-40D1-B8F2-5B8940EDBE3B@googlemail.com> Message-ID: This is corrected in R15B01 which will be out next week. /siri Den 22:54 29. mars 2012 skrev Andy Richards < andy.richards.iit@REDACTED> f?lgende: > Hi Jared, yes I ran into this problem. There is a long old thread about it > here.. > http://erlang.org/pipermail/erlang-patches/2012-January/002563.html > > I have a shell script from which I call rebar to generate my upgrades and > releases and from this script I grep the .app files and replace the > undefined tuple with a empty list which solved my problem in a simple > application when generating upgrades. I'm not sure if this is a suitable > work a round for a more complex application. > > Andy. > > Sent from my iPad > > On 29 Mar 2012, at 19:34, Jared Kofron wrote: > > > When I do ./rebar generate-upgrade previous_release=v0p1, I get "ERROR: > Systools [systools:make_relup/4] aborted with", and then a huge > > list of errors of the form: > > > > {error_reading, > > {mochiweb, > > {{bad_param, > > start_phases}, > > {application, > > mochiweb, > > [{description, > > "MochiMedia > Web Server"}, > > {vsn,"2.3.1"}, > > {id,[]}, > > {modules, > > [mochifmt, > > > mochifmt_records, > > > mochifmt_std, > > mochiglobal, > > mochihex, > > mochijson, > > mochijson2, > > mochilists, > > > mochilogfile2, > > mochinum, > > mochitemp, > > mochiutf8, > > mochiweb, > > > mochiweb_acceptor, > > > mochiweb_charref, > > > mochiweb_cookies, > > > mochiweb_cover, > > > mochiweb_echo, > > > mochiweb_headers, > > > mochiweb_html, > > > mochiweb_http, > > mochiweb_io, > > > mochiweb_mime, > > > mochiweb_multipart, > > > mochiweb_request, > > > mochiweb_request_tests, > > > mochiweb_response, > > > mochiweb_socket, > > > mochiweb_socket_server, > > > mochiweb_util, > > reloader]}, > > > {registered,[]}, > > {applications, > > > [kernel,stdlib, > > > crypto,inets, > > ssl,xmerl, > > compiler, > > > syntax_tools]}, > > > {included_applications, > > []}, > > {env,[]}, > > {start_phases, > > undefined}, > > > {maxT,infinity}, > > {maxP, > > > infinity}]}}}} > > > > Digging in a little bit it seems like systools is expecting an empty > list instead of 'undefined' for start_phases. Has anybody else run in to > this? > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ttmrichter@REDACTED Fri Mar 30 10:44:02 2012 From: ttmrichter@REDACTED (Michael Richter) Date: Fri, 30 Mar 2012 16:44:02 +0800 Subject: [erlang-questions] Fwd: Fear and Loathing in Programming La La Land In-Reply-To: References: <4F1DDB056DD742D3A07BC6ABE7804DFE@srstrong.com> Message-ID: I need to learn to read the headers before sending. ---------- Forwarded message ---------- From: Michael Richter Date: 30 March 2012 16:42 Subject: Re: [erlang-questions] Fear and Loathing in Programming La La Land To: Steve Strong On 29 March 2012 15:49, Steve Strong wrote: > The idea of create-set-call is, imho, madness. It means that you never > really know if an object is in a sane state to be used - what properties > need to be set to make the object "usable"? What happens if some properties > are set more than once? What if I set some properties, start using the > object, and then change some properties? > This. This right here. Yes, this. I have not much to add here beyond this being why I simply cannot abide most OOP APIs. I positively loathe anything that allows me to have software items in a Schr?dinger State where it is neither uninstantiated nor fully-instantiated. To me instantiation should be atomic or, if absolutely necessary, should be built up via builder objects (which can still enforce sanity within their build() methods). This doesn't mean that I like having to remember a dozen different positional parameters to an initialization method, of course. That's just as much madness as is the create-set-call cycle. If you're using a language that's so lame it can't do named parameters and/or provide sane defaults, the builder pattern is the way around the lameness. -- "Perhaps people don't believe this, but throughout all of the discussions of entering China our focus has really been what's best for the Chinese people. It's not been about our revenue or profit or whatnot." --Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra. -- "Perhaps people don't believe this, but throughout all of the discussions of entering China our focus has really been what's best for the Chinese people. It's not been about our revenue or profit or whatnot." --Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wmknapik@REDACTED Fri Mar 30 10:44:07 2012 From: wmknapik@REDACTED (Wojciech Knapik) Date: Fri, 30 Mar 2012 10:44:07 +0200 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: <4AF16C06-60D5-47DF-B69C-CBB6E5CE1FA2@feuerlabs.com> References: <20120321013825.GA3889@h82-143-167-72-static.e-wro.net.pl> <4AF16C06-60D5-47DF-B69C-CBB6E5CE1FA2@feuerlabs.com> Message-ID: <20120330084407.GA873@h82-143-167-72-static.e-wro.net.pl> On Wed, Mar 21, 2012 at 07:33:22AM +0100, Ulf Wiger wrote: > > Let's assume for a moment, that full replication is indeed unnecessary. > > We still want to run identical nodes. Or at least the static code > > delivered to all nodes should be the same and runtime decisions should > > designate a master node. That is not trivial. Would we have to > > "manually" implement an election algorithm, or are there any Erlang > > mechanisms that would make this simpler ? If "manual" is the answer, > > then would you suggest any specific algorithm ? > > A simple way to create a critical section is to use global:trans > > http://www.erlang.org/doc/man/global.html#trans-2 > > In this case, what you really need to ensure that the initial schema is created > only once, on one node, and the other nodes attach themselves to it. Thanks, that was a great hint! I'm now using global:trans to decide who will initiate the database. I have one more question though. I need replicating disc_copies, not ram_copies, so I need to create_schema(ListOfNodes). But what if a node wants to join in later ? Is it possible to intiate replicating the schema table from the node that joins in ? Also, how can I check if the node is already set for replicating before starting mnesia ? I hear nodes with disc_copies will "remember" their replication settings when you start mnesia, but the schema manipulations, if necessary, need to be done before calling mnesia:start()... > > From what everyone here is saying, full mnesia replication will give > > us fault tolerance, but at a severe cost to scalability and > > performance. > > Well, I don't know about severe? :) > > In many ways, mnesia is one of the speediest databases you can find > for Erlang applications. In terms of scalability, the practical limits > for a fully replicated mnesia database seem to be somewhere around 10 > nodes, at which point it becomes quite problematic. But some very > performant systems have been written on top of mnesia. We decided to go with 10 replicating nodes and the remaining ones will just start with extra_db_nodes. If any of the 10 goes down, we'll make one of the others start replicating. WK From ulf@REDACTED Fri Mar 30 11:08:39 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 30 Mar 2012 02:08:39 -0700 Subject: [erlang-questions] Support for newcomers and the popularity of Erlang In-Reply-To: <20120330084407.GA873@h82-143-167-72-static.e-wro.net.pl> References: <20120321013825.GA3889@h82-143-167-72-static.e-wro.net.pl> <4AF16C06-60D5-47DF-B69C-CBB6E5CE1FA2@feuerlabs.com> <20120330084407.GA873@h82-143-167-72-static.e-wro.net.pl> Message-ID: <9BCC5755-3F74-471D-A810-22C96981C652@feuerlabs.com> On 30 Mar 2012, at 01:44, Wojciech Knapik wrote: > I have one more question though. I need replicating disc_copies, not > ram_copies, so I need to create_schema(ListOfNodes). But what if a node > wants to join in later ? Is it possible to intiate replicating the > schema table from the node that joins in ? You can start mnesia with {extra_db_nodes, KnownNodes]}. This will establish a ram-only schema on the current node, containing all meta-data about the database (assuming KnownNodes contains actual schema nodes). After starting, you can call mnesia:change_table_copy_type(schema, node(), disc_copies), to make the schema on the current node persistent. BR, Ulf W -------------- next part -------------- An HTML attachment was scrubbed... URL: From janburse@REDACTED Fri Mar 30 13:31:31 2012 From: janburse@REDACTED (Jan Burse) Date: Fri, 30 Mar 2012 13:31:31 +0200 Subject: [erlang-questions] Fwd: Fear and Loathing in Programming La La Land In-Reply-To: References: <4F1DDB056DD742D3A07BC6ABE7804DFE@srstrong.com> Message-ID: <4F759993.9090301@fastmail.fm> Michael Richter schrieb: > If you're using a language that's so lame it can't do named parameters > and/or provide sane defaults, the builder pattern is the way around the > lameness. There are a couple of languages that have named parameters, i.e. ADA, SmallTalk, Objective-C, etc.. But with setters you get a little bit more than with named parameters when it comes to defaults. Take the buyMilk example. Assume that "next farmer" has a some precalculated lookup and "shopping mal" would need a real lookup. With setter we can simply do: class BuyMilk { locationAddr = AddrBook.nextFarmerAddr(); void setLocation(String location) { if (location.equals("next farmer")) { locationAddr = AddrBook.nextFarmerAddr; } else { locationAddr = AddrBook.lookup(location); } } void buyMilk() { /* ... */ } } With methods and defaults we would do: class BuyMilk { void buyMilk(String location) { Addr locationAddr; if (location.equals("next farmer")) { locationAddr = AddrBook.nextFarmerAddr; } else { locationAddr = AddrBook.lookup(location); } } void buyMilk() { buyMilk("next farmer"); } } But this is only the half story. What happens if we have more verbs? With methods and defaults something as follows would happen: class Trade { Addr find(String location) { if (location.equals("next farmer")) { return AddrBook.nextFarmerAddr; } else { return AddrBook.lookup(location); } } void buyMilk(String location) { Addr locationAddr = find(location); /* ... */ } void sellCloth(String location) { Addr locationAddr = find(location); /* ... */ } void sellCloth() { sellCloth("next farmer"); } void buyMilk() { buyMilk("next farmer"); } } So basically to avoid duplicate code, the code to sanitize the parameter will land somewhere in a separate routine. Why not put that code in a setter: class Trade { locationAddr = AddrBook.nextFarmerAddr(); void setLocation(String location) { if (location.equals("next farmer")) { locationAddr = AddrBook.nextFarmerAddr; } else { locationAddr = AddrBook.lookup(location); } } void buyMilk() { /* ... */ } void sellCloth() { /* ... */ } } Clearly the setter thing is an imperative approach. But it is not that bloathed compared with other approaches, as the example shows. And it can be applied in a quite disciplined and elegant way. Since the setter thing comes also with naming conventions, usually most of the IDE support the following convenience: - Automatic creation of setters (and getters) for selected variables, so that one can easily begin coding. - Automatic cross referencing of variables together with their setters (and getters). - Automatic refactoring of variables together with their setters (and getters). In the above one can view setters as write access to variables, and getters as read access to variables, which also allows for more varied cross referencing and code analysis. But of course there are also a couple of pitfalls. Bye From emmiller@REDACTED Fri Mar 30 15:01:29 2012 From: emmiller@REDACTED (Evan Miller) Date: Fri, 30 Mar 2012 08:01:29 -0500 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: References: Message-ID: On Thu, Mar 29, 2012 at 9:47 PM, Michael Turner wrote: > "Requiring parameters in the constructor packs much more information > per line of code than the create-set-call convention, and so I am able > to read it more quickly." > > Yes, IF you know the order of parameters and their meanings in > advance, OR you have some way to look those up almost instantly. True > in some cases, not in all. You can read any piece of code very fast. > The challenge is more often in *understanding* what you're reading. If > the name of the class/type is suggestive, e.g., Point, then > > ? ?P = Point(0,1) > > might seem pretty obvious: x=0, y=1. But how about this? > > ? ?R = Rect(a,n0,r,v) For me "readability" has to do with having a lot of code on-screen at once, and not the number of tokens I can pass my eyes over per second. I find your chained constructor example to be highly readable for this reason. But setting one attribute per line of code is not readable in this sense. In my own experience the argument semantics are usually obvious from the context of well-written code. I am always forgetting the argument order to "qsort", for instance. Is it the number of elements first or the size of the element? But when I see qsort(table, var_count, sizeof(varlookup_t), &compare_elements) I know the answer instantly. Of course, if the original author has inadvertently switched two arguments to a constructor, it will be harder to debug. So in my mind, if you have the luxury of working with very good code in familiar territory, "initialize everything" is the preferable style. But if you are condemned to working with inferior code, or code that uses unfamiliar libraries, the alternative will prove to be more merciful. Evan > > Do the parameters refer to the *ranges* ?[a,n0] and [r,v]? Or to > *points*: (a,n0) the lower left, and (r,v) the upper right? The > following (notional) syntax for construction and setting instance > variables would make it clear: > > ? ?R = Rect().lowerleft(a,n0).upperright(r,v) > > and it doesn't exclude other equivalent constructions for the same type, e.g. > > ? ?R = Rect().x_range(a,r).y_range(n0,v) > > " Initialize-everything also reduces the potential for bugs, because > with create-set-call there become N! ways to initialize an object, > instead of just 1." > > N! overstates the *likely* combinations. Order of initialization is > typically idiomatic (e.g., people set x before y, usually). Explicit > setting of parameters that ordinarily default to reasonable values > will tend to follow other initializations that are almost always set > explicitly. > > Create-all-at-once has its pitfalls too. I mentioned a job where a > co-worker said that constructors with 22 parameters were OK because > you could just copy a working constructor call from some other code, > then make any mods you felt necessary. I immediately pointed out to > him that you might be copy-pasting bugs -- bugs that were latent in > the code you were copying, and that would manifest themselves much > more perfidiously in what you were copying to. > > If create-set-call is the dominant style, I think people should at > least enforce rules about states reached by setting certain values, > and exceptions thrown if operations are performed prematurely, on any > not-quite-yet-initialized object. > > I don't think there's any single right way. Readability and > conciseness will always trade off in complex ways, depending on > people's preconceptions. I usually prefer a mix, when I write code. > > -michael turner > > On Thu, Mar 29, 2012 at 10:50 PM, Evan Miller wrote: >> Programmer "types" aside, ease of writing code is not the only desideratum. >> Something missing from your discussion is program readability and >> maintainability. Requiring parameters in the constructor packs much more >> information per line of code than the create-set-call convention, and so I >> am able to read it more quickly. Initialize-everything also reduces the >> potential for bugs, because with create-set-call there become N! ways to >> initialize an object, instead of just 1. >> >> It is short-sighted to consider only programmers' stated preferences when >> designing an API. I am sure programmers in the study would also have >> preferred not to write code comments or unit tests if given the option to >> leave early for lunch. I am reminded of a quotation that appeared in my high >> school's writing center: >> >> "What is written without effort is in general read without pleasure." >> --Samuel Johnson >> >> >> Evan >> >> On Thu, 29 Mar 2012 02:32:33 -0500, Richard O'Keefe >> wrote: >> >>> I'm to the Psychology of Programming Interest Group mailing list. >>> In a discussion of naming, reference was made to a body of work >>> containing >>> ? ? ? ?Usability Implications of Requiring Parameters >>> ? ? ? ?in Objects' Constructors >>> ? ? ? ?by Jeffrey Stylos of CMU >>> ? ? ? ?and Steven Clarke of Microsoft >>> at ? ? ?http://www.cs.cmu.edu/~NatProg/papers/Stylos2007CreateSetCall.pdf >>> >>> The abstract reads >>> ? ? ? ?The usability of APIs is increasingly important to programmer >>> ? ? ? ?productivity. ?Based on experience with usability studies of >>> ? ? ? ?specific APIs, techniques were explored for studying the usability >>> ? ? ? ?of design choices common to many APIs. ?A comparative study was >>> ? ? ? ?performed to assess how professional programmers use APIs with >>> ? ? ? ?required parameters in objects? constructors as opposed to >>> ? ? ? ?parameterless ?default? constructors. ?It was hypothesized that >>> ? ? ? ?required parameters would create more usable and self-documenting >>> ? ? ? ?APIs by guiding programmers toward the correct use of objects and >>> ? ? ? ?preventing errors. ?However, in the study, it was found that, >>> ? ? ? ?contrary to expectations, programmers strongly preferred and >>> ? ? ? ?were more effective with APIs that did not require constructor >>> ? ? ? ?parameters. ?Participants? behavior was analyzed using the >>> ? ? ? ?cognitive dimensions framework, and revealing that required >>> ? ? ? ?constructor parameters interfere with common learning strategies, >>> ? ? ? ?causing undesirable premature commitment. >>> >>> The study was done in 2005. >>> We're talking about the difference between >>> >>> ? ? ? ?fs = new FileReader("foo/bar.txt", Sharing.NONE); >>> ? ? ? ?ln = fs.ReadLine(); >>> ? ? ? ?... >>> >>> and >>> >>> ? ? ? ?fs = new FileReader(); >>> ? ? ? ?fs.SetFileName("foo/bar.txt"); >>> ? ? ? ?fs.SetSharing(Sharing.NONE); >>> ? ? ? ?ln = fs.ReadLine(); >>> ? ? ? ?... >>> >>> I think this is worth bringing up here because if functional programming >>> is about anything at all, it's certainly about NOT setting up a value one >>> field at a time! >>> >>> Their sample was carefully chosen to include three kinds of programmers: >>> >>> ?* ? ? OPPORTUNISITC programmers are more concerned with productivity >>> ? ? ? ?than control or understanding. ?For these programmers objects >>> ? ? ? ?that required constructor parameters were unfamiliar and >>> ? ? ? ?unexpected, and even after repeated exposure these programmers >>> ? ? ? ?had difficulty with these objects. >>> >>> ? ? ? ?That is, they just didn't "get" the idea of constructors having >>> ? ? ? ?parameters. >>> >>> ?* ? ? PRAGMATIC programmers balance productivity with control and >>> ? ? ? ?understanding. ?These programmers also did not expect objects >>> ? ? ? ?with required constructors, and while pragmatic programmers >>> ? ? ? ?were more effective than opportunistic programmers at using >>> ? ? ? ?these objects, the objects still provided a minor stumbling >>> ? ? ? ?block and these programmers preferred the flexibility offered >>> ? ? ? ?by objects that used the create-set-call pattern. >>> >>> ? ? ? ?Remember, this was all about .NET. ?Failing to expect >>> ? ? ? ?constructors with parameters in C# is like failing to expect >>> ? ? ? ?assignment statements in C. >>> >>> ?* ? ? SYSTEMATIC programmers program defensively and these are the >>> ? ? ? ?programmers for whom low-level APIs are targeted. ?These >>> programmers >>> ? ? ? ?were effective at using all of the objects; however, they preferred >>> ? ? ? ?create-set-call because of the finer granularity of control it >>> ? ? ? ?offered by allowing objects to be initialized one piece at a time. >>> >>> The purpose of the study was to provide guidelines for API designers at >>> Microsoft: ?apparently they now recommend create-set-call. ?Remember, >>> that's the idea where you create an object without saying *anything* about >>> what you want it to be and then successively kick it into shape. >>> >>> They later say >>> >>> ? ? ? ?[Systematic programmers] want not just to get their code working, >>> ? ? ? ?but to understand why it works, what assumptions it makes >>> ? ? ? ?and when it might fail. ?They are rare, and prefer languages that >>> ? ? ? ?give them the most detailed control such as C++, C and assembly. >>> >>> I'd like to think of myself as a systematic programmer. ?I certainly like >>> to understand all those things. ?But if I preferred assembly I would not >>> be writing in the Erlang mailing list! ?The basic conclusion seems to be >>> that you should not design APIs for such rare animals. >>> >>> I made what I thought were three obvious points: >>> >>> (1) There is a confounding factor in the study: ?the create-set-call style >>> ? ?lets you *name* the information going into an object, while at that >>> ? ?time the full-initialisation style did not. ?There are constructors >>> ? ?for System.IO.FileStream with six arguments; maybe more. ?It seemed at >>> ? ?least *possible* that if the subjects had been given a third choice, >>> ? ?constructors with *named* parameters, they might have preferred that. >>> ? ?Because the study didn't include such a choice, we certainly cannot >>> ? ?use it to argue *against* that style. >>> >>> (2) C# 4.0 has named (and optional) parameters, so the study is no longer >>> ? ?adequate to tell us about good API design in C#. >>> >>> (3) If there are programmers out there who *don't* care much about >>> ? ?understanding what they are doing, I certainly don't want them writing >>> ? ?anything that might in any way affect me or anyone I care about. >>> ? ?If they just don't "get" constructors with parameters, that's really >>> ? ?scary. >>> >>> A lengthy discussion has followed in which I've felt rather lonely. >>> I'm being beaten about the head for not noticing things in the study >>> that I did notice, and for being rather elitist. ?One quote: >>> >>> ? ? ? ?We don?t have the luxury of dismissing these types of programmers. >>> ? ? ? ?While it might strike you with terror that these programmers exist, >>> >>> It doesn't frighten me that they _exist_, >>> it frightens me that they are _programming_. >>> >>> ? ? ? ?they are successfully building applications in many different >>> domains. >>> ? ? ? ?They may work differently to you and many other programmers but >>> that >>> ? ? ? ?doesn?t necessarily mean that the code they create is worthless. >>> ? ? ? ?Within the Visual Studio team at Microsoft we?ve devoted efforts to >>> ? ? ? ?attempting to make them successful by adapting to their workstyles >>> ? ? ? ?when appropriate. >>> >>> I find myself wondering just how "successful" their applications really >>> are. >>> Oh no! ?I hope they're not writing climate modelling code! ?That would >>> explain so much... ?(Yes, I've looked at the code in the ClimateGate >>> dump.) >>> >>> Frankly, I've turned here for a bit of comfort. ?If anyone likes >>> completely >>> initialising things rather than smacking one field at a time, surely I >>> will >>> find such people here. >>> >>> Am I wrong? ?Would *you* be happy opening a file in C by doing >>> >>> ? ? ? ?FILE *f = malloc(sizeof f); >>> ? ? ? ?set_title(f, "foo/bar.txt"); >>> ? ? ? ?set_access(f, "r"); >>> ? ? ? ?gets(f, buffer); >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Fri Mar 30 15:15:38 2012 From: ok@REDACTED (ok@REDACTED) Date: Sat, 31 Mar 2012 02:15:38 +1300 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: References: Message-ID: <2d0196bdcd67d52bf35e867014547278.squirrel@chasm.otago.ac.nz> > "Requiring parameters in the constructor packs much more information > per line of code than the create-set-call convention, and so I am able > to read it more quickly." We've had NFS problems, so I haven't yet seen whoever wrote that. > > Yes, IF you know the order of parameters and their meanings in > advance, OR you have some way to look those up almost instantly. True > in some cases, not in all. You can read any piece of code very fast. > The challenge is more often in *understanding* what you're reading. If > the name of the class/type is suggestive, e.g., Point, then > > P = Point(0,1) > > might seem pretty obvious: x=0, y=1. The issue is that the study in question purports to show that create-set-call is better, without making it clear that the only empirical support for that conclusion is limited to a version of C# before it had optional and keyword parameters. Smalltalk: Point x: 0 y: 1 C# 4 makes it possible to do Point(x: 0, y: 1) Ada: Point(X => 0, Y => 1) SML: {x = 0, y = 1} : point Haskell: Point {x = 0, y = 1} But how about this? > > R = Rect(a,n0,r,v) This is what keyword parameters are for: Lisp: (make-rectangle :top y0 :left x0 :bottom y1 :right x1) Smalltalk: Rectangle origin: x0@REDACTED corner: x1@REDACTED Fortran (yes, really, Fortran!): Rectangle(top = y0, left = x0, bottom = y1, right = x1) Smalltalk has been around since 1980. Ada has been around since 1981. Common Lisp was I believe the first OO language to be standardised. Objective C (a horror movie blend of C and Smalltalk) is from the 80s. I don't know when DEC Pascal got keyword arguments, but some time before the mid-80s would be my guess. > R = Rect().lowerleft(a,n0).upperright(r,v) This makes for a much more complex interface. What if there _aren't_ any sensible default values? Erlang doesn't have keyword parameters, but it does have rect:new([{bottom,Y1},{left,X0},{top,Y0},{right,X1}]) which gets the job done. It would be possible to extend Erlang syntax according to Paul Lyons' "split procedure names" idea allowing rect:new bottom(Y1) left(X0) top(Y0) right(X1) where the splitting allows the packets after the first to be freely reordered. I meant to write up an EEP about that, but Markdown has defeated me. > > " Initialize-everything also reduces the potential for bugs, because > with create-set-call there become N! ways to initialize an object, > instead of just 1." > > N! overstates the *likely* combinations. Perhaps. But it definitely UNDERstands the possible ways. For 3 facets A B C there are A A B A B C A C B A C B B A B A C B C A B C C C A C A B C B A C B for fifteen possibilities, not just the 3!=6 you might expect. The problem is not that all these possibilities *exist* but that they must all be given *semantics*; create-set-call by intent cannot and does not *enforce* any 'idiomatic' order or choice. > Create-all-at-once has its pitfalls too. I mentioned a job where a > co-worker said that constructors with 22 parameters were OK In my Smalltalk library, there is just one method with 6 parameters; string1 from: begin1 to: end1 sameAs: string2 from: begin2 to: end2 ignoringCase: aBoolean. No genuine method has more. (ANSI ST allows methods with up to 15 parameters to be called indirectly; there is machinery to allow that.) Once we had languages where you could return a record, 22 parameters could no longer be taken seriously. > I don't think there's any single right way. Readability and > conciseness will always trade off in complex ways, depending on > people's preconceptions. Actually, conciseness just plain doesn't enter into this discussion at all. Whether you say t = new X(ping: 1, pang: 2, pong: 3, pung: 4) or t = new X(); t.ping(1); t.pang(2); t.pong(3); t.pung(4); makes very little difference. In fact C# 3 introduced a wierd quasi-constructor syntax where you write t = new X() {ping = 1, pang = 2, pong = 3, pung = 4} and what looks just like named parameters is actually compiled as a series of calls to setter methods. Given this context, it is easy to see that THERE IS NO SIGNIFICANT DIFFERENCE IN CONCISENESS BETWEEN FULL INITIALISATION (using keyword parameters) AND CREATE- SET-CALL (using C# 3 weird constructor syntax). THERE IS NO SIGNIFICANT DIFFERENCE IN READABILITY BETWEEN THE TWO APPROACHES. What is at issue is - exposing incompletely initialised objects - the lack of any distinction between 'setters for initialisation only' and 'setters that can be called at any time', resulting in anything you want to be initialised by name in the create-set-call antipattern being vulnerable to revision by ANYONE at ANY TIME even in another thread. From ttmrichter@REDACTED Fri Mar 30 15:19:31 2012 From: ttmrichter@REDACTED (Michael Richter) Date: Fri, 30 Mar 2012 21:19:31 +0800 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: <2d0196bdcd67d52bf35e867014547278.squirrel@chasm.otago.ac.nz> References: <2d0196bdcd67d52bf35e867014547278.squirrel@chasm.otago.ac.nz> Message-ID: On 30 March 2012 21:15, wrote: > What is at issue is > - exposing incompletely initialised objects > This is *THE* overriding issue for me in my hating the create-fill-use cycle. If you *must* use it, use a builder object. End of story. (But only if you have a lame language without named parameters.) -- "Perhaps people don't believe this, but throughout all of the discussions of entering China our focus has really been what's best for the Chinese people. It's not been about our revenue or profit or whatnot." --Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Mar 30 15:25:56 2012 From: ok@REDACTED (ok@REDACTED) Date: Sat, 31 Mar 2012 02:25:56 +1300 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: References: Message-ID: > harder to debug. So in my mind, if you have the luxury of working with > very good code in familiar territory, "initialize everything" is the > preferable style. But if you are condemned to working with inferior > code, or code that uses unfamiliar libraries, the alternative will > prove to be more merciful. You seem to be confounding two things, as indeed did the authors of the original study. The difference we are considering is that between var p = new Point(x: 1, y: 2); // C# 4.0 and var p = new Point() {x = 1, y = 2}; // C# 3.0 BOTH provide EQUAL amounts of information about which argument is which. BOTH provide EQUAL freedom to reorder or omit arguments. The problem is that the C# 3.0 approach * forces the interface to be more complex: you have to have constructor AND setter for x AND setter for y, whereas the C# 4.0 approach needs only the constructor * forces objects to be mutable that could otherwise be immutable (and thus be safe to share between threads) * makes objects vulnerable to having their guts twisted by strangers ANY TIME, not just during setup. Even without keyword parameters, var p = Point.Builder().SetX(1).SetY(2).Build(); provides all the clues about what is what that the create-set-call antipattern does, without ever once releasing an incompletely initialised object into the wild. There _is_ room for some analogue of keyword parameters in Erlang, and I have a proposal for that... From janburse@REDACTED Fri Mar 30 15:55:27 2012 From: janburse@REDACTED (Jan Burse) Date: Fri, 30 Mar 2012 15:55:27 +0200 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: <2d0196bdcd67d52bf35e867014547278.squirrel@chasm.otago.ac.nz> References: <2d0196bdcd67d52bf35e867014547278.squirrel@chasm.otago.ac.nz> Message-ID: <4F75BB4F.1020907@fastmail.fm> ok@REDACTED schrieb: > Actually, conciseness just plain doesn't enter into this discussion > at all. Whether you say > t = new X(ping: 1, pang: 2, pong: 3, pung: 4) > or > t = new X(); t.ping(1); t.pang(2); t.pong(3); t.pung(4); Thank you for clarifying this. > What is at issue is > - exposing incompletely initialised objects > - the lack of any distinction between 'setters for initialisation > only' and 'setters that can be called at any time', resulting > in anything you want to be initialised by name in the > create-set-call antipattern being vulnerable to revision by > ANYONE at ANY TIME even in another thread. I just noticed that the above questions only deal with the client view of a object/service whatever. On the other hand good code / maintainable code etc.. is a question that does not only deal how well an interface works for a client, but also how easy it is to maintain the implementation side of an object/service. But lets turn back to your client view question. I have already mentioned that when objects are concurrently used for tasks, that there is no need for synchronization. So the problem of vulnerability by another thread does not exist. How comes that task objects don't need synchronization? Well, in that the object is only created inside a thread and only used by this thread for a limited amount of time, and in a way that the object does not leak to other threads. Take as a very typical example Web pages that use some "beans" to access and integrate various data sources. The typical server code looks as follows: void getBookList() { out.println(""); BookListBean bean = new BookListBean(); bean.setDate(..); bean.setAuthor(..); bean.list(); while (bean.next()) { out.println(""); ... out.println(""); } bean.close(); out.println("
"); } The Web container will receive multiple requests for book lists. So the above code will be executed concurrently. Nevertheless the BookListBean need not be synchronized. Since only one thread at a time is using an instance of the class BookListBean. It is created on the fly and then left to the GC when it is not used anymore. The above pattern is found in many Web frameworks. It was initially there in JSP with the bean directive. But it can be used for any tasks you want. It is Peter Coads moment-interval approach. This approach can also be used to compose moment-intervals. Asssume for example that information about a book authors is fetched from a different data source. This can easily be done as follows: void getBookList() { BookListBean bean1 = new BookListBean(); ... bean1.list(); while (bean.next()) { AuthorInfoBean bean2 = new AuthorInfoBean(); ... bean2.list(); ... } bean1.close(); } Again no problem whatever with concurrent access to the beans bean1 and bean2, since they are solely owned by the thread that executes the web request getBookList. But maybe the above moment-interval stuff is outdated with the advent of googls map reduce. Did not yet look into the question whether alternate web request serving techniques might result in extra vulnerability. Maybe Richard can make a case where the sychronization is a problem. i.e. where a thread does go on, and expose an object to another thread in a fashion where setters would especially pose a problem. I think the solution is to declare whether a class is design to be used concurrently or not. And then in the extreme to deliver synchronized and unsynchronized versions, just as is with Hashtable and HashMap in Java. But when we have synchronized and unsynchronized versions of class we see that all methods are affected. Not only setters. In the case Hashtable and HashMap there are even hardly any setters , so the normal protocol of the object is affected anyway. Bye From emeka_1978@REDACTED Fri Mar 30 17:41:14 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Fri, 30 Mar 2012 08:41:14 -0700 (PDT) Subject: [erlang-questions] -heart question Message-ID: <288b1f74-93ee-4803-a19a-2640b3b05986@n5g2000vbf.googlegroups.com> I am deploying an erlang application in embedded form, ie the target system(suse) does not have erlang installed. I will like the application to restart each time the erlang system crashes. When i do "erl -heart ..." and i kill the process, the application is restarted. When i kill the application a second time, the heart process is gone as well. Is that supposed to be the normal behavior? This is my first time experiencing with erlang heartbeat mode. Thanks. From dave@REDACTED Fri Mar 30 23:04:23 2012 From: dave@REDACTED (David Goehrig) Date: Fri, 30 Mar 2012 17:04:23 -0400 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: <2d0196bdcd67d52bf35e867014547278.squirrel@chasm.otago.ac.nz> References: <2d0196bdcd67d52bf35e867014547278.squirrel@chasm.otago.ac.nz> Message-ID: <7555E388-0993-4133-8AFF-F9199891A83C@nexttolast.com> "exposing incompletely initialised objects" Which is why in a functional language with partial application, any constructor w/ parameters is kinda silly. Sometimes I really want a partially initialized object to clone. I've been playing around with an OO approach to Erlang, where each process is a bare loop and all state/capabilities are injected at runtime via messaging. With a prototype model it turns out partial initialization is a big win, as it allows for incremental differentiation. Dave -=-=- dave@REDACTED -=-=- From joelr1@REDACTED Fri Mar 30 23:53:14 2012 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 30 Mar 2012 22:53:14 +0100 Subject: [erlang-questions] MIX: A compiler from SQL to Erlang code using Amazon DynamoDB Message-ID: Take #2 since there wasn't much feeback on my last post :-) https://gist.github.com/d963ade3be6ab7312448 I switched to a flavor of SQL from the DSL I invented last time. Let me know what you think! Thanks, Joel P.S. I also plan to come up with a syntax to describe REST endpoints and automatically generate that code as well. Not sure what web server I will target yet. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Sat Mar 31 00:12:03 2012 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 30 Mar 2012 15:12:03 -0700 (PDT) Subject: [erlang-questions] MIX: A compiler from SQL to Erlang code using Amazon DynamoDB In-Reply-To: References: Message-ID: <2ac62a88-9d78-4946-95ec-2c90f7707d05@t16g2000yqt.googlegroups.com> Note that Amazon DynamoDB has no built-in referential integrity so I generate the code for it. For example, the code in subscription:create updates both Stack and Publisher "tables". From ok@REDACTED Sat Mar 31 03:59:51 2012 From: ok@REDACTED (Richard O'Keefe) Date: Sat, 31 Mar 2012 14:59:51 +1300 Subject: [erlang-questions] Fear and Loathing in Programming La La Land In-Reply-To: <7555E388-0993-4133-8AFF-F9199891A83C@nexttolast.com> References: <2d0196bdcd67d52bf35e867014547278.squirrel@chasm.otago.ac.nz> <7555E388-0993-4133-8AFF-F9199891A83C@nexttolast.com> Message-ID: <4CBF96FC-146A-4611-806C-170DDA6B14B0@cs.otago.ac.nz> On 31/03/2012, at 10:04 AM, David Goehrig wrote: > "exposing incompletely initialised objects" > > Which is why in a functional language with partial application, any constructor w/ parameters is kinda silly. Sometimes I really want a partially initialized object to clone. Please give a concrete example. Remember the basic distinction here: a class has a class invariant relating all the fields full initialisation establishes the entire class invariant partial initialisation establishes at most that part of the class invariant referring to a subset of the fields Here is a concrete example. Object subclass: #Complex instanceVariableNames: 're im' gettable: 're im' settable: 're im' methods: invariant ^(re isKindOf: Number) and: [im isKindOf: Number] ... a bunch of arithmetic methods ... ... inherited #new method ... You seem to be arguing that z := Complex new. z re: 0.0. i := z copy. i im: 1.0. minus_i := z copy. minus_i im: -1.0. is a good idea. In this example, the default #new method returned an object for which (z invariant) is false, the re and im fields being nil. Cloning and copying aren't exactly the same thing, and confusingly, Java uses the name 'clone' for what's really a 'copy'. The problem here is that now the #copy operation has to do something sensible with an incompletely initialised object. And so does the #+ method. And the #arcCosh method. And so on, for another fifty- odd methods. So now we have to change the definition slightly: methods: pvtPostNew re := im := 0.0. so that the default #new method (which calls #pvtPostNew, which is pretty much the analogue of Objective-C's [_ init]) fully establishes the invariant. But now this is no longer a case of PARTIAL initialisation. It is a case of DEFAULT initialisation to something which is fully operational. And you know, 'cloning' a fully initialised object works even better than 'cloning' a partially initialised one, because now the #copy method gets to assume that the invariant is true. In strict point of fact, the actual definition is Object valueSubclass: #Complex instanceVariableNames: 're im' gettable: 're im' and there is no possibility of changing a complex number, which is actually pretty vital for the way they are normally used. So can we please have a CONCRETE example from actual pre-existing code of a situation where - there is some kind of "thing" which is more than just a bundle of independent items - it makes sense to create a "thing" that is not fully initialised (that is, the information in the call that builds its initial state is not sufficient to determine values that make the invariant true) - it makes sense to perform any operation on such a "thing" in such a state other than to move closer to a fully initialised state - it makes sense to allow all initialisation commands at all times, OR - the burden of checking which operations are allowable in which states is acceptable. > I've been playing around with an OO approach to Erlang, where each process is a bare loop and all state/capabilities are injected at runtime via messaging. With a prototype model it turns out partial initialization is a big win, as it allows for incremental differentiation. Again, please provide a concrete example. Full initialisation ALSO allows for incremental differentiation. Continuing with the file stream example, it *might* make sense for r := FileReader new. r read: 'foo/bar.txt'. to work by initially connecting r to /dev/null and having #read: be like C's freopen(): close the current connection and establish a new one. That would be default-but-FULL-initialisation, not partial initialisation. In my own code, I've found it astonishing how rare it is for a plain setter to make sense. A little thought almost always shows that there is *something* to check. From michael.eugene.turner@REDACTED Sat Mar 31 08:41:35 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 31 Mar 2012 15:41:35 +0900 Subject: [erlang-questions] MIX: A compiler from SQL to Erlang code using Amazon DynamoDB In-Reply-To: <2ac62a88-9d78-4946-95ec-2c90f7707d05@t16g2000yqt.googlegroups.com> References: <2ac62a88-9d78-4946-95ec-2c90f7707d05@t16g2000yqt.googlegroups.com> Message-ID: Hmm, now if we only had a PHP-to-Erlang translator, we'd be taking over the world.... -michael turner On Sat, Mar 31, 2012 at 7:12 AM, Joel Reymont wrote: > Note that Amazon DynamoDB has no built-in referential integrity so I > generate the code for it. For example, the code in subscription:create > updates both Stack and Publisher "tables". > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From avinash@REDACTED Sat Mar 31 09:37:01 2012 From: avinash@REDACTED (Avinash Dhumane) Date: Sat, 31 Mar 2012 13:07:01 +0530 Subject: [erlang-questions] Comparing (pattern-matching) records Message-ID: I might have overlooked all the 3 books (for sections on records), but couldn't find an explanation why a pattern matching error doesn't occur in the last (7th) shell expression below. Please. Thanks. $ cat rec.hrl -record(rec, {a, b}). $ erl Erlang R14B02 (erts-5.8.3) [smp:2:2] [rq:2] [async-threads:0] Eshell V5.8.3 (abort with ^G) 1> rr("rec.hrl"). [rec] 2> A=#rec{}. #rec{a = undefined,b = undefined} 3> B=#rec{a=1}. #rec{a = 1,b = undefined} 4> A=B. ** exception error: no match of right hand side value #rec{a = 1,b = undefined} 5> {} = {1}. ** exception error: no match of right hand side value {1} 6> {rec, undefined, undefined} = {rec, 1, undefined}. ** exception error: no match of right hand side value #rec{a = 1,b = undefined} 7> #rec{} = #rec{a=1}. #rec{a = 1,b = undefined} From roe.adrian@REDACTED Sat Mar 31 09:48:23 2012 From: roe.adrian@REDACTED (Adrain Roe) Date: Sat, 31 Mar 2012 08:48:23 +0100 Subject: [erlang-questions] Comparing (pattern-matching) records In-Reply-To: References: Message-ID: 7) is saying "I'd like a rec record that matches a rec record where a is 1. It is a pattern match rather than an assertion that the lhs (with defaults for all unspecified fields) == rhs (with a =1 and b with its default value). I too found this somewhat confusing to start with, but this pattern matching is precisely where much of erlang's remarkable brevity and readability stems from. Adrian Sent from my iPhone On 31 Mar 2012, at 08:37, "Avinash Dhumane" wrote: > I might have overlooked all the 3 books (for sections on records), but couldn't find an explanation why a pattern matching error doesn't occur in the last (7th) shell expression below. > > Please. > > Thanks. > > $ cat rec.hrl > -record(rec, {a, b}). > $ erl > Erlang R14B02 (erts-5.8.3) [smp:2:2] [rq:2] [async-threads:0] > > Eshell V5.8.3 (abort with ^G) > 1> rr("rec.hrl"). > [rec] > 2> A=#rec{}. > #rec{a = undefined,b = undefined} > 3> B=#rec{a=1}. > #rec{a = 1,b = undefined} > 4> A=B. > ** exception error: no match of right hand side value #rec{a = 1,b = undefined} > 5> {} = {1}. > ** exception error: no match of right hand side value {1} > 6> {rec, undefined, undefined} = {rec, 1, undefined}. > ** exception error: no match of right hand side value #rec{a = 1,b = undefined} > 7> #rec{} = #rec{a=1}. > #rec{a = 1,b = undefined} > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Sat Mar 31 10:06:03 2012 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 31 Mar 2012 10:06:03 +0200 Subject: [erlang-questions] Comparing (pattern-matching) records In-Reply-To: References: Message-ID: You wrote > #rec{} = #rec{a=1}. When used on the righthand side of '=' #rec{} is a constructor which creates a new record. On the left hand side of '=' #rec{} is a *pattern* the above line means the same as if you had written: > #rec{a=_,b=_} = #rec{a=1}. In a record pattern you only need to include the tags you want to constrain. So > #rec{a=1} = #rec{} Means the same as > #rec{a=1,b=_} = #rec{} The RHS is a constructor and evaluates to #rec{a=1,b=undefined} The Lhs is a pattern and the match works. /Joe On Sat, Mar 31, 2012 at 9:37 AM, Avinash Dhumane wrote: > I might have overlooked all the 3 books (for sections on records), but > couldn't find an explanation why a pattern matching error doesn't occur in > the last (7th) shell expression below. > > Please. > > Thanks. > > $ cat rec.hrl > -record(rec, {a, b}). > $ erl > Erlang R14B02 (erts-5.8.3) [smp:2:2] [rq:2] [async-threads:0] > > Eshell V5.8.3 ?(abort with ^G) > 1> rr("rec.hrl"). > [rec] > 2> A=#rec{}. > #rec{a = undefined,b = undefined} > 3> B=#rec{a=1}. > #rec{a = 1,b = undefined} > 4> A=B. > ** exception error: no match of right hand side value #rec{a = 1,b = > undefined} > 5> {} = {1}. > ** exception error: no match of right hand side value {1} > 6> {rec, undefined, undefined} = {rec, 1, undefined}. > ** exception error: no match of right hand side value #rec{a = 1,b = > undefined} > 7> #rec{} = #rec{a=1}. > #rec{a = 1,b = undefined} > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From fhoerni@REDACTED Sat Mar 31 11:23:04 2012 From: fhoerni@REDACTED (Frederic Hoerni) Date: Sat, 31 Mar 2012 11:23:04 +0200 Subject: [erlang-questions] ttb:get_et_handler Message-ID: <20120331092304.GB22828@fredbox> Hello all, The Erlang documentation of module ttb mentions ttb:get_et_handler(), that would format traces for et_viewer. But it looks like that this function does not exist. What is the history about this ? Is the documentation osbolete ? Or am I missing the correct version of the tbb module ? (I tried with R13B03 and R14B01) What other easy way is there for viewing with et_viewer ? (viewing sequence charts is a reather interesting feature, I think) Frederic From antoine.koener@REDACTED Sat Mar 31 12:32:15 2012 From: antoine.koener@REDACTED (Antoine Koener) Date: Sat, 31 Mar 2012 12:32:15 +0200 Subject: [erlang-questions] MIX: A compiler from SQL to Erlang code using Amazon DynamoDB In-Reply-To: References: Message-ID: <4A403CAE-B2A4-4C38-8F32-D1A07519CA3B@gmail.com> Hi Joel, On Mar 30, 2012, at 23:53 , Joel Reymont wrote: > P.S. I also plan to come up with a syntax to describe REST endpoints > and automatically generate that code as well. Not sure what web server > I will target yet. Why not try to be webserver agnostic ? There is, for example, Joe's work about that... And of course, well done ! From mrtndimitrov@REDACTED Sat Mar 31 14:18:54 2012 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Sat, 31 Mar 2012 15:18:54 +0300 Subject: [erlang-questions] Performing gen_server:call to multiple processes in parallel Message-ID: <4F76F62E.1040503@gmail.com> Hello, What I am trying to achieve is to call several gen_servers in parallel. First, I thought I could use gen_server:multi_call/3 but reading the documentation (if I understood it correctly) I realized the gen_servers need to be on different nodes: one registered gen_server per node. Any ideas on how to achieve calling multiple processes on the same node in parallel? Thank you very much. Best regards, Martin From freza@REDACTED Sat Mar 31 14:45:45 2012 From: freza@REDACTED (Jachym Holecek) Date: Sat, 31 Mar 2012 08:45:45 -0400 Subject: [erlang-questions] Performing gen_server:call to multiple processes in parallel In-Reply-To: <4F76F62E.1040503@gmail.com> References: <4F76F62E.1040503@gmail.com> Message-ID: <20120331124545.GA6068@circlewave.net> # Martin Dimitrov 2012-03-31: > What I am trying to achieve is to call several gen_servers in parallel. > First, I thought I could use gen_server:multi_call/3 but reading the > documentation (if I understood it correctly) I realized the gen_servers > need to be on different nodes: one registered gen_server per node. > > Any ideas on how to achieve calling multiple processes on the same node > in parallel? Basically steal the body of gen:do_call/4 and adapt a bit. It's easy: for each destination process create a monitor and reuse the Mref as transaction reference, then send out request; when done schedule timeout and go to a loop receiving responses -- you'll probably want to accumulate one list with successes and other with failures. In the loop that accumulates results for you be very careful not to accept message that's not related to your multi_call -- the easiest way is to loop over generated Mrefs in sequence and block receive for respective response (or overall timeout). The Label value in gen:do_call/4 would be one of: '$gen_call' '$gen_sync_event' '$gen_sync_all_state_event' You want gen_server today, but will want gen_fsm calls tomorrow :-). HTH, -- Jachym From mrtndimitrov@REDACTED Sat Mar 31 19:11:19 2012 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Sat, 31 Mar 2012 20:11:19 +0300 Subject: [erlang-questions] Performing gen_server:call to multiple processes in parallel In-Reply-To: <20120331124545.GA6068@circlewave.net> References: <4F76F62E.1040503@gmail.com> <20120331124545.GA6068@circlewave.net> Message-ID: <4F773AB7.6000004@gmail.com> Thanks. After digging through gen:do_call/4 and erlang:monitor/2 I can see how to implement it. One question, why would gen_fsm be more appropriate? Thanks again. Best regards, Martin On 3/31/2012 3:45 PM, Jachym Holecek wrote: > # Martin Dimitrov 2012-03-31: >> What I am trying to achieve is to call several gen_servers in parallel. >> First, I thought I could use gen_server:multi_call/3 but reading the >> documentation (if I understood it correctly) I realized the gen_servers >> need to be on different nodes: one registered gen_server per node. >> >> Any ideas on how to achieve calling multiple processes on the same node >> in parallel? > Basically steal the body of gen:do_call/4 and adapt a bit. It's easy: > for each destination process create a monitor and reuse the Mref as > transaction reference, then send out request; when done schedule > timeout and go to a loop receiving responses -- you'll probably > want to accumulate one list with successes and other with failures. > > In the loop that accumulates results for you be very careful not to > accept message that's not related to your multi_call -- the easiest > way is to loop over generated Mrefs in sequence and block receive > for respective response (or overall timeout). > > The Label value in gen:do_call/4 would be one of: > > '$gen_call' > '$gen_sync_event' > '$gen_sync_all_state_event' > > You want gen_server today, but will want gen_fsm calls tomorrow :-). > > HTH, > -- Jachym