From zabrane3@REDACTED Wed Aug 1 00:14:16 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 1 Aug 2012 00:14:16 +0200 Subject: [erlang-questions] file:open/2: 'compressed' doesn't like like 'raw' mode Message-ID: Hi, This is OK: {ok, H} = file:open("foo.gz", [read, compressed]). {ok, _} = file:pread(H, 0, 4096). And this is not: {ok, H} = file:open("foo.gz", [read, raw, compressed]). file:pread(H, 0, 4096). {error,ebadf} Why? The doc doesn't warn about this limitation. Regards, Zabrane From cgsmcmlxxv@REDACTED Wed Aug 1 00:52:15 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Wed, 1 Aug 2012 00:52:15 +0200 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: <5017E5D5.2030508@gmail.com> References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> Message-ID: On Tue, Jul 31, 2012 at 4:04 PM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > On 07/31/2012 01:48 PM, Ian wrote: > >> << A "string" is a list of integers where the integers >>> represent Unicode codepoints. >> >>> >> >> I think this is technically correct, but it is very confusing because it >> implies that the source file may be encoded as unicode. >> >> As I understand it, source files are always treated as being in Latin-1. >> This means that string literals are lists of Latin-1 values, and not >> lists of unicode codepoints. (The values from 128 to 255 have >> different/no meanings, and values > 255 will not happen). >> > > No, you're confusing Unicode (a sequence of code points) with specific > encodings such as UTF-8 and UTF-16. The first is downwards compatible with > Latin-1: the values from 128 to 255 are the same. In UTF-8 they're not. At > runtime, Erlang's strings are just plain sequences of Unicode code points > (you can think of it as UTF-32 if you like). Whether the source code is > encoded in UTF-8 or Latin-1 or any other encoding is irrelevant as long as > the compiler knows how to transform the input to the single-codepoint > representation. > > For example, reversing a Unicode string is a bad idea anyway because it > could contain combining characters, and reversing the order of the > codepoints in that case will create an illegal string. But an expression > like lists:reverse("a?b") will be working on the list [97, 8734, 98] (once > the compiler has been extended to accept other encodings than Latin-1... Actually, try this: 1. set your environment to UTF-8 (in my case, whatever Linux terminal with BASH environment, export LANG="en_US.utf8", use locale to find your environment language definition - "en_US.latin1" for LATIN-1) 2. in a module: test_reverse(String) -> lists:reverse(String). 3. Give as parameter the example given by yourself. 4. Check the output. Pretty interesting to see how Erlang "knows" about UTF-8 encoding, isn't it? (You can try directly in the shell lists:reverse("a?b") and it will transform as expected (using 3-elements list).) Actually, it knows nothing about, but relying on the environment to extract the integers for the list (which it mimics here the knowledge about UTF-8). ...), not the list [97,226,136,158,98], so it will produce the intended > "b?a". This string might then become encoded as UTF-8 on its way to your > terminal, but that's another story. I would add to the last part ("on its way to your terminal") also "from" and not leaving only "on" (it seems that the both ways are valid even if that can break the code). I agree that for string literals, what you said is always true. CGS -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Wed Aug 1 01:55:38 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 1 Aug 2012 08:55:38 +0900 Subject: [erlang-questions] installing erlang documentation In-Reply-To: <20120731141336.10964@web002.roc2.bluetie.com> References: <20120731141336.10964@web002.roc2.bluetie.com> Message-ID: IIRC: you only need fop to generate the PDFs. If you don't need those, you can skip them. I ran across this problem myself, trying to build on a FreeBSD system, and just figured out how to specify skipping parts of the build. -michael turner On Wed, Aug 1, 2012 at 3:13 AM, 7stud <7stud@REDACTED> wrote: > Hi, > > I successfully installed Erlang on Mac OSX 10.6, after untaring and doing: > > ./configure > make > sudo make install > > and I've been practicing Erlang for a couple of weeks as I read "Erlang Programming". > > But I'm having no luck installing the erlang documentation. I downloaded the binary version of fop and set my JAVA_HOME environment variable to /Library/Java/Home, and I can successfully echo $JAVA_HOME, but the fop command is not recognized on the command line: > > -bash: fop: command not found > > so I assume that means that the elrang installation can't use the fop command to install the documentation either. At this point, I don't know what to do. > > $ java -version > java version "1.6.0_15" > Java(TM) SE Runtime Environment (build 1.6.0_15-b03-219) > Java HotSpot(TM) 64-Bit Server VM (build 14.1-b02-90, mixed mode) > $ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.eugene.turner@REDACTED Wed Aug 1 02:51:21 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 1 Aug 2012 09:51:21 +0900 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: <031E469A-2B7A-46E3-93A8-4A20A16E6DF2@masklinn.net> References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5017ED96.30606@gmail.com> <031E469A-2B7A-46E3-93A8-4A20A16E6DF2@masklinn.net> Message-ID: On Wed, Aug 1, 2012 at 12:13 AM, Masklinn wrote: > On 2012-07-31, at 16:37 , Richard Carlsson wrote: > >> On 07/31/2012 04:19 PM, Michael Turner wrote: >>>> At runtime, Erlang's strings are just plain sequences of Unicode code points >>>> (you can think of it as UTF-32 if you like). >>> >>> Can you go further and say that it actually *is* UTF-32? A footnote >>> like "[*] Basically, UTF-32; see ref XYZ for details" might be >>> helpful. >> >> I'm loath to say that it *is* UTF-32, because with that term follows a bunch of connotations such as word width and endianism, which don't apply to the representation as Erlang integers. I'd like to just refer to it as Unicode, but apparently that makes most people think it's either UTF-8 or UTF-16. > > Say it's a sequence of code points (reified as integers)? That's exactly > what it is. But as Richard Carlsson points out, that's NOT quite "exactly what it is." > ... If people don't know what a code point is, they can look it > up. In any case, this shouldn't bring along any undue semantic baggage > and misconception. The perfect is often the enemy of the good. Perfect precision is sometimes the enemy of good initial comprehension. In my experience, that's *definitely* true of most approaches to Unicode. I hope we haven't lost track of Joe's goal here: a reasonably accurate description of what Erlang strings are, in a passage that, at the very least, should not intimidate newbies. A quick and approximate gloss of "codepoint" in the main text, together with a footnote apologizing for the oversimplification and suggesting a more detailed reference, strikes me as the best compromise between precision and the need to appeal to the reader. Appealing to the reader is not exactly optional for Erlang. Every day, I hear more about Scala and Node.js. -michael turner From emmiller@REDACTED Wed Aug 1 03:38:36 2012 From: emmiller@REDACTED (Evan Miller) Date: Tue, 31 Jul 2012 21:38:36 -0400 Subject: [erlang-questions] Chicagoboss doesn't work?? In-Reply-To: References: <5018157a.8790980a.6c9c.0d78@mx.google.com> Message-ID: On Tue, Jul 31, 2012 at 1:31 PM, Paul Barry wrote: > That latest CB (0.8.0) runs fine on Mac OS X and MongoDB (for me), and > the latest Erlang (of course). > > This is also the *wrong list* to talk about this type of thing... as > CB has its own mailing list. Please join us here for CB discussion/debugging: https://groups.google.com/forum/?fromgroups#!forum/chicagoboss > > Paul. > > On 31 July 2012 18:26, dva.traktorista@REDACTED > wrote: >> Please be more verbose when you ask for help in the mailing list. >> Also, try to search for similar questions. >> For instance there was a good sir having troubles with launching ChicagoBoss >> on his Windows box recently. >> Answering to the two questions you asked in your mail: >> 1. ChicagoBoss does work. >> 2. Yes, there are people here (may be even myself) who can help you. >> >> Jonn Mostovoy (on the road), >> DA234FE7 >> >> >> ----- Reply message ----- >> From: "Winston S?nchez" >> Date: Tue, Jul 31, 2012 20:15 >> Subject: [erlang-questions] Chicagoboss doesn't work?? >> To: "erlang-questions" >> >> Hi everybody. >> >> I have problems with chicagoboss when i compile them, the application does >> not load routes and controllers don't work. >> >> Can anybody help me ?? >> -- >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dan@REDACTED Wed Aug 1 03:42:29 2012 From: dan@REDACTED (Daniel Dormont) Date: Tue, 31 Jul 2012 21:42:29 -0400 Subject: [erlang-questions] Process migration/ Process state extraction/ injection In-Reply-To: References: <20120731175513.GA21145@localhost> Message-ID: One of the first things I did in Erlang was implement node migration for a certain type of Ejabberd process similar to the one below, but less general and I entirely failed to implement message relaying. I had a few things going for me: - the process was a gen_fsm, so there was no mystery about how to obtain its state at a given moment in time. In fact the module in question already had handle_sync_event capable of setting or getting the state as a whole (normally bad from a design standpoint I suppose, but it was helpful here) - the process was created by a simple_one_for_one supervisor but had no other links or monitors - the process did not own any ports or ETS tables - no other processes kept direct references to it. Messages sent to it were immediately preceded by a Mnesia lookup to get its PID. So all I had to do was spawn the process on the new node, push its gen_fsm state, update the Mnesia table to point to the new PID, and kill the old one. There was still a risk of losing messages; a relay would be the right way to handle that I think. Dan On Tue, Jul 31, 2012 at 4:18 PM, Ali Sabil wrote: > Hi Tyron, > > You could implement process migration yourself, I remember seeing > something like that implemented in ejabberd: > > https://github.com/esl/ejabberd/blob/master/apps/ejabberd/src/p1_fsm.erl#L557 > > The idea is to spawn another process in the remote node, and use a > handover protocol to coordinate the migration. > > Best, > Ali > > On Tue, Jul 31, 2012 at 8:21 PM, Tyron Zerafa > wrote: > > Hey again, > > I am thinking that a weaker form of such mobility might suffice for > my > > project, but I am not sure. In this weaker form, it is enough to transfer > > the code and execution can restart again. I need this primary to > implement > > the code-on-demand and remote-evaluation architectures. I do not believe > > that this is so complex in Erlang, in fact if I remember correctly some > > functionality already exists for such. > > > > > > On Tue, Jul 31, 2012 at 8:15 PM, Gleb Peregud > wrote: > >> > >> On Tue, Jul 31, 2012 at 8:11 PM, Tyron Zerafa > >> wrote: > >> > Hey, > >> > My primary intention is to implement strong mobility of processes. > >> > Let's > >> > say that process A is running on Node 1, I want to be able to suspend > >> > this > >> > process, transfer it to Node 2 and resume such process from where it > >> > halted. > >> > I believe that in order to achieve such I need to somehow preserve the > >> > stack > >> > trace, memory and other info. > >> > >> This task is non-trivial and will require a lot of work with ERTS and > >> whole Erlang VM. Things you have to handle are: > >> - stack > >> - heap > >> - binary refs > >> - monitors > >> - links > >> - ets ownership > >> - port ownership > >> - messages sent to old process pid? > >> - replacing old pid with new in other processes? > >> - replacing old process with "replay" process? > >> > >> And I'm sure that those are not all details which will have to be > handled. > >> > >> Cheers, > >> Gleb > > > > > > > > > > -- > > Best Regards, > > Tyron Zerafa > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Aug 1 03:56:49 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 1 Aug 2012 13:56:49 +1200 Subject: [erlang-questions] unicode in string literals In-Reply-To: References: <50168ABD.50304@gmail.com> Message-ID: On 31/07/2012, at 7:36 PM, Vlad Dumitrescu wrote: > By the question above, do you mean to imply that '-encoding(...)' will > allow mixed encodings in a project, which is not a reasonable > alternative? It's not clear to me what you mean by a 'project', but why should a module written by someone who wants comments in M?ori (note the macron? Latin-4 or Unicode needed) use a module written by someone who wants comments in Swedish? It's no worse (and no better!) than having a 'project' where some of the files assume tabs are set every 8 characters and some of them assume tabs are set every 4 characters. It's a thing you need written down; it's a thing your tools need to understand; and it's a situation that doesn't need to persist with sources that are under your control. > I don't think that would be the single problem, but also all the code > that assumes that source code is latin-1. Also, tools that handle > source code will need to be able to recognize both the old and new > encodings, as they might need to have to work with an older version of > a file, before the conversion. The whole point of an -encoding directive is that it is something that syntaxtools should handle; by the time your code gets an AST or a token list, encodings are entirely a thing of the past. Gambit Scheme allows different files in a program to use different encodings. It's no big deal: _only_ the code that converts between a stream of bytes and a stream of characters knows anything about encodings; internally it's all Unicode. I haven't done this yet for my Smalltalk compiler because there are other more urgent issues (like working around C compilers that are trying to be helpful but fail), but the design work is done and it should leave the tokeniser running at about the same speed as the old Latin-1-only tokeniser. There *will* be a period when I want to keep my old Latin-1 files (don't fix what isn't broken) but want to start using Unicode in new work. SWI Prolog actually lets you change the encoding within a file, which sounds crazy but maybe Jan wanted the machinery to be there in case someone wanted ISO 2022 support. (Because that's basically what 2022 *is*: switching encoding aspects on the fly.) Why should a Japanese programmer be forbidden to write in her own script just because some of the source files that get loaded at run time are encoded in Latin 1? > > Another question that needs to be answered is also what encoding will > the source code use outside strings and quoted atoms and comments "Encoding" is a whole-file property. If the comments are encoded in ISO 8859-5 (ISO Cyrillic), so are the strings, and if the strings are encoded in ISO 8859-5, so are the atoms, both quoted and unquoted. Encoding logically concerns the interface between the tokeniser and the external byte stream (in the Unisys ClearPath MCP systems translation between encodings is done by the operating system before the data become available to the program). Once the changeover has been made, the tokeniser should think that *all* characters are Unicode characters. > : do > we want atoms and variable names to be utf8 too? Because I've seen at > least an example of code that uses extended latin-1 characters in > those places. That's not a problem. If a file is encoded in ISO Latin 1, then certain Unicode characters are encoded a certain way, BUT once into the tokeniser, nobody knows or cares what that was. If another file is encoded in UTF-8, then certain Unicode characters are encoded in a different way, BUT once into the tokeniser, nobody knows or cares what that was. Encode "(a?2)?4 = ?a" as 28,61,47,32,29,f7,34,20,3d,20,bd,61 (Latin-1) or as 28,61,c3,97,32,29,c3,b7,34,20,3d,20,c2,bd,61 (UTF-8), and as long as the tokeniser knows what it's getting, it should make *no* difference to what you get, namely the list [40,97,215,50,41,247,52,32,61,32,189,97] of integers one per Unicode code-point. That's how it works in SWI Prolog. > Also, what should string manipulation functions do by default, should > they assume an encoding? No. That would make life insanely complicated. (Well, let's face it, Unicode is already barking mad; this would make it *rabid* barking mad.) > I think the only way to remain sane would be > to have a special string type, tagged with the encoding No, that's a way to go completely crazy. The simple way is to distinguish between an inside and an outside. INSIDE, everything is just Unicode. OUTSIDE is where the wild things are. Encodings are *ONLY* relevant when you switch between text encoded as byte sequences and text represented as Unicode code point sequences. I mean, can you *imagine* the complexity if "0" =:= "0" fails because the first is tagged as Latin-1 and the second is tagged as UTF-8? How Unicode code-point sequences are represented inside the machine-level representation of an Erlang atom, Erlang source code should have no reason whatever to care. They could be UTF8; they could be UTF16; they could be SCSU; they could be BOCU; they could be something else entirely. Converting between strings and binaries is the one place where Erlang source code should have any reason to care, and it does have a reason to care. But you will perceive that it is the *binary* that needs to be associated with an encoding, not the *string*. of the system > > Would a syntactic construct like u"some string" that returns a tagged > utf8 string help? No. However, <<"some string"/utf8>> *would* make sense. From 7stud@REDACTED Wed Aug 1 05:11:17 2012 From: 7stud@REDACTED (7stud) Date: Tue, 31 Jul 2012 23:11:17 -0400 Subject: [erlang-questions] installing erlang documentation Message-ID: <20120731231117.24982@web006.roc2.bluetie.com> -----Original Message----- From: "Michel Rijnders" [g.a.c.rijnders@REDACTED] Date: 07/31/2012 03:02 PM To: "7stud" <7stud@REDACTED> CC: erlang-questions@REDACTED Subject: Re: [erlang-questions] installing erlang documentation On Tue, Jul 31, 2012 at 8:13 PM, 7stud <7stud@REDACTED> wrote: > Hi, > > I successfully installed Erlang on Mac OSX 10.6, after untaring and doing: > > ./configure > make > sudo make install > > and I've been practicing Erlang for a couple of weeks as I read "Erlang Programming". > > But I'm having no luck installing the erlang documentation. I downloaded the binary version of fop and set my JAVA_HOME environment variable to /Library/Java/Home, and I can successfully echo $JAVA_HOME, but the fop command is not recognized on the command line: > > -bash: fop: command not found > > so I assume that means that the elrang installation can't use the fop command to install the documentation either. At this point, I don't know what to do. You probably know this, but just to be sure, both HTML documentation and man pages are available as separate downloads from: http://www.erlang.org/download.html > ====== Thanks, I didn't know about the docs download. But once again I am having trouble accessing the docs. I can certainly untar the file: otp_doc_man_R15B01.tar.gz ...but then what?? The README file gives no hints about how to install the docs. And this is what I get: ~/Downloads/otp_docs_man$ erl -man lists No manual entry for lists From norton@REDACTED Wed Aug 1 05:29:25 2012 From: norton@REDACTED (Joseph Wayne Norton) Date: Wed, 1 Aug 2012 12:29:25 +0900 Subject: [erlang-questions] installing erlang documentation In-Reply-To: <20120731231117.24982@web006.roc2.bluetie.com> References: <20120731231117.24982@web006.roc2.bluetie.com> Message-ID: <93574725-4267-454B-9943-F7855F9166CC@lovely.email.ne.jp> I'm not sure if this has been mentioned by someone else before but "kerl" makes it really easy to download, to build, and to install your own Erlang system. Here are the basics needed to get going quickly. $ kerlinstall $ kerlbuildotp $ kerlinstallotp Please customise as you see fit. $ alias | grep kerl alias kerlbuildotp='kerl build R13B04 r13b04; kerl build R14B04 r14b04; kerl build R15B01 r15b01' alias kerlinstall='mkdir -p ~/bin; wget -q -O - https://raw.github.com/spawngrid/kerl/master/kerl > ~/bin/kerl; chmod a+x ~/bin/kerl; kerl update releases' alias kerlinstallotp='kerl install r13b04 ~/.kerl/installations/r13b04; kerl install r14b04 ~/.kerl/installations/r14b04; kerl install r15b01 ~/.kerl/installations/r15b01' alias kerlunbuildotp='kerl delete build r13b04; kerl delete build r14b04; kerl delete build r15b01' alias kerluninstallotp='kerl delete installation ~/.kerl/installations/r13b04; kerl delete installation ~/.kerl/installations/r14b04; kerl delete installation ~/.kerl/installations/r15b01' $ cat ~/.kerlrc # -*- Sh -*- # vim: set syntax=sh: KERL_DISABLE_AGNER="yes" KERL_USE_AUTOCONF="yes" KERL_INSTALL_MANPAGES="yes" KERL_INSTALL_HTMLDOCS="yes" KERL_CONFIGURE_OPTIONS="--enable-kernel-poll --enable-threads --enable-dynamic-ssl-lib --enable-shared-zlib --enable-smp-support" case `uname -s` in Darwin) case `uname -m` in x86_64) KERL_CONFIGURE_OPTIONS="$KERL_CONFIGURE_OPTIONS --enable-darwin-64bit" ;; *) KERL_CONFIGURE_OPTIONS="$KERL_CONFIGURE_OPTIONS --enable-darwin-universal" ;; esac ;; esac KERL_DEPLOY_SSH_OPTIONS="-qx -o PasswordAuthentication=no" KERL_DEPLOY_RSYNC_OPTIONS="--delete" On Aug 1, 2012, at 12:11 PM, "7stud" <7stud@REDACTED> wrote: > > > -----Original Message----- > From: "Michel Rijnders" [g.a.c.rijnders@REDACTED] > Date: 07/31/2012 03:02 PM > To: "7stud" <7stud@REDACTED> > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] installing erlang documentation > > On Tue, Jul 31, 2012 at 8:13 PM, 7stud <7stud@REDACTED> wrote: >> Hi, >> >> I successfully installed Erlang on Mac OSX 10.6, after untaring and doing: >> >> ./configure >> make >> sudo make install >> >> and I've been practicing Erlang for a couple of weeks as I read "Erlang Programming". >> >> But I'm having no luck installing the erlang documentation. I downloaded the binary version of fop and set my JAVA_HOME environment variable to /Library/Java/Home, and I can successfully echo $JAVA_HOME, but the fop command is not recognized on the command line: >> >> -bash: fop: command not found >> >> so I assume that means that the elrang installation can't use the fop command to install the documentation either. At this point, I don't know what to do. > > You probably know this, but just to be sure, both HTML documentation > and man pages are available as separate downloads from: > http://www.erlang.org/download.html > >> > > ====== > > Thanks, I didn't know about the docs download. But once again I am having trouble accessing the docs. I can certainly untar the file: > > otp_doc_man_R15B01.tar.gz > > ...but then what?? The README file gives no hints about how to install the docs. And this is what I get: > > ~/Downloads/otp_docs_man$ erl -man lists > No manual entry for lists > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From 7stud@REDACTED Wed Aug 1 06:05:52 2012 From: 7stud@REDACTED (7stud) Date: Wed, 01 Aug 2012 00:05:52 -0400 Subject: [erlang-questions] installing erlang documentation Message-ID: <20120801000552.10327@web005.roc2.bluetie.com> I figured out how to install the erlang man documents: 1) Google "installing man pages for erlang": http://www.erlang.org/download/otp_src_R9C-0.readme 2) Scroll down to the section: "How to install the Erlang/OTP documentation": ======= For some graphical tools to find the on-line help you have to install the HTML documentation on top of the installed OTP applications, i.e. cd /lib/erlang gunzip -c otp_html_R9C-0.tar.gz | tar xf - For "erl -man " to work the Unix manual pages have to be installed in the same way, i.e. cd /lib/erlang gunzip -c otp_man_R9C-0.tar.gz | tar xf - ======== 3) From reading some other stuff in the document, I understood the default to be /usr/local. 4) cd /usr/local/lib/erlang tar xfvz ~/Downloads/otp_doc_man_R15B01.tar.gz 5) erl -man lists Thanks. From ok@REDACTED Wed Aug 1 06:14:00 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 1 Aug 2012 16:14:00 +1200 Subject: [erlang-questions] unicode in string literals In-Reply-To: References: <50168ABD.50304@gmail.com> <501792E6.1020805@ninenines.eu> Message-ID: <88047BF7-4507-470E-A5F5-65D24945EDF7@cs.otago.ac.nz> On 31/07/2012, at 9:36 PM, CGS wrote: > There are many pros and cons for switching from Latin-1 to UTF-8 (or whatever else which will nullify pretty much the understanding of byte character). On one hand, lists:reverse/1 really messes up the characters in the list Yes, and that's not all it messes up by any means. - If you have a sequence of lines represented as a string with network line terminators (CR+LF) then the reversal of that list is NOT a sequence of lines with network line terminators (applies to ASCII) - If you use Unicode language tags, then the reversal of a language tag is a language tag for a different language and applies to the wrong characters - The reversal of a Unicode string including variant selectors (or other character shaping codes like ZWNJ or ZWJ) is a Unicode string including variant selectors &c applied to the wrong characters - The reversal of a Unicode string including a directional command and a POP DIRECTIONAL FORMATTING code is a string in which there is a POP before anything has been pushed. ... So simply forming code points into [base,diacritical...] packets, reversing the packets, and then flattening *still* isn't nearly enough to make sense of a reversed string. Indeed, I am not sure that there *is* any way to make sense of the notion of reversing a Unicode string. So I do not take 'lists:reverse/1 will not reverse a Unicodepoint string correctly' as a criticism of representing strings as lists of Unicodepoints. NOTHING will. I don't think there is any such thing as "correctly" reversing such a string. There are other operations you can easily do with a list that don't make sense for Unicode strings either. Take just one example: splitting a string at an arbitrary position. That can separate a directional override from its pop. And having a distinct data type is no protection against that problem: Java and Javascript both have opaque string datatypes, but both allow slicing a well formed string into pieces that are not well formed. > (to follow the first example, the output of "a?b" in Latin-1 is totally different from the output of lists:reverse("b?a") in Latin-1 - the default now). On the other hand, having, for example, Polish characters like "? ? ?" or French "? ?" or German "? ?" or Turkish "?" and so on (things become more complicated if we add languages based on different alphabet/symbols) in the code would require your editor to have support for those languages or else you will see really strange characters there. Well, yes. But now you are asking whether the editor supports Unicode. There are now plenty of editors that do. Right now I am composing mail in an unbelievably crude text editor (the Mail program on Mac OS X) and it displays these characters just fine. > > "-encoding()" can make quite a mess in a file. Think of an open source project in which devs from different countries append their own code. You will see a lot of "-encoding()" directives in a single file. Nobody is suggesting that there should be an -encoding directive anywhere but the first line of a file (or possibly the second). In fact it is precisely the existence of -encoding directives that would make it possible to *avoid* the mess you are describing. Here's what you do. (1) Write a tiny little program. Here is a first draft. #!/usr/bin/awk -f # Usage: epaste.awk file1.erl... >pasted.erl # Purpose: paste files in various encodings giving one file in UTF-8. BEGIN { print "-encoding(utf_8)." for (i = 1; i < ARGC; i++) { input = ARGV[i] getline x < input if (x ~ /^[ \t]*-[ \t]*encoding\([ \t']*[a-zA-Z0-9_]*[ \t']*\)/) { sub(/^[ \t]*-[ \t]*encoding\([ \t']*/, "", x) sub(/[ \t']*\).*$/, "", x) x = toupper(x) gsub(/_/, "-", x) cmd = "iconv -f " x " -t UTF-8" } else { cmd = "iconv -f ISO-8859-1 -t UTF-8" print x >cmd } while ((getline x 0) print x >cmd close(cmd) } } (2) Instead of pasting together several files by doing cat foo.erl ugh.erl bar.erl >fub.erl just do epaste.awk foo.erl ugh.erl bar.erl >fub.erl What makes this *possible* is the existence of the -encoding lines. Without it you are FUBAR. > I might be wrong, but, switching to default UTF-8, wouldn't that force the compiler to use 2-byte (at least) per character? Yes, you are wrong. Unicode is a 21-bit character set. There are currently (6.1) more than 100,000 defined characters, so 2 bytes is definitely not enough. But UTF-8 is an *external* format. What the compiler uses is entirely up to itself. What the run-time system uses is something different again. Atom names, for example, could be stored in some compressed format. > If so, for example, what about the databases based on Erlang for projects using strict Latin-1? What about them? Do not make the mistake of confusing a particular set of characters with a way of encoding them. From ok@REDACTED Wed Aug 1 06:33:08 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 1 Aug 2012 16:33:08 +1200 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: References: Message-ID: On 31/07/2012, at 9:53 PM, Michael Turner wrote: >> << An Erlang "string" is simply a list of integers. Each integer can >> represent any Unicode codepoint/character. >> > > Except that Unicode codepoints represents characters, right? Wrong. One Unicode codepoint may represent what a particular language views as two distinct graphemes. (This occurs in encoding English, for example: in 'belov?d' the diacritical mark is a stress accent and so ? counts as two separate graphemes.) One grapheme may require two or more Unicode codepoints. Some characters, well, 26FD FE0E is a black-and white picture of a petrol pump, but 26FD FE0F is a colour version. Either of these is perceived by users as a single 'character'. FE0E represents "select text style for the previous thingy"; FEOF represents "select emoji style for it". You'd be hard pressed to call either FE0E or FE0F a "character". The majority of codepoints represent nothing at all (yet). The thing people *still* don't get about Unicode is that with ASCII and EBCDIC and Latin-1 there really was such a thing as a "character" that a string was a sequence of, but in Unicode, a string is *not* a sequence of characters but a *well-formed* sequence of codepoints. You *can't* represent the "emoji style FUEL PUMP" <> by a single number, only by a *sequence* of codepoints. I keep meaning to write a small book called "Strings Made Difficult." From the Unicode FAQ: Q: So is a combining character sequence the same as a ?character?? A: That depends. For a programmer, a Unicode code value represents a single character (for exceptions, see below). For an end user, it may not. The better word for what end-users think of as characters is grapheme (as defined in the Unicode glossary): a minimally distinctive unit of writing in the context of a particular writing system. ... Q: How should characters (particularly composite characters) be counted, for the purposes of length, substrings, positions in a string, etc.? A: In general, there are 3 different ways to count characters. Each is illustrated with the following sample string. ?a? + umlaut + greek_alpha + \uE0000. (the latter is a private use character) 1. Code Units: e.g. how many bytes are in the physical representation of the string. Example: In UTF-8, the sample has 9 bytes. [61 CC 88 CE B1 F3 A0 80 80] In UTF-16BE, it has 10 bytes. [00 61 03 08 03 B1 DB 40 DC 00] In UTF-32BE, it has 16 bytes. [00 00 00 61 00 00 03 08 00 00 03 B1 00 0E 00 00] 2. Codepoints: how may code points are in the string. The sample has 4 code points. This is equivalent to the UTF-32BE count divided by 4. 3. Graphemes: what end-users consider as characters. A default grapheme cluster is specified in UAX #29, Unicode Text Segmentation, as well as in UTS #18, Unicode Regular Expressions. The choice of which one to use depends on the tradeoffs between efficiency and comprehension. For example, Java, Windows and ICU use #1 with UTF-16 for all low-level string operations, and then also supply layers above that provide for #2 and #3 boundaries when circumstances require them. This approach allows for efficient processing, with allowance for higher-level usage. However, for a very high level application, such as word-processing macros, graphemes alone will probably be sufficient. Q > You can't > have a representation of a representation.[*] > > I suggest: > > << In Erlang, strings are represented as lists of integers. These > integers are Unicode codepoints, each representing a character. >> > > That way, anybody who's unclear on what "codepoint" means gets a > freebie definition of it. In the Unicode context, it's probably wrong, > technically, but perhaps good enough for this purpose. > > -michael turner > > [*] Douglas Hofstadter might beg to differ, but he's not on this list. > > > On Tue, Jul 31, 2012 at 6:41 PM, Paul Barry wrote: >> Hi Joe. >> >> I think "string literal" is pretty widely understood (it even has a >> WikiPedia entry, here: http://en.wikipedia.org/wiki/String_literal). >> >> What threw me about your sentence was the use of the word 'codepoint', >> which will be OK for those already familiar with Unicode, but might >> confuse those who are not. My feeling (and this might be a gross >> over-simplification) is that most North-American programmers know >> about Unicode but don't let it worry them too much, resulting in less >> of a familiarity with it than might be necessary (and I apologize to >> any North-American programmers that this comment rubs the wrong way). >> Perhaps "unicode characters" might be easier to read/understand? >> Although not probably totally technically correct... >> >> Another thing that you might wish to consider is breaking the sentence >> in two, as follows: >> >> << An Erlang "string" is simply a list of integers. Each integer can >> represent any Unicode codepoint/character. >> >> >> Just my 2 cent. >> >> Paul. >> >> On 31 July 2012 10:24, Joe Armstrong wrote: >>> I'm working on a 2'nd edition of my book, and have got to strings :-) >>> Strings confuse everybody, including me so I have a few questions: >>> >>> To start with Erlang doesn't have strings - it has lists (not strings) >>> and it has string literals. >>> >>> I want to define a string - is this correct: >>> >>> << A "string" is a list of integers where the integers >>> represent Unicode codepoints. >> >>> >>> Questions: >>> Is the sentence inside << .. >> using the correct terminology? >>> If not what should it say? >>> >>> Is the sentence inside << ... >> widely understood, do you think this >>> would confuse a lot of people? >>> >>> Is the phrase "string literal" widely understood? >>> >>> >>> Cheers >>> >>> /Joe >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> -- >> Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED >> Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Wed Aug 1 06:45:15 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 1 Aug 2012 16:45:15 +1200 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: <50180C83.5020504@gmail.com> References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5017F77B.502@ferd.ca> <5017FEB3.3090001@gmail.com> <501801B5.3080208@ferd.ca> <50180C83.5020504@gmail.com> Message-ID: On 1/08/2012, at 4:49 AM, Richard Carlsson wrote: > Yes. That's why there needs to be a new Unicode-aware string library. Operating directly on lists (e.g. using lists:reverse/1, or even length/1) is always going to have surprising effects, and the old 'string' module in stdlib probably can't be modernized while maintaining backwards compatibility. It should be noted that the "length" of a Unicode string is an inherently ambiguous concept anyway. It makes sense to ask how many codepoints there are in the string as given, or how many there would be in a particular normalisation form, but neither of those is how many characters the *user* would count (which is not just script-sensitive, not just locale-sensitive, but very context-sensitive). Oh, and none of them is the same as "how many columns would this require in a fixed-width font." From ok@REDACTED Wed Aug 1 07:13:50 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 1 Aug 2012 17:13:50 +1200 Subject: [erlang-questions] installing erlang documentation In-Reply-To: <20120731141336.10964@web002.roc2.bluetie.com> References: <20120731141336.10964@web002.roc2.bluetie.com> Message-ID: <27275676-2EA7-4350-8BB0-97A074B14B70@cs.otago.ac.nz> On 1/08/2012, at 6:13 AM, 7stud wrote: > But I'm having no luck installing the erlang documentation. I downloaded the binary version of fop and set my JAVA_HOME environment variable to /Library/Java/Home, and I can successfully echo $JAVA_HOME, but the fop command is not recognized on the command line: > > -bash: fop: command not found > > so I assume that means that the elrang installation can't use the fop command to install the documentation either. At this point, I don't know what to do. (A) Search the archives. This problem is already documented on the Erlang web site. When I ran into it, I installed fop. http://xmlgraphics.apache.org/fop/download.html I then ran into further problems, which weren't that hard to patch around, but which I don't remember. I'd have to search the Erlang mailing list archives too. (B) When I got fop going, the output was wrongly formatted. A lot of the spaces in C prototypes were missing. And they were missing in all generated formats. So my recommendation is to visit http://www.erlang.org/download.html and download the HTML Documentation File and the Man Pages File and use those. From michal.piotrowski@REDACTED Wed Aug 1 08:37:17 2012 From: michal.piotrowski@REDACTED (=?utf-8?Q?Micha=C5=82_Piotrowski?=) Date: Wed, 1 Aug 2012 08:37:17 +0200 Subject: [erlang-questions] Process migration/ Process state extraction/ injection In-Reply-To: References: Message-ID: <661E5FD6-A7F7-4C42-BF84-10CC50085931@erlang-solutions.com> Hi, I'm currently working on so-colled proc_mobility behavior. It is a prat of my thesis which aims to add agent migration to eXAT platform. proc_mobility doesn't depend on eXAT platform so can be used in whatever you want. Here is link to the repository https://github.com/michalwski/proc_mobility/tree/over_tcp. The brach "over_tcp" has more features than master. Unfortunately I haven't write any documentation so far, so feel free to ask if you think that proc_mobility can be useful. Best regards, Micha? Piotrowski On Jul 31, 2012, at 2:29 PM, Tyron Zerafa wrote: > Hello everyone, > Does Erlang support process migration? Meaning, is there a way in which I can stop a running process (A) on one node and resume it on another completely different node? > If this is not supported, one way to go around it is to extract the process state before suspending it and somehow inject it into the new process on the remote node. Does Erlang provide such functionality? > > Any tips and suggestions on how one can extract/ inject the process state from/ into processes? > > - Thanks in advance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From Sergey_Zhemzhitsky@REDACTED Wed Aug 1 08:47:59 2012 From: Sergey_Zhemzhitsky@REDACTED (Zhemzhitsky Sergey) Date: Wed, 1 Aug 2012 06:47:59 +0000 Subject: [erlang-questions] Sending messages to a process from the port driver from a separate thread Message-ID: <06139A918ACCA041BF46A0F36940C7FA39D04C06@exch-mbx1.msk.trd.ru> Hi erlang gurus, Is it legal to send messages to any erlang process by means of driver_send_term from a separate thread started by erl_drv_thread_create? I.e. can I send messages to the process not from the port driver callback, but at any time from the function specified when invoking erl_drv_thread_create? Best Regards, Sergey _______________________________________________________ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp -------------- next part -------------- An HTML attachment was scrubbed... URL: From masklinn@REDACTED Wed Aug 1 08:57:43 2012 From: masklinn@REDACTED (Masklinn) Date: Wed, 1 Aug 2012 08:57:43 +0200 Subject: [erlang-questions] unicode in string literals In-Reply-To: <88047BF7-4507-470E-A5F5-65D24945EDF7@cs.otago.ac.nz> References: <50168ABD.50304@gmail.com> <501792E6.1020805@ninenines.eu> <88047BF7-4507-470E-A5F5-65D24945EDF7@cs.otago.ac.nz> Message-ID: <411474AB-6810-4C9C-8014-3A1310D60547@masklinn.net> On 2012-08-01, at 06:14 , Richard O'Keefe wrote: > And having a > distinct data type is no protection against that problem: Java > and Javascript both have opaque string datatypes, but both > allow slicing a well formed string into pieces that are not > well formed. To be fair, they've got the further compounding issue that strings types are dedicated but not opaque: they are sequences of UTF-16 code units (on account of originally being UCS2 sequences). As a result, not only do you have the usual Unicode issues which may or may not be (non-trivially) solvable (with grapheme-aware unicode handling[0]) that's further compounded by the ability to see and break apart surrogate pairs (so you can e.g. split a string in the middle of a surrogate pair). CPython 3.3 has implemented a fully opaque string type, it exposes unicode codepoints (if I remember correctly) but that may or may not be the underlying binary data (the underlying representation can dynamically switch between latin-1, UCS2 and UCS4) [0] Which also needs to be locale-aware, for instance a conversion to lower/upper case is not a 1:1 mapping in unicode as different cultures may have different uppercases for the same lower and the other way around, the usual example being Turkish in which "I"'s lowercase is "?" and the uppercase of "i" is "?") From vladdu55@REDACTED Wed Aug 1 09:30:19 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 1 Aug 2012 09:30:19 +0200 Subject: [erlang-questions] unicode in string literals In-Reply-To: References: <50168ABD.50304@gmail.com> Message-ID: Hi Richard, First, thanks for the detailed explanation. I see I am still confusing some of the issues. On Wed, Aug 1, 2012 at 3:56 AM, Richard O'Keefe wrote: > On 31/07/2012, at 7:36 PM, Vlad Dumitrescu wrote: > It's not clear to me what you mean by a 'project', I mean a set of related code, some of it possibly third-party. > but why should a module written by someone who wants > comments in M?ori (note the macron? Latin-4 or Unicode needed) > use a module written by someone who wants comments in Swedish? Maybe not in the long run, but there will be a (long) transition period where legacy code will still be used by new code. > The whole point of an -encoding directive is that it is something > that syntaxtools should handle; by the time your code gets an AST > or a token list, encodings are entirely a thing of the past. Yes, but I am one of the guys that is going to write some of the tools that will handle this conversion, so I do care about the details. > SWI Prolog actually lets you change the encoding within a file, > which sounds crazy but maybe Jan wanted the machinery to be there > in case someone wanted ISO 2022 support. (Because that's basically > what 2022 *is*: switching encoding aspects on the fly.) Are there any editors that can load/save a file with mixed encodings like that? <...snip...> > Converting between strings and binaries is the one place where Erlang > source code should have any reason to care, and it does have a reason > to care. But you will perceive that it is the *binary* that needs to > be associated with an encoding, not the *string*. > of the system Right. Good explanation! I am still a little worried about two things: - debugging a remote system that has different locale - reading logs created by modules that have different encodings (some modules might be legacy and not be aware that the world is not Latin-1 anymore). regards, Vlad From michael.eugene.turner@REDACTED Wed Aug 1 09:32:57 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 1 Aug 2012 16:32:57 +0900 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: References: Message-ID: On Wed, Aug 1, 2012 at 1:33 PM, Richard O'Keefe wrote: > > On 31/07/2012, at 9:53 PM, Michael Turner wrote: > >>> << An Erlang "string" is simply a list of integers. Each integer can >>> represent any Unicode codepoint/character. >> >> >> Except that Unicode codepoints represents characters, right? > > Wrong. Actually, what's *really* wrong in my statement is the grammar -- bad plural agreement. > One Unicode codepoint may represent what a particular language > views as two distinct graphemes. (This occurs in encoding English, > for example: in 'belov?d' the diacritical mark is a stress accent > and so ? counts as two separate graphemes.) [snip much more] I'm certain this is correct, Richard, but ... what problem are we trying to solve again? IIRC: Joe is trying to come up with a short passage that explains what strings are, in Erlang. If he writes all that you wrote above, the reader (who might have been initially excited about Erlang) will come away with the impression, "Erlang people are excruciatingly pedantic". > I keep meaning to write a small book called "Strings Made Difficult." Sounds like you're the man to do it, Richard. As I wrote earlier: >> << In Erlang, strings are represented as lists of integers. These >> integers are Unicode codepoints, each representing a character. >> >> >> That way, anybody who's unclear on what "codepoint" means gets a >> freebie definition of it. In the Unicode context, it's probably wrong, >> technically, but perhaps good enough for this purpose. Can anyone tell me why this *wouldn't* serve Joe's (== the typical reader's) purposes? [*] -michael turner [*] Why do I suspect we're now going to have a long digression on whether the "==" in "(== the typical reader's)" should really be "=:="? >> On Tue, Jul 31, 2012 at 6:41 PM, Paul Barry wrote: >>> Hi Joe. >>> >>> I think "string literal" is pretty widely understood (it even has a >>> WikiPedia entry, here: http://en.wikipedia.org/wiki/String_literal). >>> >>> What threw me about your sentence was the use of the word 'codepoint', >>> which will be OK for those already familiar with Unicode, but might >>> confuse those who are not. My feeling (and this might be a gross >>> over-simplification) is that most North-American programmers know >>> about Unicode but don't let it worry them too much, resulting in less >>> of a familiarity with it than might be necessary (and I apologize to >>> any North-American programmers that this comment rubs the wrong way). >>> Perhaps "unicode characters" might be easier to read/understand? >>> Although not probably totally technically correct... >>> >>> Another thing that you might wish to consider is breaking the sentence >>> in two, as follows: >>> >>> << An Erlang "string" is simply a list of integers. Each integer can >>> represent any Unicode codepoint/character. >> >>> >>> Just my 2 cent. >>> >>> Paul. >>> >>> On 31 July 2012 10:24, Joe Armstrong wrote: >>>> I'm working on a 2'nd edition of my book, and have got to strings :-) >>>> Strings confuse everybody, including me so I have a few questions: >>>> >>>> To start with Erlang doesn't have strings - it has lists (not strings) >>>> and it has string literals. >>>> >>>> I want to define a string - is this correct: >>>> >>>> << A "string" is a list of integers where the integers >>>> represent Unicode codepoints. >> >>>> >>>> Questions: >>>> Is the sentence inside << .. >> using the correct terminology? >>>> If not what should it say? >>>> >>>> Is the sentence inside << ... >> widely understood, do you think this >>>> would confuse a lot of people? >>>> >>>> Is the phrase "string literal" widely understood? >>>> >>>> >>>> Cheers >>>> >>>> /Joe >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> -- >>> Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED >>> Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From carlsson.richard@REDACTED Wed Aug 1 10:39:07 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 01 Aug 2012 10:39:07 +0200 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> Message-ID: <5018EB2B.5030700@gmail.com> On 08/01/2012 12:52 AM, CGS wrote: > Actually, try this: > > 1. set your environment to UTF-8 (in my case, whatever Linux terminal > with BASH environment, export LANG="en_US.utf8", use locale to find your > environment language definition - "en_US.latin1" for LATIN-1) > 2. in a module: > > test_reverse(String) -> lists:reverse(String). > > 3. Give as parameter the example given by yourself. > 4. Check the output. Ah, but when you say "give as parameter" you mean "pass it a string literal from the shell", right? I never said anything about strings in the shell - that's a different environment from source files, and as you described, the shell nowadays detects your locale and translates UTF-8 console input into a string literal containing Unicode code points. This is exactly how it would happen in source code as well, if the compiler only knew how to detect that a source file is in a different encoding from Latin1. So the compiler is really the main thing that needs to be fixed, and then there should be no surprises on the encoding level anymore. /Richard From max.lapshin@REDACTED Wed Aug 1 11:38:22 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 1 Aug 2012 13:38:22 +0400 Subject: [erlang-questions] erlang packages on erlang-solutions for wheezy is only for armh Message-ID: Intel architectures are lost From hobson42@REDACTED Wed Aug 1 11:52:28 2012 From: hobson42@REDACTED (Ian) Date: Wed, 01 Aug 2012 10:52:28 +0100 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: <5018EB2B.5030700@gmail.com> References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5018EB2B.5030700@gmail.com> Message-ID: <5018FC5C.70705@gmail.com> On 01/08/2012 09:39, Richard Carlsson wrote: > [the shell is] a different environment from source files, and as you > described, the shell nowadays detects your locale and translates UTF-8 > console input into a string literal containing Unicode code points. > This is exactly how it would happen in source code as well, if the > compiler only knew how to detect that a source file is in a different > encoding from Latin1. So the compiler is really the main thing that > needs to be fixed, and then there should be no surprises on the > encoding level anymore. Anyone any idea about timescales for making the compiler aware of the environment? Ian From cgsmcmlxxv@REDACTED Wed Aug 1 14:25:50 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Wed, 1 Aug 2012 14:25:50 +0200 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: <5018EB2B.5030700@gmail.com> References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5018EB2B.5030700@gmail.com> Message-ID: On Wed, Aug 1, 2012 at 10:39 AM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > On 08/01/2012 12:52 AM, CGS wrote: > >> Actually, try this: >> >> 1. set your environment to UTF-8 (in my case, whatever Linux terminal >> with BASH environment, export LANG="en_US.utf8", use locale to find your >> environment language definition - "en_US.latin1" for LATIN-1) >> 2. in a module: >> >> test_reverse(String) -> lists:reverse(String). >> >> 3. Give as parameter the example given by yourself. >> 4. Check the output. >> > > Ah, but when you say "give as parameter" you mean "pass it a string > literal from the shell", right? Yes. Sorry for the confusion. > I never said anything about strings in the shell - that's a different > environment from source files, and as you described, the shell nowadays > detects your locale and translates UTF-8 console input into a string > literal containing Unicode code points. The shell was just an example which is a part of the problem. Another part is the communication over HTTP/HTTPS protocol. And there may be some other examples. My point here is the interaction of Erlang with any environment which uses UTF-8. If Erlang compiler doesn't become aware of UTF-8, the code may be unstable without even understanding why. As you said, it seems that the compiler needs a fix. > This is exactly how it would happen in source code as well, if the > compiler only knew how to detect that a source file is in a different > encoding from Latin1. So the compiler is really the main thing that needs > to be fixed, and then there should be no surprises on the encoding level > anymore. I couldn't agree more. CGS -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Aug 1 16:12:48 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 1 Aug 2012 16:12:48 +0200 Subject: [erlang-questions] External sorting for large files in Erlang In-Reply-To: References: Message-ID: On Tue, Jul 31, 2012 at 11:32 PM, Zabrane Mickael wrote: > Hi, > > I'm looking for something similar to this, but in Erlang: > http://code.google.com/p/externalsortinginjava/ > > I found an old post suggesting file_sorter: > http://www.erlang.org/doc/man/file_sorter.html > But file_sorter seems to only work on binary files. This is one of my favorite modules - it is very fast. file_sorter sorts binary encoded terms. Each entry is a 4 byte length header followed by term_to_binary(Term) Here's an example of how to encode some terms, write them to a file sort the file and read them back. -- example -module(test1). -compile(export_all). test() -> L = [encode(I) || I <- [{yes,6,1},no, {yes,1,2}, {hello,22},{yes,12,10}, {hello,12}]], file:write_file("foo", L), file_sorter:sort("foo"), {ok, Bin} = file:read_file("foo"), decode(Bin). %% encode(Term) makes a 4 byte length header followed %% by term_to_binary(Term) encode(T) -> B = term_to_binary(T), Len = size(B), <>. decode(<>) -> T = binary_to_term(B), [T|decode(B2)]; decode(<<>>) -> []. --- end It happily sorts extremely large files .... well worth using > In my cas, I need something more flexible. > > What about controlling the Unix sort command from Erlang? os:cmd("sort out"). /Joe > > Any hints, ideas, suggestions, code? > > Regards, > Zabrane > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From sam@REDACTED Wed Aug 1 18:37:01 2012 From: sam@REDACTED (Sam Elliott) Date: Wed, 1 Aug 2012 17:37:01 +0100 Subject: [erlang-questions] Erlang Message Guarantees In-Reply-To: References: <9422D177-3F3A-4BC4-85F0-54FE4E4A88A5@lenary.co.uk> Message-ID: <1A3074C0-531C-4CE9-9B06-C2113B475B6D@lenary.co.uk> Fantastic, Thanks to both of you! Sam On 31 Jul 2012, at 18:49, Daniel Dormont wrote: > 10.9 above it as well - that has the answer to the question about delivery order. > > On Tue, Jul 31, 2012 at 1:37 PM, Gleb Peregud wrote: > Take a look here: http://www.erlang.org/faq/academic.html at 10.10 > > On Tue, Jul 31, 2012 at 7:35 PM, Sam Elliott wrote: > > Hi, > > > > Where I can find information on the inherent guarantees inbuilt into Erlang? > > > > I'm looking for information on whether order is preserved and associated issues. > > > > Sam > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Samuel Elliott sam@REDACTED +44 (0)7891 993 664 -- From zabrane3@REDACTED Wed Aug 1 18:47:50 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 1 Aug 2012 18:47:50 +0200 Subject: [erlang-questions] External sorting for large files in Erlang In-Reply-To: References: Message-ID: <7A623F1E-B92C-447E-AB36-B5E455BADFE5@gmail.com> Hey Joe, On Aug 1, 2012, at 4:12 PM, Joe Armstrong wrote: > On Tue, Jul 31, 2012 at 11:32 PM, Zabrane Mickael wrote: >> Hi, >> >> I'm looking for something similar to this, but in Erlang: >> http://code.google.com/p/externalsortinginjava/ >> >> I found an old post suggesting file_sorter: >> http://www.erlang.org/doc/man/file_sorter.html >> But file_sorter seems to only work on binary files. > > This is one of my favorite modules - it is very fast. > > file_sorter sorts binary encoded terms. > Each entry is a 4 byte length header followed by term_to_binary(Term) > > Here's an example of how to encode some terms, write them to a file > sort the file and read them back. Thanks for sharing this code. Very useful. > It happily sorts extremely large files .... well worth using It seems to work fine, but not yet very flexible. Can we imagine a module on top of file_sorter which will mimics the Unix sort command to work on plain text files (and not binary)? At the end, I wanna be able to sort any disk file as with Unix sort. > In my case, I need something more flexible. >> What about controlling the Unix sort command from Erlang? > > os:cmd("sort out"). Yep. That's what I had in mind (or port command). Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From brum76@REDACTED Wed Aug 1 19:34:39 2012 From: brum76@REDACTED (Radu Brumariu) Date: Wed, 1 Aug 2012 13:34:39 -0400 Subject: [erlang-questions] yecc question Message-ID: Hi, let's say that I am parsing a string and that string has a backreference to something previously seen. like this : {key: 'nameA', value: n1}, {key: 'nameB', value:n3}, {key: 1, value:n2}, {key: 2, value:n4} here the values of key of 1 and 2 are backreferences to the 'nameA' and 'nameB'. Is there a way to lookup the value of reference 1 during parsing, rather than afterwards ? Thanks, Radu From greg@REDACTED Wed Aug 1 20:14:53 2012 From: greg@REDACTED (Greg Martin) Date: Wed, 1 Aug 2012 11:14:53 -0700 Subject: [erlang-questions] Building R15B on Ubuntu Message-ID: I'm working on a dev laptop running Ubuntu - the only Linux that seems familiar with my wireless card. Configure is not happy with my wx libs. They are 2.8 but don't include a static lib. Is that required for erlang wx linkage? They aren't part of the Ubuntu wx packages so would have to be built spearately. If so, is it just a matter of building the wx sources to a static library clean and running configure again? Alternatively thee does seem to be an Ubuntu package for what appears to be 14b. Should I be using that version? Thanks for your help, Greg. From desired.mta@REDACTED Wed Aug 1 20:19:29 2012 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Wed, 1 Aug 2012 20:19:29 +0200 Subject: [erlang-questions] Building R15B on Ubuntu In-Reply-To: References: Message-ID: <20120801181929.GA7963@localhost> On Wed, Aug 01, 2012 at 11:14:53AM -0700, Greg Martin wrote: > I'm working on a dev laptop running Ubuntu - the only Linux that seems > familiar with my wireless card. > > Configure is not happy with my wx libs. They are 2.8 but don't include a > static lib. Is that required for erlang wx linkage? They aren't part of the > Ubuntu wx packages so would have to be built spearately. > > If so, is it just a matter of building the wx sources to a static library > clean and running configure again? > > Alternatively thee does seem to be an Ubuntu package for what appears to be > 14b. Should I be using that version? Hi, here you can have the latest greatest: http://www.erlang-solutions.com/section/132/download-erlang-otp Works OK. Motiejus -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From cgsmcmlxxv@REDACTED Wed Aug 1 20:25:04 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Wed, 1 Aug 2012 20:25:04 +0200 Subject: [erlang-questions] Building R15B on Ubuntu In-Reply-To: References: Message-ID: I compiled R15B01 from source with wx 2.8 library from Ubuntu repository and it works fine. You just need to download the dev and headers (if I memory serves) package as well. CGS On Wed, Aug 1, 2012 at 8:14 PM, Greg Martin wrote: > I'm working on a dev laptop running Ubuntu - the only Linux that seems > familiar with my wireless card. > > Configure is not happy with my wx libs. They are 2.8 but don't include a > static lib. Is that required for erlang wx linkage? They aren't part of the > Ubuntu wx packages so would have to be built spearately. > > If so, is it just a matter of building the wx sources to a static library > clean and running configure again? > > Alternatively thee does seem to be an Ubuntu package for what appears to be > 14b. Should I be using that version? > > Thanks for your help, > Greg. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomasl_erlang@REDACTED Wed Aug 1 20:42:54 2012 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 1 Aug 2012 11:42:54 -0700 (PDT) Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: <5018EB2B.5030700@gmail.com> References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5018EB2B.5030700@gmail.com> Message-ID: <1343846574.80867.YahooMailNeo@web111403.mail.gq1.yahoo.com> ----- Original Message ----- > From: Richard Carlsson ... > Ah, but when you say "give as parameter" you mean "pass it a > string literal from the shell", right? I never said anything about strings > in the shell - that's a different environment from source files, and as you > described, the shell nowadays detects your locale and translates UTF-8 console > input into a string literal containing Unicode code points. This is exactly how > it would happen in source code as well, if the compiler only knew how to detect > that a source file is in a different encoding from Latin1. So the compiler is > really the main thing that needs to be fixed, and then there should be no > surprises on the encoding level anymore. How about adding compiler warnings about string literals that do not obey the designated encoding? (There should then, of course, be multiple possibilities to choose from.) E.g., "warn if strings not UTF8", "warn if not Latin-1", "warn if not compatible with current locale" ... If so, it might also be nice to check strings in object files for the same properties,? and perhaps make the shell interpreter complain too. Or maybe you start your erl or erlc with, say, "--string-encoding=..." Best, Thomas From vladdu55@REDACTED Wed Aug 1 20:57:23 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 1 Aug 2012 20:57:23 +0200 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: <1343846574.80867.YahooMailNeo@web111403.mail.gq1.yahoo.com> References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5018EB2B.5030700@gmail.com> <1343846574.80867.YahooMailNeo@web111403.mail.gq1.yahoo.com> Message-ID: Hi, On Wed, Aug 1, 2012 at 8:42 PM, Thomas Lindgren wrote: > How about adding compiler warnings about string literals that do not obey > the designated encoding? (There should then, of course, be multiple possibilities to choose from.) > > E.g., "warn if strings not UTF8", "warn if not Latin-1", "warn if not compatible with current locale" ... The problem is that there aren't any invalid Latin-1 characters and thus strings. One can guess that for example "??" is not a meaningful string, and that it's actually "?", but it's not 100% reliable... regards, Vlad From greg@REDACTED Wed Aug 1 21:21:55 2012 From: greg@REDACTED (Greg Martin) Date: Wed, 1 Aug 2012 12:21:55 -0700 Subject: [erlang-questions] Building R15B on Ubuntu In-Reply-To: <20120801181929.GA7963@localhost> References: <20120801181929.GA7963@localhost> Message-ID: <0CBFA125670E47598B3BB9B4E5F04470@my.domain> Thank you. I've added the repository and used it and it worked a charm. -----Original Message----- From: Motiejus Jak?tys [mailto:desired.mta@REDACTED] Sent: Wednesday, August 01, 2012 11:19 AM To: Greg Martin Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Building R15B on Ubuntu On Wed, Aug 01, 2012 at 11:14:53AM -0700, Greg Martin wrote: > I'm working on a dev laptop running Ubuntu - the only Linux that seems > familiar with my wireless card. > > Configure is not happy with my wx libs. They are 2.8 but don't include > a static lib. Is that required for erlang wx linkage? They aren't part > of the Ubuntu wx packages so would have to be built spearately. > > If so, is it just a matter of building the wx sources to a static > library clean and running configure again? > > Alternatively thee does seem to be an Ubuntu package for what appears > to be 14b. Should I be using that version? Hi, here you can have the latest greatest: http://www.erlang-solutions.com/section/132/download-erlang-otp Works OK. Motiejus From thomasl_erlang@REDACTED Wed Aug 1 22:03:53 2012 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 1 Aug 2012 13:03:53 -0700 (PDT) Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5018EB2B.5030700@gmail.com> <1343846574.80867.YahooMailNeo@web111403.mail.gq1.yahoo.com> Message-ID: <1343851433.12696.YahooMailNeo@web111416.mail.gq1.yahoo.com> ----- Original Message ----- > From: Vlad Dumitrescu ... > > On Wed, Aug 1, 2012 at 8:42 PM, Thomas Lindgren > wrote: >> How about adding compiler warnings about string literals that do not obey >> the designated encoding? (There should then, of course, be multiple > possibilities to choose from.) >> >> E.g., "warn if strings not UTF8", "warn if not > Latin-1", "warn if not compatible with current locale" ... > > The problem is that there aren't any invalid Latin-1 characters and > thus strings. One can guess that for example "??" is not a meaningful > string, and that it's actually "?", but it's not 100% > reliable... Good catch, maybe one could check for subsets of Latin-1 or something if so desired.?I guess I'm mainly interested in UTF8 myself. Best, Thomas From cgsmcmlxxv@REDACTED Wed Aug 1 22:16:49 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Wed, 1 Aug 2012 22:16:49 +0200 Subject: [erlang-questions] Latin-1 to UTF-8 rudimentary convertor Message-ID: Hi everyone, The latest discussions about Latin-1 and UTF-8 characters encoding gave me an idea to create a module for some of the problems raised there. It is not so much because I didn't have time for more, but let me know what would you like to have more and I will do my best. You can find it here: https://github.com/cgsmcmlxxv/UnicodeCodePoints Regards, CGS -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Aug 2 03:28:35 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 2 Aug 2012 13:28:35 +1200 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: <1343846574.80867.YahooMailNeo@web111403.mail.gq1.yahoo.com> References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5018EB2B.5030700@gmail.com> <1343846574.80867.YahooMailNeo@web111403.mail.gq1.yahoo.com> Message-ID: On 2/08/2012, at 6:42 AM, Thomas Lindgren wrote: > > How about adding compiler warnings about string literals that do not obey > the designated encoding? (There should then, of course, be multiple possibilities to choose from.) > What does this actually mean? There is no byte sequence valid in UTF-8 that is not also valid in Latin-1. Yes, codes 128..159 are control characters, but nobody ever said that control characters weren't legal in strings. Checking the mappings that came with Unicode 4, there is no byte sequence valid in UTF-8 that is not also valid in ISO 8859-{1,2,4,5,9,10,13,14,15}, PC code pages 437, 737, 775, 850, 852, 885, 86[012356], and Apple Arabic, Central European, Croatian, Cyrillic, Farsi, Greek, Hebrew, Icelandic, Roman, Romanian, Squeak, and Turkish. So I have no idea what "string literals that do not obey the designated encoding" means or how to operationalise it. From ok@REDACTED Thu Aug 2 03:42:43 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 2 Aug 2012 13:42:43 +1200 Subject: [erlang-questions] unicode in string literals In-Reply-To: <411474AB-6810-4C9C-8014-3A1310D60547@masklinn.net> References: <50168ABD.50304@gmail.com> <501792E6.1020805@ninenines.eu> <88047BF7-4507-470E-A5F5-65D24945EDF7@cs.otago.ac.nz> <411474AB-6810-4C9C-8014-3A1310D60547@masklinn.net> Message-ID: On 1/08/2012, at 6:57 PM, Masklinn wrote: > On 2012-08-01, at 06:14 , Richard O'Keefe wrote: > >> And having a >> distinct data type is no protection against that problem: Java >> and Javascript both have opaque string datatypes, but both >> allow slicing a well formed string into pieces that are not >> well formed. > > To be fair, they've got the further compounding issue that strings types > are dedicated but not opaque: they are sequences of UTF-16 code units > (on account of originally being UCS2 sequences). You are right. I should not have "opaque". The implementation is *encapsulated*, but the fact that it's a slice of an array of 16-bit units shows through. As it happens, I *wasn't* referring to the possibility of splitting a codepoint between two surrogates. If we restrict our attention to the Basic Multilingual Plane, it is *still* possible to slice a well formed BMP string into pieces that are not well formed. I have in mind things like the way Apple used to have two plus signs, one for left to right text and one for right to left text, but since Unicode has only one, the way to encode ?+? was [Aleph, left-to-right override, plus, pop directional formatting, Beth], and a division that gives the left part either 2 or 3 codepoints is one that gives you two strings that make no sense. As it happens, I don't know any programming language that deals with this. My basic point is that any data structure for text that *doesn't* ensure that all the 'strings' you deal with are well formed has already lost its virginity and might as well be frankly and openly just a sequence of code points. From ok@REDACTED Thu Aug 2 03:50:40 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 2 Aug 2012 13:50:40 +1200 Subject: [erlang-questions] unicode in string literals In-Reply-To: References: <50168ABD.50304@gmail.com> Message-ID: <1989E4E9-F924-4D8A-9D16-967EA0861C35@cs.otago.ac.nz> On 1/08/2012, at 7:30 PM, Vlad Dumitrescu wrote: > >> but why should a module written by someone who wants >> comments in M?ori (note the macron? Latin-4 or Unicode needed) >> use a module written by someone who wants comments in Swedish? > > Maybe not in the long run, but there will be a (long) transition > period where legacy code will still be used by new code. Sorry, my typing mistake here. What I *meant* to write was "why should a [M?ori] module *NOT* use a [Swedish] one"? You were saying, or so I thought, that there should be one project = one encoding, and I was saying I thought that was too restrictive in practice. > >> The whole point of an -encoding directive is that it is something >> that syntaxtools should handle; by the time your code gets an AST >> or a token list, encodings are entirely a thing of the past. > > Yes, but I am one of the guys that is going to write some of the tools > that will handle this conversion, so I do care about the details. And by the time it gets to you, there won't *be* any details to care about. > >> SWI Prolog actually lets you change the encoding within a file, >> which sounds crazy but maybe Jan wanted the machinery to be there >> in case someone wanted ISO 2022 support. (Because that's basically >> what 2022 *is*: switching encoding aspects on the fly.) > > Are there any editors that can load/save a file with mixed encodings like that? I have no idea. There are a number of editors that claim to support ISO 2022, which does mid-stream code switching, so they could presumably be extended to support this. See for example A model for input and output of multilingual text in a windowing environment by Yutaka Kataoka, Masato Morisaki, Hiroshi Kuribayashi, and Hiroyoshi Ohara ACM Transactions on Information Systems (TOIS) Volume 10 Issue 4, Oct. 1992 > > I am still a little worried about two things: > - debugging a remote system that has different locale > - reading logs created by modules that have different encodings (some > modules might be legacy and not be aware that the world is not Latin-1 > anymore). Ouch. And then there are all those documents that lie about the encoding they're using. (Web pages claiming Latin 1 but being CP 1252 does not exhaust the possibilities.) From ok@REDACTED Thu Aug 2 03:58:42 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 2 Aug 2012 13:58:42 +1200 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: References: Message-ID: <2CF49DCA-BB99-4719-ABC6-14C5C4CE4F73@cs.otago.ac.nz> On 1/08/2012, at 7:32 PM, Michael Turner wrote: > On Wed, Aug 1, 2012 at 1:33 PM, Richard O'Keefe wrote: >> >> On 31/07/2012, at 9:53 PM, Michael Turner wrote: >> >>>> << An Erlang "string" is simply a list of integers. Each integer can >>>> represent any Unicode codepoint/character. >> >>> >>> Except that Unicode codepoints represents characters, right? >> >> Wrong. > > Actually, what's *really* wrong in my statement is the grammar -- bad > plural agreement. No, what's wrong is the assertion that Unicode codepoints represent characters. > I'm certain this is correct, Richard, but ... what problem are we > trying to solve again? Having Joe not accidentally lying to his readers. > IIRC: Joe is trying to come up with a short > passage that explains what strings are, in Erlang. If he writes all > that you wrote above, the reader (who might have been initially > excited about Erlang) will come away with the impression, "Erlang > people are excruciatingly pedantic". Nobody ever said that >Joe< should say all that. What he needs to say is something like +++ Unicode is actually insanely complicated, +++ but a good starting point is that each character +++ named in the Unicode standard is assigned a +++ unique integer called a codepoint, +++ and an Erlang string is just a list of these numbers. The phrase 'named in the standard' makes this literally true; it also asserts that characters are given codepoints (true), not that odepoints represent characters (false). >>> << In Erlang, strings are represented as lists of integers. These >>> integers are Unicode codepoints, each representing a character. >> >>> >>> That way, anybody who's unclear on what "codepoint" means gets a >>> freebie definition of it. In the Unicode context, it's probably wrong, >>> technically, but perhaps good enough for this purpose. > > Can anyone tell me why this *wouldn't* serve Joe's (== the typical > reader's) purposes? [*] Because it is dangerously wrong and misleading. Why say what is untrue, when you can say something true that is nearly as simple and serves the job just as well? From eric@REDACTED Thu Aug 2 05:18:47 2012 From: eric@REDACTED (Eric Moritz) Date: Wed, 1 Aug 2012 23:18:47 -0400 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5018EB2B.5030700@gmail.com> <1343846574.80867.YahooMailNeo@web111403.mail.gq1.yahoo.com> Message-ID: > There is no byte sequence valid in UTF-8 that is not also > valid in Latin-1. This is incorrect. Latin-1 code points are a subset of Unicode codepoints. Codepoints are not bytes. Codepoints are indexes in character tables. latin-1 is a table of a possible 256 characters where as Unicode is at this point a table of more that 100,000 characters. There are actually codepoints in the range of 127-159 which are unused and if used are technically invalid Latin-1 and Unicode. When it comes to the binary representation of these codepoints. Latin-1 is encoded as literal bytes because all codepoints are less than 256. Unicode codepoints on the other hand can be larger than 255 so in order to represent them as bytes they need to be encoded. Latin-1 bytes larger than 126 are not the same character in UTF-8 because UTF-8 uses the 8th bit for encoding multi byte sequences to represent Unicode codepoints which are larger than 126. So while values in a list greater than 126 are valid Latin-1, if those values represent UTF-8 bytes, the characters are not the same. For instance, 233 is the codepoint for an accented e in Latin-1 and Unicode, the binary representation of that character in Latin-1 is literally the byte <<233>> but when the codepoint is encoded as UTF-8, it is the bytes <<195,169>>. The list [195,169] is never going to be an accented e in Erlang because as far as Erlang is concerned, that is a list of Latin-1 codepoints which are the characters ? and ?. Ever see Caf?? on a webpage? That is because they told the browser that their HTML was latin-1 when it was actually UTF-8. It just so happens that [195,169] is also of type chardata() because all valid latin-1 codepoints are also valid Unicode codepoints. In either case, [195,169] is not an accented e. At the very least it is a list of integers whose values represent UTF-8 encoded bytes but until you convert those UTF-8 bytes to Unicode codepoints it'll never be chardata() with the correct characters. To summarize: Unicode is a table of codepoints. A codepoint is an index in the table. UTF-8 is a codec for turning codepoints to and from bytes. UTF-8 cannot be used to refer to what Erlang calls chardata(). chardata() is a list of integer() whose value is a valid Unicode codepoint. UTF-8 can only refer to a sequence of bytes. Eric. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric@REDACTED Thu Aug 2 05:32:51 2012 From: eric@REDACTED (Eric Moritz) Date: Wed, 1 Aug 2012 23:32:51 -0400 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: <5018EB2B.5030700@gmail.com> References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5018EB2B.5030700@gmail.com> Message-ID: So currently if you encode your source code as UTF-8, string literals become the literal byte sequences. This is different than in the shell where string literals get automatically turned into their Unicode codepoints. It appears that the solution is a compiler flag that tells the compiler that string literals should be decoded as UTF-8. So when the compiler reads the byte sequence 16#C3A9 it knows that it should be a 233 in the list because 16#C3A9 is the UTF-8 encoded sequence for the codepoint 233. It don't know what the overall support the chardata() and charlist() is in the standard lib so doing this may cause many headaches when someone tries to stuff a charlist() where a iolist() goes or chardata() where a string() goes. This may introduce subtle bugs that only occur when non-latin-1 characters are used. Eric. On Aug 1, 2012 4:39 AM, "Richard Carlsson" wrote: > On 08/01/2012 12:52 AM, CGS wrote: > >> Actually, try this: >> >> 1. set your environment to UTF-8 (in my case, whatever Linux terminal >> with BASH environment, export LANG="en_US.utf8", use locale to find your >> environment language definition - "en_US.latin1" for LATIN-1) >> 2. in a module: >> >> test_reverse(String) -> lists:reverse(String). >> >> 3. Give as parameter the example given by yourself. >> 4. Check the output. >> > > Ah, but when you say "give as parameter" you mean "pass it a string > literal from the shell", right? I never said anything about strings in the > shell - that's a different environment from source files, and as you > described, the shell nowadays detects your locale and translates UTF-8 > console input into a string literal containing Unicode code points. This is > exactly how it would happen in source code as well, if the compiler only > knew how to detect that a source file is in a different encoding from > Latin1. So the compiler is really the main thing that needs to be fixed, > and then there should be no surprises on the encoding level anymore. > > /Richard > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Aug 2 06:39:58 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 2 Aug 2012 16:39:58 +1200 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5018EB2B.5030700@gmail.com> <1343846574.80867.YahooMailNeo@web111403.mail.gq1.yahoo.com> Message-ID: On 2/08/2012, at 3:18 PM, Eric Moritz wrote: > > > There is no byte sequence valid in UTF-8 that is not also > > valid in Latin-1. > > This is incorrect. Let's be pedantic here. There is no sequence of bytes B such that (1) B conforms to the rules of UTF-8 and (2) B can also be decoded as Latin 1 This is 100% correct. > > Latin-1 code points are a subset of Unicode codepoints. True and totally irrelevant. The statement in question has nothing to say about codepoints. > Codepoints are not bytes. Also true and totally irrelevant. The statement in question has nothing to say about codepoints. > Codepoints are indexes in character tables. latin-1 is a table of a possible 256 characters where as Unicode is at this point a table of more that 100,000 characters. There are actually codepoints in the range of 127-159 which are unused and if used are technically invalid Latin-1 and Unicode. I suppose it depends on what you mean by "Latin 1". If you look at the code tables in http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-094.pdf you're right: 127 is not there. But then neither are TAB, CR, or LF there. If you want to talk about "Latin 1" in any sense that includes those control characters, you have to admit the others. The framework is specified by ECMA 43, which requires ESC and DEL. So byte 127 is not invalid. If you want TAB, CR, LF, and so on, then you get them from ECMA 48, the C0 set. Bytes with values 128 to 159 *also* come from ECMA 48. So when I talk about "Latin 1" I mean all the printing characters *and* all the ECMA C0 and C1 control characters. It's not just me. Look for example at http://www.madore.org/~david/computers/unicode/cstab.html#Latin-1 which shows the control character names in red. More importantly, look at the mapping tables produced by the Unicode consortium, specifically 8859-1.TXT. 0x7F 0x007F # DELETE 0x80 0x0080 # ... 0x9F 0x009F # 0xA0 0x00A0 # NO-BREAK SPACE The Unicode consortium think that 0x7F to 0x9F are Latin-1 control characters -- they use #UNDEFINED to mark characters that are not defined at all in the source character set -- and for what it's worth, U+007F to U+009F are listed in the Unicode character data base as *defined* characters with class Cc, and they formerly even named the functions they perform. > > When it comes to the binary representation of these codepoints. I specifically wrote about BYTE SEQUENCES. Nothing else is relevant. I did not write about codepoints. > Latin-1 is encoded as literal bytes because all codepoints are less than 256. You can encode Latin 1 in all sorts of ways. Bytes work because it's a member of the ECMA "8-Bit Coded Character Set" family.' > Unicode codepoints on the other hand can be larger than 255 so in order to represent them as bytes they need to be encoded. That's not relevant. It doesn't matter *what* UTF-8 encodes here, the only point is that since a UTF-8 sequence is a byte sequence, and since every byte sequence is a valid Latin 1 encoding, there is no byte sequence that is a valid UTF-8 sequence but not a valid Latin 1 sequence. There are of course many ways to encode Unicode as sequences of bytes. We could, to be ridiculous, represent each Unicode codepoint as a sequence of 21 bytes each with value 0x30 or 0x31. More realistically, SCSU and BOCU have advantages. The thing is, there is no byte sequence that cannot be interpreted as representing a sequence of Latin 1 characters (including control characters), so there is no way of being certain what you have. Of course an XML document must start with zero or more white space characters followed by a left angle bracket. A higher level protocol like that _may_ impose constraints that let you figure out what you have. Similarly an Erlang module must start with a zero or more white space characters or % comments followed by a hyphen-minus character. That is enough to allow XML-style discrimination between big- and little-endian 4-byte and 2-byte representations, some flavour of EBCDIC, and some extension of ASCII, but not to discriminate between Latin 1 and UTF-8. I've deleted the rest of the message as also beside the point. From eric@REDACTED Thu Aug 2 06:54:50 2012 From: eric@REDACTED (Eric Moritz) Date: Thu, 2 Aug 2012 00:54:50 -0400 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5018EB2B.5030700@gmail.com> <1343846574.80867.YahooMailNeo@web111403.mail.gq1.yahoo.com> Message-ID: Sorry. I took your statement out of context. On Aug 2, 2012 12:40 AM, "Richard O'Keefe" wrote: > > On 2/08/2012, at 3:18 PM, Eric Moritz wrote: > > > > > > There is no byte sequence valid in UTF-8 that is not also > > > valid in Latin-1. > > > > This is incorrect. > > Let's be pedantic here. > There is no sequence of bytes B such that > (1) B conforms to the rules of UTF-8 and > (2) B can also be decoded as Latin 1 > > This is 100% correct. > > > > Latin-1 code points are a subset of Unicode codepoints. > > True and totally irrelevant. The statement in question has > nothing to say about codepoints. > > > Codepoints are not bytes. > > Also true and totally irrelevant. The statement in question > has nothing to say about codepoints. > > > Codepoints are indexes in character tables. latin-1 is a table of a > possible 256 characters where as Unicode is at this point a table of more > that 100,000 characters. There are actually codepoints in the range of > 127-159 which are unused and if used are technically invalid Latin-1 and > Unicode. > > I suppose it depends on what you mean by "Latin 1". > If you look at the code tables in > http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-094.pdf > you're right: 127 is not there. > > But then neither are TAB, CR, or LF there. > > If you want to talk about "Latin 1" in any sense that includes > those control characters, you have to admit the others. > The framework is specified by ECMA 43, which requires ESC and > DEL. So byte 127 is not invalid. > If you want TAB, CR, LF, and so on, then you get them from > ECMA 48, the C0 set. Bytes with values 128 to 159 *also* come from > ECMA 48. > > So when I talk about "Latin 1" I mean all the printing characters > *and* all the ECMA C0 and C1 control characters. > > It's not just me. Look for example at > http://www.madore.org/~david/computers/unicode/cstab.html#Latin-1 > which shows the control character names in red. > More importantly, look at the mapping tables produced by the > Unicode consortium, specifically 8859-1.TXT. > 0x7F 0x007F # DELETE > 0x80 0x0080 # > ... > 0x9F 0x009F # > 0xA0 0x00A0 # NO-BREAK SPACE > > The Unicode consortium think that 0x7F to 0x9F are Latin-1 > control characters -- they use #UNDEFINED to mark characters > that are not defined at all in the source character set -- > and for what it's worth, U+007F to U+009F are listed in the > Unicode character data base as *defined* characters with > class Cc, and they formerly even named the functions they > perform. > > > > When it comes to the binary representation of these codepoints. > > I specifically wrote about BYTE SEQUENCES. Nothing else is > relevant. I did not write about codepoints. > > > Latin-1 is encoded as literal bytes because all codepoints are less > than 256. > > You can encode Latin 1 in all sorts of ways. > Bytes work because it's a member of the > ECMA "8-Bit Coded Character Set" family.' > > > Unicode codepoints on the other hand can be larger than 255 so in order > to represent them as bytes they need to be encoded. > > That's not relevant. It doesn't matter *what* UTF-8 encodes here, > the only point is that since a UTF-8 sequence is a byte sequence, > and since every byte sequence is a valid Latin 1 encoding, there > is no byte sequence that is a valid UTF-8 sequence but not a valid > Latin 1 sequence. > > There are of course many ways to encode Unicode as sequences of bytes. > We could, to be ridiculous, represent each Unicode codepoint as a > sequence of 21 bytes each with value 0x30 or 0x31. More realistically, > SCSU and BOCU have advantages. The thing is, there is no byte > sequence that cannot be interpreted as representing a sequence of > Latin 1 characters (including control characters), so there is no way > of being certain what you have. > > Of course an XML document must start with zero or more white space > characters followed by a left angle bracket. A higher level protocol > like that _may_ impose constraints that let you figure out what you > have. Similarly an Erlang module must start with a zero or more > white space characters or % comments followed by a hyphen-minus character. > That is enough to allow XML-style discrimination between big- and > little-endian 4-byte and 2-byte representations, some flavour of > EBCDIC, and some extension of ASCII, but not to discriminate between > Latin 1 and UTF-8. > > I've deleted the rest of the message as also beside the point. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.vb80@REDACTED Thu Aug 2 10:00:17 2012 From: pablo.vb80@REDACTED (Pablo Vieytes) Date: Thu, 2 Aug 2012 10:00:17 +0200 Subject: [erlang-questions] spawning processes Message-ID: Hello everybody, I've been reading you for a long time but this is my first message. I don't know how to spawn a process with args without export it. is there any way to do that? For funs without args you can do spawn(fun_in_the_same_mod/0). Another option is to use spawn(Module, Function, Args) but Function should be exported although it's in the same mod. Any suggestion? Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu Aug 2 10:07:21 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 2 Aug 2012 10:07:21 +0200 Subject: [erlang-questions] spawning processes In-Reply-To: References: Message-ID: Hi and welcome! On Thu, Aug 2, 2012 at 10:00 AM, Pablo Vieytes wrote: > I don't know how to spawn a process with args without export it. is there > any way to do that? > > For funs without args you can do spawn(fun_in_the_same_mod/0). Another > option is to use spawn(Module, Function, Args) but Function should be > exported although it's in the same mod. Try spawn(fun() -> Function(Args) end) regards, Vlad From watson.timothy@REDACTED Thu Aug 2 10:17:32 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 2 Aug 2012 09:17:32 +0100 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: <09A191ED-C75E-487C-AE5C-5F1AA6436685@gmail.com> References: <09A191ED-C75E-487C-AE5C-5F1AA6436685@gmail.com> Message-ID: Any takers for this? I can't be the only person who's had to figure this out. The context is a distributed systems testing framework that needs to support resolving 'localhost' to a proper host name so the framework can become a hidden node and interact with the other erlang nodes It is testing against. Having to specify the real host name would make configuring the tool across different machines and development environments unpleasantly complicated so figuring out the right hostname is a boon in terms of keeping the configuration overhead down, but doing this for long names nodes is proving highly awkward. Can anyone suggest a good portable solution? On 31 Jul 2012, at 13:43, Tim Watson wrote: > Is there a way to calculate the hostname reliably across platforms in Erlang? I have a non-distributed node that I wish to become a distributed node. Normally I call net_kernel:start([Name, shortnames]) and this is just fine. It also works with [Name, longnames] *sometimes and on some systems* - but other times it pukes. I've tried looking in the 'domain' or 'search' entries from inet:get_rc/0 but these are not always populated, even when dns config is clearly in place. I've also tried using 'inet_db:get_searchlist/0' but again, sometimes this returns [[]] but net_kernel:start([foobar, longnames]) doesn't work, whereas doing net_kernel:start([foobar@REDACTED, longnames]) does. > > Am I missing something incredibly obvious here? *is* there actually a simple way of determining what the proper fqdn for the machine should be, without breaking out to the os? I had even considered doing inet:gethostbyname/1 but again, the search domains entry seems to be empty, so I'd assume that -name foobar will work whereas in fact, -name foobar@REDACTED is required otherwise net_kernel won't start. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu Aug 2 10:38:14 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 2 Aug 2012 10:38:14 +0200 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: References: <09A191ED-C75E-487C-AE5C-5F1AA6436685@gmail.com> Message-ID: Hi, On Thu, Aug 2, 2012 at 10:17 AM, Tim Watson wrote: > Any takers for this? I can't be the only person who's had to figure this > out. The context is a distributed systems testing framework that needs to > support resolving 'localhost' to a proper host name so the framework can > become a hidden node and interact with the other erlang nodes It is testing > against. Having to specify the real host name would make configuring the > tool across different machines and development environments unpleasantly > complicated so figuring out the right hostname is a boon in terms of keeping > the configuration overhead down, but doing this for long names nodes is > proving highly awkward. I'm not a guru for this kind of issues, but even looking at Java's support for this, the docs say String java.net.InetAddress.getCanonicalHostName() Gets the fully qualified domain name for this IP address. Best effort method, meaning we may not be able to return the FQDN depending on the underlying system configuration. String java.net.InetAddress.getHostName() Gets the host name for this IP address. If this InetAddress was created with a host name, this host name will be remembered and returned; otherwise, a reverse name lookup will be performed and the result will be returned based on the system configured name lookup service. If a lookup of the name service is required, call getCanonicalHostName. So it depends on how the system is configured... regards, Vlad > Can anyone suggest a good portable solution? > > On 31 Jul 2012, at 13:43, Tim Watson wrote: > > Is there a way to calculate the hostname reliably across platforms in > Erlang? I have a non-distributed node that I wish to become a distributed > node. Normally I call net_kernel:start([Name, shortnames]) and this is just > fine. It also works with [Name, longnames] *sometimes and on some systems* - > but other times it pukes. I've tried looking in the 'domain' or 'search' > entries from inet:get_rc/0 but these are not always populated, even when dns > config is clearly in place. I've also tried using 'inet_db:get_searchlist/0' > but again, sometimes this returns [[]] but net_kernel:start([foobar, > longnames]) doesn't work, whereas doing > net_kernel:start([foobar@REDACTED, longnames]) does. > > Am I missing something incredibly obvious here? *is* there actually a simple > way of determining what the proper fqdn for the machine should be, without > breaking out to the os? I had even considered doing inet:gethostbyname/1 but > again, the search domains entry seems to be empty, so I'd assume that -name > foobar will work whereas in fact, -name foobar@REDACTED is required otherwise > net_kernel won't start. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From gustav.simonsson@REDACTED Thu Aug 2 10:40:03 2012 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Thu, 02 Aug 2012 09:40:03 +0100 (BST) Subject: [erlang-questions] reliably figure out hostname In-Reply-To: Message-ID: <6b095368-ace4-461f-adbd-243b36a8d4d1@knuth> {ok, LocalHostName} = inet:gethostname(). inet_res:gethostbyname(LocalHostName). You get a hostent record back which has the FQDN. // Gustav Simonsson Sent from my PC ----- Original Message ----- > From: "Tim Watson" > To: "Tim Watson" > Cc: "erlang-questions Questions" > Sent: Thursday, 2 August, 2012 10:17:32 AM > Subject: Re: [erlang-questions] reliably figure out hostname > > > > Any takers for this? I can't be the only person who's had to figure > this out. The context is a distributed systems testing framework > that needs to support resolving 'localhost' to a proper host name so > the framework can become a hidden node and interact with the other > erlang nodes It is testing against. Having to specify the real host > name would make configuring the tool across different machines and > development environments unpleasantly complicated so figuring out > the right hostname is a boon in terms of keeping the configuration > overhead down, but doing this for long names nodes is proving highly > awkward. > > > Can anyone suggest a good portable solution? > > On 31 Jul 2012, at 13:43, Tim Watson < watson.timothy@REDACTED > > wrote: > > > > > > Is there a way to calculate the hostname reliably across platforms in > Erlang? I have a non-distributed node that I wish to become a > distributed node. Normally I call net_kernel:start([Name, > shortnames]) and this is just fine. It also works with [Name, > longnames] *sometimes and on some systems* - but other times it > pukes. I've tried looking in the 'domain' or 'search' entries from > inet:get_rc/0 but these are not always populated, even when dns > config is clearly in place. I've also tried using > 'inet_db:get_searchlist/0' but again, sometimes this returns [[]] > but net_kernel:start([foobar, longnames]) doesn't work, whereas > doing net_kernel:start([ foobar@REDACTED , longnames]) does. > > > Am I missing something incredibly obvious here? *is* there actually a > simple way of determining what the proper fqdn for the machine > should be, without breaking out to the os? I had even considered > doing inet:gethostbyname/1 but again, the search domains entry seems > to be empty, so I'd assume that -name foobar will work whereas in > fact, -name foobar @fqdn is required otherwise net_kernel won't > start. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From pablo.vb80@REDACTED Thu Aug 2 10:48:04 2012 From: pablo.vb80@REDACTED (Pablo Vieytes) Date: Thu, 2 Aug 2012 10:48:04 +0200 Subject: [erlang-questions] spawning processes In-Reply-To: References: Message-ID: Hi, I tried that but I had problems using self(). I have to communicate the spawned process with other pocesses, so I need to know the pid of the spawned process. This DOESN'T work -> parent() -> Args = data, spawn(fun() -> son(Args) end). son(Args) -> %%some code -> NewData ReplyTo = self(), AnotherProcess ! {ReplyTo, NewData}. ReplyTo has the pid of process who is running parent fun and I want the pid of the new process spawned with the son fun. This WORKS but I have to export the son fun. parent() -> Args = data, spawn(?MODULE, son, [Args]). son(Args) -> %%some code -> NewData ReplyTo = self(), AnotherProcess ! {ReplyTo, NewData}. 2012/8/2 Vlad Dumitrescu > Hi and welcome! > > On Thu, Aug 2, 2012 at 10:00 AM, Pablo Vieytes > wrote: > > I don't know how to spawn a process with args without export it. is there > > any way to do that? > > > > For funs without args you can do spawn(fun_in_the_same_mod/0). Another > > option is to use spawn(Module, Function, Args) but Function should be > > exported although it's in the same mod. > > Try spawn(fun() -> Function(Args) end) > > regards, > Vlad > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.jarvstrand@REDACTED Thu Aug 2 10:54:12 2012 From: thomas.jarvstrand@REDACTED (=?ISO-8859-1?Q?Thomas_J=E4rvstrand?=) Date: Thu, 2 Aug 2012 10:54:12 +0200 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: References: Message-ID: Hi, This is an introductory book right, so how about something like: << A "string" is a list of integers where the integers represent characters (actually, they are Unicode codepoints that represent characters, but don't worry about that right now). >> Thomas On Tue, Jul 31, 2012 at 11:24 AM, Joe Armstrong wrote: > I'm working on a 2'nd edition of my book, and have got to strings :-) > Strings confuse everybody, including me so I have a few questions: > > To start with Erlang doesn't have strings - it has lists (not strings) > and it has string literals. > > I want to define a string - is this correct: > > << A "string" is a list of integers where the integers > represent Unicode codepoints. >> > > Questions: > Is the sentence inside << .. >> using the correct terminology? > If not what should it say? > > Is the sentence inside << ... >> widely understood, do you think this > would confuse a lot of people? > > Is the phrase "string literal" widely understood? > > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From maruthavanan_s@REDACTED Thu Aug 2 11:00:46 2012 From: maruthavanan_s@REDACTED (Maruthavanan Subbarayan) Date: Thu, 2 Aug 2012 05:00:46 -0400 Subject: [erlang-questions] Crash while using mnesia In-Reply-To: References: Message-ID: Hi All, I am an application that receives series of requests and stores in mnesia table for processing and once that is processed it is deleted from the table. I have done this so that I dont miss out any requests. so while starting the application each time, I process the pending requests before I process new requests. It seemed to be running while for couple of days and after that it gets crashed. I could see from crash dump that enough memory was not allocated. But I could not identify where the memory is being used. because the mnesia db directory was hardly 10K including logs. Attached dump is the recent one (http://www.sendspace.com/file/w2b2tm). So I just included a heart while starting up so that it erlang restart itself. But again caused a dump stopping the application without restarting. I am running erlang using a pipe. Kindly help me to identify why these two issues happen. Thanks, Marutha -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu Aug 2 11:03:10 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 2 Aug 2012 11:03:10 +0200 Subject: [erlang-questions] spawning processes In-Reply-To: References: Message-ID: Hi, On Thu, Aug 2, 2012 at 10:48 AM, Pablo Vieytes wrote: > This DOESN'T work -> > > parent() -> > Args = data, > spawn(fun() -> son(Args) end). > > son(Args) -> > %%some code -> NewData > ReplyTo = self(), > AnotherProcess ! {ReplyTo, NewData}. > > ReplyTo has the pid of process who is running parent fun and I want the pid > of the new process spawned with the son fun. It does work, maybe there is something else creating trouble? parent() -> io:format("P:~p~n", [self()]), Args = data, C = spawn(fun() -> son(Args) end), io:format("C:~p~n", [C]). son(_Args) -> ReplyTo = self(), io:format("X:~p~n", [ReplyTo]). gives the output 4> main:parent(). P:<0.102.0> C:<0.122.0> X:<0.122.0> ok regards, Vlad From pablo.vb80@REDACTED Thu Aug 2 11:43:52 2012 From: pablo.vb80@REDACTED (Pablo Vieytes) Date: Thu, 2 Aug 2012 11:43:52 +0200 Subject: [erlang-questions] spawning processes In-Reply-To: References: Message-ID: Hi, You are right. I don't know what I did. Your code works. Thanks. 2012/8/2 Vlad Dumitrescu > Hi, > > On Thu, Aug 2, 2012 at 10:48 AM, Pablo Vieytes > wrote: > > This DOESN'T work -> > > > > parent() -> > > Args = data, > > spawn(fun() -> son(Args) end). > > > > son(Args) -> > > %%some code -> NewData > > ReplyTo = self(), > > AnotherProcess ! {ReplyTo, NewData}. > > > > ReplyTo has the pid of process who is running parent fun and I want the > pid > > of the new process spawned with the son fun. > > It does work, maybe there is something else creating trouble? > > parent() -> > io:format("P:~p~n", [self()]), > Args = data, > C = spawn(fun() -> son(Args) end), > io:format("C:~p~n", [C]). > > son(_Args) -> > ReplyTo = self(), > io:format("X:~p~n", [ReplyTo]). > > gives the output > > 4> main:parent(). > P:<0.102.0> > C:<0.122.0> > X:<0.122.0> > ok > > regards, > Vlad > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.vb80@REDACTED Thu Aug 2 12:05:20 2012 From: pablo.vb80@REDACTED (Pablo Vieytes) Date: Thu, 2 Aug 2012 12:05:20 +0200 Subject: [erlang-questions] spawning processes In-Reply-To: References: Message-ID: hi, You are right. I don't know what I did but your code works. Thanks. 2012/8/2 Vlad Dumitrescu > Hi, > > On Thu, Aug 2, 2012 at 10:48 AM, Pablo Vieytes > wrote: > > This DOESN'T work -> > > > > parent() -> > > Args = data, > > spawn(fun() -> son(Args) end). > > > > son(Args) -> > > %%some code -> NewData > > ReplyTo = self(), > > AnotherProcess ! {ReplyTo, NewData}. > > > > ReplyTo has the pid of process who is running parent fun and I want the > pid > > of the new process spawned with the son fun. > > It does work, maybe there is something else creating trouble? > > parent() -> > io:format("P:~p~n", [self()]), > Args = data, > C = spawn(fun() -> son(Args) end), > io:format("C:~p~n", [C]). > > son(_Args) -> > ReplyTo = self(), > io:format("X:~p~n", [ReplyTo]). > > gives the output > > 4> main:parent(). > P:<0.102.0> > C:<0.122.0> > X:<0.122.0> > ok > > regards, > Vlad > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomasl_erlang@REDACTED Thu Aug 2 14:05:41 2012 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 2 Aug 2012 05:05:41 -0700 (PDT) Subject: [erlang-questions] correct terminology for referring to strings References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5018EB2B.5030700@gmail.com> <1343846574.80867.YahooMailNeo@web111403.mail.gq1.yahoo.com> Message-ID: <1343909141.62398.YahooMailNeo@web111404.mail.gq1.yahoo.com> It might well be a wooly-headed idea when you look at the details, and I confess to not being an expert in this area. The basic concept would be to warn when, for instance, you've entered your string literals in Latin-1 when the compiler or system options decree that you should use UTF8. I like the overall idea of not leaving encoding problems to the good will of external tools, but if it can't be detected reliably, then it's of course just dreaming.? Another approach might be to use a heuristic tool a la xref to detect "suspicious" string literals. Not sure if that helps. Best, Thomas ----- Original Message ----- > From: Richard O'Keefe > To: Thomas Lindgren > Cc: "erlang-questions@REDACTED" > Sent: Thursday, August 2, 2012 3:28 AM > Subject: Re: [erlang-questions] correct terminology for referring to strings > > > On 2/08/2012, at 6:42 AM, Thomas Lindgren wrote: >> >>? How about adding compiler warnings about string literals that do not obey >>? the designated encoding? (There should then, of course, be multiple > possibilities to choose from.) >> > > What does this actually mean? > > There is no byte sequence valid in UTF-8 that is not also > valid in Latin-1.? Yes, codes 128..159 are control characters, > but nobody ever said that control characters weren't legal in > strings.? Checking the mappings that came with Unicode 4, > there is no byte sequence valid in UTF-8 that is not also > valid in ISO 8859-{1,2,4,5,9,10,13,14,15}, PC code pages > 437, 737, 775, 850, 852, 885, 86[012356], and Apple Arabic, > Central European, Croatian, Cyrillic, Farsi, Greek, Hebrew, > Icelandic, Roman, Romanian, Squeak, and Turkish. > > So I have no idea what "string literals that do not obey > the designated encoding" means or how to operationalise it. > From bourinov@REDACTED Thu Aug 2 14:29:29 2012 From: bourinov@REDACTED (Max Bourinov) Date: Thu, 2 Aug 2012 16:29:29 +0400 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: References: Message-ID: Perfectly clear for me. Best regards, Max On Tue, Jul 31, 2012 at 1:24 PM, Joe Armstrong wrote: > I'm working on a 2'nd edition of my book, and have got to strings :-) > Strings confuse everybody, including me so I have a few questions: > > To start with Erlang doesn't have strings - it has lists (not strings) > and it has string literals. > > I want to define a string - is this correct: > > << A "string" is a list of integers where the integers > represent Unicode codepoints. >> > > Questions: > Is the sentence inside << .. >> using the correct terminology? > If not what should it say? > > Is the sentence inside << ... >> widely understood, do you think this > would confuse a lot of people? > > Is the phrase "string literal" widely understood? > > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgsmcmlxxv@REDACTED Thu Aug 2 15:46:06 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Thu, 2 Aug 2012 15:46:06 +0200 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: References: Message-ID: Hi Joe, Regarding the clarity, you can see from the length of this thread how clear your definition is. :) Regarding the correctness, your definition is a bit tricky (arguable if taking into account the difference in between Unicode code points and Unicode encoding schemes) in my opinion (non-expert opinion, though). That's because even if using UTF-8 encoding scheme, for example, Erlang knows nothing about the correlation in between the elements of the list, so, the sequence can be interpreted as code points in Latin-1 region even if those code points may make no real sense in Latin-1 when replaced with the indexed characters (especially in the region 128 - 255). For clarity, your famous "a?b" in Unicode code points is [97,8734,98] (this format may break the code) while in UTF-8 encoding scheme is reading [97,226,136,158,98] (Erlang compiler has no idea that the sequence [226,136,158] should be built back to 8734 before passing it back to the environment, so, strange symbols may appear if the environment interprets the integers as Unicode code points - which usually does). When UTF-8 support will be available in Erlang, I suppose the string will be accepted internally also as Unicode code points for the range from U+0080 - U+1FFFFF, but until then the accepted integers represent the disconnected UTF-8 encoding scheme sequence of bytes. It is still the user's job to transform them back in Unicode code points for the environment to display correctly the symbols (e.g., io:format("~ts~n",[[97,8734,98]]) will reproduce the correct string in an UTF-8 environment). This is my 2c opinion (I hope I offended no expert). CGS On Tue, Jul 31, 2012 at 11:24 AM, Joe Armstrong wrote: > I'm working on a 2'nd edition of my book, and have got to strings :-) > Strings confuse everybody, including me so I have a few questions: > > To start with Erlang doesn't have strings - it has lists (not strings) > and it has string literals. > > I want to define a string - is this correct: > > << A "string" is a list of integers where the integers > represent Unicode codepoints. >> > > Questions: > Is the sentence inside << .. >> using the correct terminology? > If not what should it say? > > Is the sentence inside << ... >> widely understood, do you think this > would confuse a lot of people? > > Is the phrase "string literal" widely understood? > > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Thu Aug 2 16:31:50 2012 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 2 Aug 2012 16:31:50 +0200 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: <6b095368-ace4-461f-adbd-243b36a8d4d1@knuth> References: <6b095368-ace4-461f-adbd-243b36a8d4d1@knuth> Message-ID: <20120802143150.GA16659@erix.ericsson.se> On Thu, Aug 02, 2012 at 09:40:03AM +0100, Gustav Simonsson wrote: > > {ok, LocalHostName} = inet:gethostname(). > inet_res:gethostbyname(LocalHostName). That forces using the OTP DNS resolver, which may be in conflict with the current node configuration. The first approach should be: {ok, LocalHostName} = inet:gethostname(). inet:gethostbyname(LocalHostName). / Raimo Niskanen > > You get a hostent record back which has the FQDN. > > // Gustav Simonsson > > Sent from my PC > > ----- Original Message ----- > > From: "Tim Watson" > > To: "Tim Watson" > > Cc: "erlang-questions Questions" > > Sent: Thursday, 2 August, 2012 10:17:32 AM > > Subject: Re: [erlang-questions] reliably figure out hostname > > > > > > > > Any takers for this? I can't be the only person who's had to figure > > this out. The context is a distributed systems testing framework > > that needs to support resolving 'localhost' to a proper host name so > > the framework can become a hidden node and interact with the other > > erlang nodes It is testing against. Having to specify the real host > > name would make configuring the tool across different machines and > > development environments unpleasantly complicated so figuring out > > the right hostname is a boon in terms of keeping the configuration > > overhead down, but doing this for long names nodes is proving highly > > awkward. > > > > > > Can anyone suggest a good portable solution? > > > > On 31 Jul 2012, at 13:43, Tim Watson < watson.timothy@REDACTED > > > wrote: > > > > > > > > > > > > Is there a way to calculate the hostname reliably across platforms in > > Erlang? I have a non-distributed node that I wish to become a > > distributed node. Normally I call net_kernel:start([Name, > > shortnames]) and this is just fine. It also works with [Name, > > longnames] *sometimes and on some systems* - but other times it > > pukes. I've tried looking in the 'domain' or 'search' entries from > > inet:get_rc/0 but these are not always populated, even when dns > > config is clearly in place. I've also tried using > > 'inet_db:get_searchlist/0' but again, sometimes this returns [[]] > > but net_kernel:start([foobar, longnames]) doesn't work, whereas > > doing net_kernel:start([ foobar@REDACTED , longnames]) does. > > > > > > Am I missing something incredibly obvious here? *is* there actually a > > simple way of determining what the proper fqdn for the machine > > should be, without breaking out to the os? I had even considered > > doing inet:gethostbyname/1 but again, the search domains entry seems > > to be empty, so I'd assume that -name foobar will work whereas in > > fact, -name foobar @fqdn is required otherwise net_kernel won't > > start. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From watson.timothy@REDACTED Thu Aug 2 16:33:37 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 2 Aug 2012 15:33:37 +0100 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: <6b095368-ace4-461f-adbd-243b36a8d4d1@knuth> References: <6b095368-ace4-461f-adbd-243b36a8d4d1@knuth> Message-ID: On 2 Aug 2012, at 09:40, Gustav Simonsson wrote: > > {ok, LocalHostName} = inet:gethostname(). > inet_res:gethostbyname(LocalHostName). > > You get a hostent record back which has the FQDN. > Cool thanks for pointing that out Gustav, I'll give this a try on the various machines we had troubles with. Cheers, Tim -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 235 bytes Desc: Message signed with OpenPGP using GPGMail URL: From zabrane3@REDACTED Thu Aug 2 17:19:34 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 2 Aug 2012 17:19:34 +0200 Subject: [erlang-questions] zlib: how many bytes were used during the uncompression (one call of zlib:inflate/2) Message-ID: <09D55B88-E90E-4E19-AD78-92BF3DB6F46C@gmail.com> Hi guys, I'm playing a bit with the zlib module today. Let say I wan to uncompress a GzipData binary with zlib:inflate/2: InflatedData = zlib:inflate(Z, GzipData). In case of success, I want to know how many bytes from GzipData were used internally to get the InflatedData? Regards, Zabrane From watson.timothy@REDACTED Thu Aug 2 17:29:22 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 2 Aug 2012 16:29:22 +0100 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: <20120802143150.GA16659@erix.ericsson.se> References: <6b095368-ace4-461f-adbd-243b36a8d4d1@knuth> <20120802143150.GA16659@erix.ericsson.se> Message-ID: On 2 Aug 2012, at 15:31, Raimo Niskanen wrote: > On Thu, Aug 02, 2012 at 09:40:03AM +0100, Gustav Simonsson wrote: >> >> {ok, LocalHostName} = inet:gethostname(). >> inet_res:gethostbyname(LocalHostName). > > That forces using the OTP DNS resolver, which may be in conflict > with the current node configuration. The first approach should be: > > {ok, LocalHostName} = inet:gethostname(). > inet:gethostbyname(LocalHostName). > Thanks both of you, I'll look at both these and see what comes up on the machines that are causing issues. My fallback position is to make the test operator provide a settings file with their FQDN when using longnames configurations, which at least minimises the impact, but still it'd be nice to do this without forcing the user to intervene. I will post back to the list if/when I find out the best approach for my own particular use case. Cheers! Tim -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 235 bytes Desc: Message signed with OpenPGP using GPGMail URL: From watson.timothy@REDACTED Thu Aug 2 18:33:42 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 2 Aug 2012 17:33:42 +0100 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: <20120802143150.GA16659@erix.ericsson.se> References: <6b095368-ace4-461f-adbd-243b36a8d4d1@knuth> <20120802143150.GA16659@erix.ericsson.se> Message-ID: On 2 August 2012 15:31, Raimo Niskanen wrote: > On Thu, Aug 02, 2012 at 09:40:03AM +0100, Gustav Simonsson wrote: >> >> {ok, LocalHostName} = inet:gethostname(). >> inet_res:gethostbyname(LocalHostName). > > That forces using the OTP DNS resolver, which may be in conflict > with the current node configuration. The first approach should be: > > {ok, LocalHostName} = inet:gethostname(). > inet:gethostbyname(LocalHostName). > That's clearly not what net_kernel is doing though: ============================================== t4@REDACTED:x-test $ erl -name foobar Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.1 (abort with ^G) (foobar@REDACTED)1> BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution a t4@REDACTED:x-test $ erl -name foobar@REDACTED Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.1 (abort with ^G) (foobar@REDACTED)1> BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution a t4@REDACTED:x-test $ erl Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.1 (abort with ^G) 1> {ok, Host} = inet:gethostname(). {ok,"frigg"} 2> inet:gethostbyname(Host). {ok,{hostent,"frigg",[],inet,4,[{127,0,0,1}]}} 3> BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution a t4@REDACTED:x-test $ ============================================== And the problem I've got is that whilst on this machine (CentOS 6.2, R15B01-64bit) the network configuration is *somehow* set up so that net_kernal magically picks up the right domain info, this doesn't work consistently everywhere else. And neither does calling inet_rc, as Gustav suggested: ============================================== t4@REDACTED:x-test $ erl Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.1 (abort with ^G) 1> {ok, LocalHostName} = inet:gethostname(). {ok,"frigg"} 2> inet_res:gethostbyname(LocalHostName). {error,nxdomain} 3> ============================================= On some other machines, inet_res:gethostbyname/1 returns {error, timeout}. > / Raimo Niskanen > >> >> You get a hostent record back which has the FQDN. >> >> // Gustav Simonsson >> >> Sent from my PC >> >> ----- Original Message ----- >> > From: "Tim Watson" >> > To: "Tim Watson" >> > Cc: "erlang-questions Questions" >> > Sent: Thursday, 2 August, 2012 10:17:32 AM >> > Subject: Re: [erlang-questions] reliably figure out hostname >> > >> > >> > >> > Any takers for this? I can't be the only person who's had to figure >> > this out. The context is a distributed systems testing framework >> > that needs to support resolving 'localhost' to a proper host name so >> > the framework can become a hidden node and interact with the other >> > erlang nodes It is testing against. Having to specify the real host >> > name would make configuring the tool across different machines and >> > development environments unpleasantly complicated so figuring out >> > the right hostname is a boon in terms of keeping the configuration >> > overhead down, but doing this for long names nodes is proving highly >> > awkward. >> > >> > >> > Can anyone suggest a good portable solution? >> > >> > On 31 Jul 2012, at 13:43, Tim Watson < watson.timothy@REDACTED > >> > wrote: >> > >> > >> > >> > >> > >> > Is there a way to calculate the hostname reliably across platforms in >> > Erlang? I have a non-distributed node that I wish to become a >> > distributed node. Normally I call net_kernel:start([Name, >> > shortnames]) and this is just fine. It also works with [Name, >> > longnames] *sometimes and on some systems* - but other times it >> > pukes. I've tried looking in the 'domain' or 'search' entries from >> > inet:get_rc/0 but these are not always populated, even when dns >> > config is clearly in place. I've also tried using >> > 'inet_db:get_searchlist/0' but again, sometimes this returns [[]] >> > but net_kernel:start([foobar, longnames]) doesn't work, whereas >> > doing net_kernel:start([ foobar@REDACTED , longnames]) does. >> > >> > >> > Am I missing something incredibly obvious here? *is* there actually a >> > simple way of determining what the proper fqdn for the machine >> > should be, without breaking out to the os? I had even considered >> > doing inet:gethostbyname/1 but again, the search domains entry seems >> > to be empty, so I'd assume that -name foobar will work whereas in >> > fact, -name foobar @fqdn is required otherwise net_kernel won't >> > start. >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jbothma@REDACTED Thu Aug 2 18:42:37 2012 From: jbothma@REDACTED (JD Bothma) Date: Thu, 2 Aug 2012 18:42:37 +0200 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: References: <6b095368-ace4-461f-adbd-243b36a8d4d1@knuth> <20120802143150.GA16659@erix.ericsson.se> Message-ID: I remember having to add something to the hosts file of a recent ubuntu release before erlang would find out its own hostname properly. Not sure if that's related. I'll see if I can dig up more info. Does a machine necessarily have a FQDN? Is this necessarily exactly one? Perhaps you'll have to work out what erlang does to get some hostname and then set the appropriate setting (once) on each relevant machine to make sure it gets what you expect it to. JD On 2 August 2012 18:33, Tim Watson wrote: > On 2 August 2012 15:31, Raimo Niskanen > wrote: >> On Thu, Aug 02, 2012 at 09:40:03AM +0100, Gustav Simonsson wrote: >>> >>> {ok, LocalHostName} = inet:gethostname(). >>> inet_res:gethostbyname(LocalHostName). >> >> That forces using the OTP DNS resolver, which may be in conflict >> with the current node configuration. The first approach should be: >> >> {ok, LocalHostName} = inet:gethostname(). >> inet:gethostbyname(LocalHostName). >> > > That's clearly not what net_kernel is doing though: > > ============================================== > > t4@REDACTED:x-test $ erl -name foobar > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.9.1 (abort with ^G) > (foobar@REDACTED)1> > BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded > (v)ersion (k)ill (D)b-tables (d)istribution > a > t4@REDACTED:x-test $ erl -name foobar@REDACTED > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.9.1 (abort with ^G) > (foobar@REDACTED)1> > BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded > (v)ersion (k)ill (D)b-tables (d)istribution > a > t4@REDACTED:x-test $ erl > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.9.1 (abort with ^G) > 1> {ok, Host} = inet:gethostname(). > {ok,"frigg"} > 2> inet:gethostbyname(Host). > {ok,{hostent,"frigg",[],inet,4,[{127,0,0,1}]}} > 3> > BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded > (v)ersion (k)ill (D)b-tables (d)istribution > a > t4@REDACTED:x-test $ > > ============================================== > > And the problem I've got is that whilst on this machine (CentOS 6.2, > R15B01-64bit) the network configuration is *somehow* set up so that > net_kernal magically picks up the right domain info, this doesn't work > consistently everywhere else. And neither does calling inet_rc, as > Gustav suggested: > > ============================================== > t4@REDACTED:x-test $ erl > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.9.1 (abort with ^G) > 1> {ok, LocalHostName} = inet:gethostname(). > {ok,"frigg"} > 2> inet_res:gethostbyname(LocalHostName). > {error,nxdomain} > 3> > > ============================================= > > On some other machines, inet_res:gethostbyname/1 returns {error, timeout}. > >> / Raimo Niskanen >> >>> >>> You get a hostent record back which has the FQDN. >>> >>> // Gustav Simonsson >>> >>> Sent from my PC >>> >>> ----- Original Message ----- >>> > From: "Tim Watson" >>> > To: "Tim Watson" >>> > Cc: "erlang-questions Questions" >>> > Sent: Thursday, 2 August, 2012 10:17:32 AM >>> > Subject: Re: [erlang-questions] reliably figure out hostname >>> > >>> > >>> > >>> > Any takers for this? I can't be the only person who's had to figure >>> > this out. The context is a distributed systems testing framework >>> > that needs to support resolving 'localhost' to a proper host name so >>> > the framework can become a hidden node and interact with the other >>> > erlang nodes It is testing against. Having to specify the real host >>> > name would make configuring the tool across different machines and >>> > development environments unpleasantly complicated so figuring out >>> > the right hostname is a boon in terms of keeping the configuration >>> > overhead down, but doing this for long names nodes is proving highly >>> > awkward. >>> > >>> > >>> > Can anyone suggest a good portable solution? >>> > >>> > On 31 Jul 2012, at 13:43, Tim Watson < watson.timothy@REDACTED > >>> > wrote: >>> > >>> > >>> > >>> > >>> > >>> > Is there a way to calculate the hostname reliably across platforms in >>> > Erlang? I have a non-distributed node that I wish to become a >>> > distributed node. Normally I call net_kernel:start([Name, >>> > shortnames]) and this is just fine. It also works with [Name, >>> > longnames] *sometimes and on some systems* - but other times it >>> > pukes. I've tried looking in the 'domain' or 'search' entries from >>> > inet:get_rc/0 but these are not always populated, even when dns >>> > config is clearly in place. I've also tried using >>> > 'inet_db:get_searchlist/0' but again, sometimes this returns [[]] >>> > but net_kernel:start([foobar, longnames]) doesn't work, whereas >>> > doing net_kernel:start([ foobar@REDACTED , longnames]) does. >>> > >>> > >>> > Am I missing something incredibly obvious here? *is* there actually a >>> > simple way of determining what the proper fqdn for the machine >>> > should be, without breaking out to the os? I had even considered >>> > doing inet:gethostbyname/1 but again, the search domains entry seems >>> > to be empty, so I'd assume that -name foobar will work whereas in >>> > fact, -name foobar @fqdn is required otherwise net_kernel won't >>> > start. >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> -- >> >> / Raimo Niskanen, Erlang/OTP, Ericsson AB >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Thu Aug 2 18:58:30 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 2 Aug 2012 17:58:30 +0100 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: References: <6b095368-ace4-461f-adbd-243b36a8d4d1@knuth> <20120802143150.GA16659@erix.ericsson.se> Message-ID: On 2 August 2012 17:42, JD Bothma wrote: > I remember having to add something to the hosts file of a recent > ubuntu release before erlang would find out its own hostname properly. > Not sure if that's related. I'll see if I can dig up more info. > Thanks, I appreciate that. > Does a machine necessarily have a FQDN? Is this necessarily exactly > one? Perhaps you'll have to work out what erlang does to get some > hostname and then set the appropriate setting (once) on each relevant > machine to make sure it gets what you expect it to. > Well, the issue is that the testing framework knows nothing about this issue, but at the moment it does 2 things: 1. on startup, it uses net_kernel:start/1 to become a distributed erlang node 2. when (optionally) connecting to remote nodes (i.e., test subjects) it uses net_kernel:hidden_connect_node/1 This all works fine for long and short names, as long as the machine configuration is correct, but there's another catch. When you specify the test nodes you're going to start, you can optionally provide them as [{localhost, [node1, node2]}] etc. In *this* case, localhost must be converted to the correct machine name, and as I said that works *fine* for shortnames, but for longnames it appears to depend on the configuration of the machine. Perhaps as Vlad suggested, this is an insurmountable problem because of the varying configurations that could exist in different environments. From jbothma@REDACTED Thu Aug 2 22:21:07 2012 From: jbothma@REDACTED (JD Bothma) Date: Thu, 2 Aug 2012 22:21:07 +0200 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: References: <6b095368-ace4-461f-adbd-243b36a8d4d1@knuth> <20120802143150.GA16659@erix.ericsson.se> Message-ID: I like insurmountable problems :) So I know you have various different environments and setups, but I still think if it's actually set up in a decent way on each machine, it should work. At the end of the day if you want things to guess correctly you need to be sure they're really set up correctly - you didn't say how you checked so you might have it really correct already. For us it was that ubuntu didn't give useful FQDNs unless we set it properly according to http://www.leonardoborda.com/blog/127-0-1-1-ubuntu-debian/ The bits to take from it are: "some applications like GNOME expects that the hostname to be resolved to an ip address with a canonical fully qualified domain name" (context is that ubuntu uses a hack with FQDN pointing to 127.0.1.1 in /etc/hosts) and "In order to support that software, it is necessary to ensure that the system hostname can be resolved. Most often this is done by putting a line in /etc/hosts containing some IP address and the system hostname. If your system has a permanent IP address then use that; otherwise use the address 127.0.1.1." and a bit more help "To see whether your system hostname can be resolved to an IP address with a fully qualified domain name, use the hostname ?fqdn command. Also it could be any ip address in the 127.0.0.1/8 address block since according to the RFC 1700 , 127.0.0.0/8 addresses are reserved for loopback purposes." I think this made things work properly when looking up the domain name for tests that depended on automatically knowing the FQDN of the machine as suggested by others in the thread, but I might be totally on the wrong track :) {ok, Hostname} = inet:gethostname(), {ok,{hostent,FullHostname,[],inet,_,[_]}} = inet:gethostbyname(Hostname), What the article about ubuntu suggests I think is that localhost can still be 127.0.0.1 and you can have dynamic ips and whatever while the machine has a hardcoded FQDN that it should call itself (e.g. when sending a message that need's a hostname for return path). If you want a machine to automatically pick up the hostname pointing to it (a la hostname.local), I think that's tricky feature of your DHCP or name server or something, and then the machines have to play along. Hope that helps ! JD On 2 August 2012 18:58, Tim Watson wrote: > On 2 August 2012 17:42, JD Bothma wrote: >> I remember having to add something to the hosts file of a recent >> ubuntu release before erlang would find out its own hostname properly. >> Not sure if that's related. I'll see if I can dig up more info. >> > > Thanks, I appreciate that. > >> Does a machine necessarily have a FQDN? Is this necessarily exactly >> one? Perhaps you'll have to work out what erlang does to get some >> hostname and then set the appropriate setting (once) on each relevant >> machine to make sure it gets what you expect it to. >> > > Well, the issue is that the testing framework knows nothing about this > issue, but at the moment it does 2 things: > > 1. on startup, it uses net_kernel:start/1 to become a distributed erlang node > 2. when (optionally) connecting to remote nodes (i.e., test subjects) > it uses net_kernel:hidden_connect_node/1 > > This all works fine for long and short names, as long as the machine > configuration is correct, but there's another catch. When you specify > the test nodes you're going to start, you can optionally provide them > as [{localhost, [node1, node2]}] etc. In *this* case, localhost must > be converted to the correct machine name, and as I said that works > *fine* for shortnames, but for longnames it appears to depend on the > configuration of the machine. > > Perhaps as Vlad suggested, this is an insurmountable problem because > of the varying configurations that could exist in different > environments. From ok@REDACTED Thu Aug 2 22:58:09 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 3 Aug 2012 08:58:09 +1200 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: <6b095368-ace4-461f-adbd-243b36a8d4d1@knuth> References: <6b095368-ace4-461f-adbd-243b36a8d4d1@knuth> Message-ID: On 2/08/2012, at 8:40 PM, Gustav Simonsson wrote: > {ok, LocalHostName} = inet:gethostname(). > inet_res:gethostbyname(LocalHostName). > > You get a hostent record back which has the FQDN. You *may* do so. Then again, you may *not*. Erlang (BEAM) emulator version 5.6.3 [source] [smp:2] [async-threads:0] [kernel-poll:false] Eshell V5.6.3 (abort with ^G) 1> {ok, LocalHostName} = inet:gethostname(). {ok,"oucs1251"} 2> inet_res:gethostbyname(LocalHostName). {error,timeout} Tested on an intel Core 2 duo Mac running OSX 10.6.8. From zabrane3@REDACTED Thu Aug 2 23:18:38 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 2 Aug 2012 23:18:38 +0200 Subject: [erlang-questions] zlib: how many bytes were used during the uncompression (one call of zlib:inflate/2) In-Reply-To: <09D55B88-E90E-4E19-AD78-92BF3DB6F46C@gmail.com> References: <09D55B88-E90E-4E19-AD78-92BF3DB6F46C@gmail.com> Message-ID: <88246A8F-1DEE-4CDE-8E1F-CBF3F0046966@gmail.com> Need help on this please! On Aug 2, 2012, at 5:19 PM, Zabrane Mickael wrote: > Hi guys, > > I'm playing a bit with the zlib module today. > > Let say I wan to uncompress a GzipData binary with zlib:inflate/2: > InflatedData = zlib:inflate(Z, GzipData). > > In case of success, I want to know how many bytes from GzipData were used > internally to get the InflatedData? > > Regards, > Zabrane > From ulf@REDACTED Thu Aug 2 23:47:42 2012 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 2 Aug 2012 23:47:42 +0200 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: References: <6b095368-ace4-461f-adbd-243b36a8d4d1@knuth> Message-ID: <168765F1-E444-411C-895E-2B9CFB4C5282@feuerlabs.com> On 2 Aug 2012, at 22:58, Richard O'Keefe wrote: > Eshell V5.6.3 (abort with ^G) > 1> {ok, LocalHostName} = inet:gethostname(). > {ok,"oucs1251"} > 2> inet_res:gethostbyname(LocalHostName). > {error,timeout} > > Tested on an intel Core 2 duo Mac running OSX 10.6.8. This works on my Mac: hostname() -> case {inet_db:gethostname(),inet_db:res_option(domain)} of {H,D} when is_list(D), is_list(H), length(D)> 0, length(H)>0 -> H ++ "." ++ D; Other -> error({hostname, Other}) end. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From nem@REDACTED Thu Aug 2 23:58:09 2012 From: nem@REDACTED (Geoff Cant) Date: Thu, 02 Aug 2012 14:58:09 -0700 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: (Tim Watson's message of "Thu, 2 Aug 2012 09:17:32 +0100") References: <09A191ED-C75E-487C-AE5C-5F1AA6436685@gmail.com> Message-ID: >From (hazy) memory, longnames are configured using net_adm:localhost(). This differs from the local host name resolution methods given in the thread in that it uses the contents of 'resolv.conf' (specifically the domain option) to construct the FQDN from the hostname. https://github.com/erlang/otp/blob/master/lib/kernel/src/net_adm.erl#L74 The last time I tried to write a reliable distribution based start/stop/status control script, I'm pretty sure I ended up inspecting the machine configuration, then manually starting distribution via something like: https://github.com/archaelus/egc_examples/blob/master/rescriptmsh#L6 to ensure that I would get the name I wanted. Unfortunately I don't have permission yet to release the control script that implements all this. Cheers, -Geoff Tim Watson writes: > Any takers for this? I can't be the only person who's had to figure > this out. The context is a distributed systems testing framework that > needs to support resolving 'localhost' to a proper host name so the > framework can become a hidden node and interact with the other erlang > nodes It is testing against. Having to specify the real host name > would make configuring the tool across different machines and > development environments unpleasantly complicated so figuring out the > right hostname is a boon in terms of keeping the configuration > overhead down, but doing this for long names nodes is proving highly > awkward. > > Can anyone suggest a good portable solution? > > On 31 Jul 2012, at 13:43, Tim Watson wrote: > >> Is there a way to calculate the hostname reliably across platforms >> in Erlang? I have a non-distributed node that I wish to become a >> distributed node. Normally I call net_kernel:start([Name, >> shortnames]) and this is just fine. It also works with [Name, >> longnames] *sometimes and on some systems* - but other times it >> pukes. I've tried looking in the 'domain' or 'search' entries from >> inet:get_rc/0 but these are not always populated, even when dns >> config is clearly in place. I've also tried using >> inet_db:get_searchlist/0' but again, sometimes this returns [[]] but >> net_kernel:start([foobar, longnames]) doesn't work, whereas doing >> net_kernel:start([foobar@REDACTED, longnames]) does. >> >> Am I missing something incredibly obvious here? *is* there actually >> a simple way of determining what the proper fqdn for the machine >> should be, without breaking out to the os? I had even considered >> doing inet:gethostbyname/1 but again, the search domains entry seems >> to be empty, so I'd assume that -name foobar will work whereas in >> fact, -name foobar@REDACTED is required otherwise net_kernel won't >> start. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Geoff Cant From ok@REDACTED Fri Aug 3 00:19:43 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 3 Aug 2012 10:19:43 +1200 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: <1343909141.62398.YahooMailNeo@web111404.mail.gq1.yahoo.com> References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5018EB2B.5030700@gmail.com> <1343846574.80867.YahooMailNeo@web111403.mail.gq1.yahoo.com> <1343909141.62398.YahooMailNeo@web111404.mail.gq1.yahoo.com> Message-ID: <9385AF6E-E800-4557-B56F-CE37E7C0AD59@cs.otago.ac.nz> On 3/08/2012, at 12:05 AM, Thomas Lindgren wrote: > > Another approach might be to use a heuristic tool a la xref to detect "suspicious" string literals. Not sure if that helps. Now _that_ is possible and could well be helpful. For that matter, I invite you to consider this example in pure ASCII: 1> c(snark). {ok,snark} where in the shell m% cat snark.erl -module(snark). -export([f/0]). f() -> [ '', '', '', '', '', '', '', '' ]. but in my text editor, f() -> [ '^@', '^A', '^B', '^C', '^D', '^E', '^F', '^G' ]. It would probably be a Good Thing if Erlang gave at least a warning about control characters in atom, string, or character literals written without using \ . I'd write this up as an EEP if EEPS didn't have to be in Markdown these days. From nem@REDACTED Fri Aug 3 00:28:10 2012 From: nem@REDACTED (Geoff Cant) Date: Thu, 2 Aug 2012 15:28:10 -0700 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: References: <09A191ED-C75E-487C-AE5C-5F1AA6436685@gmail.com> Message-ID: <7A58D0DA-82DE-449E-BD56-C33518036E38@erlang.geek.nz> Oh hey, that rabbit-hole goes way deeper. My system (dns/inet_db config is setup depending on os:type(), mac os gets generic unix treatment). There's a whole lot going on in inet_config to configure inet_db which affects name lookup. But the bit that figures out the hostname for my machine seems to boil down to: 5> {ok, U} = gen_udp:open(0, []). {ok,#Port<0.597>} 6> inet:gethostname(U). {ok,"Balder.local"}. Which seems to be more or less the result of calling 'man 3 gethostname'. Bleh. You could just cheat and make the hostname part of your longname either a) the hostname of the system you're trying to connect to, or b) a local ip address converted to an atom (i.e. '127.0.0.1') if you make a hidden connection. It's a little non-kosher, but generally works. Erlang distribution is about the only thing I use on a regular basis that expects DNS to be configured such that: 1. Any machine can construct its own hostname. 2. These hostnames can be passed off to any third party, and they can subsequently use that hostname to connect back to the original host. In most places I've worked, 1 is dicey but fixable, and 2 prompts weird looks from the ops/network team. Good luck, -Geoff On 2012-08-02, at 14:58 , Geoff Cant wrote: > > From (hazy) memory, longnames are configured using > net_adm:localhost(). This differs from the local host name resolution > methods given in the thread in that it uses the contents of > 'resolv.conf' (specifically the domain option) to construct the FQDN > from the hostname. > > https://github.com/erlang/otp/blob/master/lib/kernel/src/net_adm.erl#L74 > > > The last time I tried to write a reliable distribution based > start/stop/status control script, I'm pretty sure I ended up inspecting > the machine configuration, then manually starting distribution via > something like: > > https://github.com/archaelus/egc_examples/blob/master/rescriptmsh#L6 > > to ensure that I would get the name I wanted. Unfortunately I don't have > permission yet to release the control script that implements all this. > > Cheers, > -Geoff > > Tim Watson writes: > >> Any takers for this? I can't be the only person who's had to figure >> this out. The context is a distributed systems testing framework that >> needs to support resolving 'localhost' to a proper host name so the >> framework can become a hidden node and interact with the other erlang >> nodes It is testing against. Having to specify the real host name >> would make configuring the tool across different machines and >> development environments unpleasantly complicated so figuring out the >> right hostname is a boon in terms of keeping the configuration >> overhead down, but doing this for long names nodes is proving highly >> awkward. >> >> Can anyone suggest a good portable solution? >> >> On 31 Jul 2012, at 13:43, Tim Watson wrote: >> >>> Is there a way to calculate the hostname reliably across platforms >>> in Erlang? I have a non-distributed node that I wish to become a >>> distributed node. Normally I call net_kernel:start([Name, >>> shortnames]) and this is just fine. It also works with [Name, >>> longnames] *sometimes and on some systems* - but other times it >>> pukes. I've tried looking in the 'domain' or 'search' entries from >>> inet:get_rc/0 but these are not always populated, even when dns >>> config is clearly in place. I've also tried using >>> inet_db:get_searchlist/0' but again, sometimes this returns [[]] but >>> net_kernel:start([foobar, longnames]) doesn't work, whereas doing >>> net_kernel:start([foobar@REDACTED, longnames]) does. >>> >>> Am I missing something incredibly obvious here? *is* there actually >>> a simple way of determining what the proper fqdn for the machine >>> should be, without breaking out to the os? I had even considered >>> doing inet:gethostbyname/1 but again, the search domains entry seems >>> to be empty, so I'd assume that -name foobar will work whereas in >>> fact, -name foobar@REDACTED is required otherwise net_kernel won't >>> start. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > Geoff Cant > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Geoff Cant From watson.timothy@REDACTED Fri Aug 3 00:51:53 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 2 Aug 2012 23:51:53 +0100 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: <168765F1-E444-411C-895E-2B9CFB4C5282@feuerlabs.com> References: <6b095368-ace4-461f-adbd-243b36a8d4d1@knuth> <168765F1-E444-411C-895E-2B9CFB4C5282@feuerlabs.com> Message-ID: On 2 August 2012 22:47, Ulf Wiger wrote: > > This works on my Mac: > > hostname() -> > case {inet_db:gethostname(),inet_db:res_option(domain)} of > {H,D} when is_list(D), is_list(H), > length(D)> 0, length(H)>0 -> > H ++ "." ++ D; > Other -> > error({hostname, Other}) > end. Sweet, thanks Ulf. I'll try that elsewhere, though it'll mean I've got to carefully audit new OTP releases just in case the APIs change or move, but I think pinning the framework to "R15.*" with rebar is probably sufficient to do so for now. From watson.timothy@REDACTED Fri Aug 3 00:56:06 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 2 Aug 2012 23:56:06 +0100 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: References: <09A191ED-C75E-487C-AE5C-5F1AA6436685@gmail.com> Message-ID: On 2 August 2012 22:58, Geoff Cant wrote: > > From (hazy) memory, longnames are configured using > net_adm:localhost(). This differs from the local host name resolution > methods given in the thread in that it uses the contents of > 'resolv.conf' (specifically the domain option) to construct the FQDN > from the hostname. Now that looks a bit more promising... But what of systems where resolv.conf isn't used? > > The last time I tried to write a reliable distribution based > start/stop/status control script, I'm pretty sure I ended up inspecting > the machine configuration, then manually starting distribution via > something like: > > https://github.com/archaelus/egc_examples/blob/master/rescriptmsh#L6 > Ok that's interesting, I'll play around with it and see how things behave across unices. Thanks for the pointer! > to ensure that I would get the name I wanted. Unfortunately I don't have > permission yet to release the control script that implements all this. > Not to worry. My framework starts as a dist node only so it can make hidden connections if required. By and large, it just run scripts like the one you're describing to do the actual work of interfacing with the various nodes in a system under test, but I wanted rpc support where it made sense (as it did for the initial use case). Thanks again for pointing out net_adm - will look into that when I get back from holiday the week after next! :) From watson.timothy@REDACTED Fri Aug 3 01:02:01 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 3 Aug 2012 00:02:01 +0100 Subject: [erlang-questions] reliably figure out hostname In-Reply-To: <7A58D0DA-82DE-449E-BD56-C33518036E38@erlang.geek.nz> References: <09A191ED-C75E-487C-AE5C-5F1AA6436685@gmail.com> <7A58D0DA-82DE-449E-BD56-C33518036E38@erlang.geek.nz> Message-ID: On 2 August 2012 23:28, Geoff Cant wrote: > > You could just cheat and make the hostname part of your longname either a) the hostname of the system you're trying to connect to, or b) a local ip address converted to an atom (i.e. '127.0.0.1') if you make a hidden connection. It's a little non-kosher, but generally works. > This has crossed my mind! The gen_udp based approach doesn't fly, as you noticed too, for the same reasons that calling 'hostname' via the OS doesn't always tell you whether or not `-name foo` will work without the FQDN. I'm not actually sure how using an IP address plays with short versus longnames - I'll experiment with that, thanks. > > Erlang distribution is about the only thing I use on a regular basis that expects DNS to be configured such that: > 1. Any machine can construct its own hostname. > 2. These hostnames can be passed off to any third party, and they can subsequently use that hostname to connect back to the original host. > > In most places I've worked, 1 is dicey but fixable, and 2 prompts weird looks from the ops/network team. > Yep. The main thing I'm using the test framework for completely depends on distributed Erlang though, so there's little point in skirting the issue. I'll probably simply force users to enter the full hostname for longnames and provide a way to override the resolution of localhost per machine using a config file, so developers don't have to keep editing the test setup when they check out from source control. For those using shortnames, none of this seems to really matter. From steven.charles.davis@REDACTED Fri Aug 3 01:53:44 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 2 Aug 2012 16:53:44 -0700 (PDT) Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: References: Message-ID: <527dbd0b-3eea-4042-80f3-4408d126637a@googlegroups.com> > > A "string" is an arbitrary representation of some original binary stream > that has been stripped of any information about its encoding format - > whether this was a direct transformation or not. No wonder nobody can agree. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jws@REDACTED Fri Aug 3 02:15:37 2012 From: jws@REDACTED (Jeff Schultz) Date: Fri, 3 Aug 2012 10:15:37 +1000 Subject: [erlang-questions] correct terminology for referring to strings In-Reply-To: <9385AF6E-E800-4557-B56F-CE37E7C0AD59@cs.otago.ac.nz> References: <5017C60F.1050807@gmail.com> <5017E5D5.2030508@gmail.com> <5018EB2B.5030700@gmail.com> <1343846574.80867.YahooMailNeo@web111403.mail.gq1.yahoo.com> <1343909141.62398.YahooMailNeo@web111404.mail.gq1.yahoo.com> <9385AF6E-E800-4557-B56F-CE37E7C0AD59@cs.otago.ac.nz> Message-ID: <20120803001537.GA25753@mulga.csse.unimelb.edu.au> On Fri, Aug 03, 2012 at 10:19:43AM +1200, Richard O'Keefe wrote: > m% cat snark.erl > -module(snark). > -export([f/0]). > f() -> [ '', '', '', '', '', '', '', '' ]. > > but in my text editor, > > f() -> [ '^@', '^A', '^B', '^C', '^D', '^E', '^F', '^G' ]. > > It would probably be a Good Thing if Erlang gave at least a warning > about control characters in atom, string, or character literals > written without using \ . Could we just disallow control characters altogether? Or at least require an option to enable them. Please. They're just accidents waiting to happen when UTF-8 is eventually implemented and serve no useful purpose that I can see. Jeff Schultz From erlang@REDACTED Fri Aug 3 11:12:02 2012 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 3 Aug 2012 11:12:02 +0200 Subject: [erlang-questions] bug in erlang:size Message-ID: I think this is a bug: $erl 1> B = <<1:17>>. <<0,0,1:1>> 2> size(B) 2 B is not a binary or tuple to size(B) should give badarg /Joe From maruthavanan_s@REDACTED Fri Aug 3 12:35:09 2012 From: maruthavanan_s@REDACTED (Maruthavanan Subbarayan) Date: Fri, 3 Aug 2012 06:35:09 -0400 Subject: [erlang-questions] Crash while using mnesia In-Reply-To: References: , Message-ID: Hi Again, When a dump is created due to failure in allocation of memory, Is there a way I can check what was the memory size that was free in the machine from the erl_crash.dump? Thanks,Marutha From: maruthavanan_s@REDACTED To: erlang-questions@REDACTED Date: Thu, 2 Aug 2012 05:00:46 -0400 Subject: [erlang-questions] Crash while using mnesia Hi All, I am an application that receives series of requests and stores in mnesia table for processing and once that is processed it is deleted from the table. I have done this so that I dont miss out any requests. so while starting the application each time, I process the pending requests before I process new requests. It seemed to be running while for couple of days and after that it gets crashed. I could see from crash dump that enough memory was not allocated. But I could not identify where the memory is being used. because the mnesia db directory was hardly 10K including logs. Attached dump is the recent one (http://www.sendspace.com/file/w2b2tm). So I just included a heart while starting up so that it erlang restart itself. But again caused a dump stopping the application without restarting. I am running erlang using a pipe. Kindly help me to identify why these two issues happen. Thanks, Marutha _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From gustav.simonsson@REDACTED Fri Aug 3 13:16:24 2012 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Fri, 03 Aug 2012 12:16:24 +0100 (BST) Subject: [erlang-questions] Crash while using mnesia In-Reply-To: Message-ID: <34b381c2-e7da-404a-81b2-cdeea74764e8@knuth> Hi Maruthavanan, Looking at the crash dump I see the process started with request_processor:start_queued_items/3 is the one using all that memory. You have to look at what Erlang terms, and how many, are stored in the state of that process. If you can paste the code of your request_processor module that would be helpful. You can also look at http://www.erlang.org/doc/efficiency_guide/advanced.html#id68680 to see the memory size of various Erlang terms. // Gustav Sent from my PC ----- Original Message ----- > From: "Maruthavanan Subbarayan" > To: erlang-questions@REDACTED > Sent: Friday, 3 August, 2012 12:35:09 PM > Subject: Re: [erlang-questions] Crash while using mnesia > > > > Hi Again, > > > When a dump is created due to failure in allocation of memory, Is > there a way I can check what was the memory size that was free in > the machine from the erl_crash.dump? > > Thanks, > Marutha > > > > From: maruthavanan_s@REDACTED > To: erlang-questions@REDACTED > Date: Thu, 2 Aug 2012 05:00:46 -0400 > Subject: [erlang-questions] Crash while using mnesia > > > > > Hi All, > > I am an application that receives series of requests and stores in > mnesia table for processing and once that is processed it is deleted > from the table. I have done this so that I dont miss out any > requests. so while starting the application each time, I process the > pending requests before I process new requests. > > It seemed to be running while for couple of days and after that it > gets crashed. I could see from crash dump that enough memory was not > allocated. But I could not identify where the memory is being used. > because the mnesia db directory was hardly 10K including logs. > Attached dump is the recent one > (http://www.sendspace.com/file/w2b2tm). > > So I just included a heart while starting up so that it erlang > restart itself. But again caused a dump stopping the application > without restarting. I am running erlang using a pipe. > > Kindly help me to identify why these two issues happen. > > Thanks, > Marutha > > _______________________________________________ erlang-questions > mailing list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From cgsmcmlxxv@REDACTED Fri Aug 3 13:56:42 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Fri, 3 Aug 2012 13:56:42 +0200 Subject: [erlang-questions] zlib: how many bytes were used during the uncompression (one call of zlib:inflate/2) In-Reply-To: <88246A8F-1DEE-4CDE-8E1F-CBF3F0046966@gmail.com> References: <09D55B88-E90E-4E19-AD78-92BF3DB6F46C@gmail.com> <88246A8F-1DEE-4CDE-8E1F-CBF3F0046966@gmail.com> Message-ID: Hi, The reason you get no answer is because nobody can know the answer. And the reason no one can know the answer is that you haven't provided any information about what you have as compressed data. For example, a simple text can be compressed to in between 30% and 80% from its decompressed size (these numbers are not official, just examples), so, when deflated, the size can reach from 1.25 to 3.33 times the size of compressed text size. >From my own experience (these are simple observations only), the range of compression is in between 10% and 100% (no compression could be applied) from the uncompressed data size. That means, the range of deflated data size is from 1 to 10 times the size of compressed data. Again, these numbers are for the sake of discussion and may not apply for your case. You can find more info about zlib from: http://zlib.net/ http://en.wikipedia.org/wiki/Zlib http://en.wikipedia.org/wiki/LZ77 (LZ77 only) CGS On Thu, Aug 2, 2012 at 11:18 PM, Zabrane Mickael wrote: > Need help on this please! > > On Aug 2, 2012, at 5:19 PM, Zabrane Mickael wrote: > > > Hi guys, > > > > I'm playing a bit with the zlib module today. > > > > Let say I wan to uncompress a GzipData binary with zlib:inflate/2: > > InflatedData = zlib:inflate(Z, GzipData). > > > > In case of success, I want to know how many bytes from GzipData were used > > internally to get the InflatedData? > > > > Regards, > > Zabrane > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgsmcmlxxv@REDACTED Fri Aug 3 13:58:49 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Fri, 3 Aug 2012 13:58:49 +0200 Subject: [erlang-questions] zlib: how many bytes were used during the uncompression (one call of zlib:inflate/2) In-Reply-To: References: <09D55B88-E90E-4E19-AD78-92BF3DB6F46C@gmail.com> <88246A8F-1DEE-4CDE-8E1F-CBF3F0046966@gmail.com> Message-ID: Oh! Sorry, I got your problem wrongly. Please, ignore my previous answer. On Fri, Aug 3, 2012 at 1:56 PM, CGS wrote: > Hi, > > The reason you get no answer is because nobody can know the answer. And > the reason no one can know the answer is that you haven't provided any > information about what you have as compressed data. For example, a simple > text can be compressed to in between 30% and 80% from its decompressed size > (these numbers are not official, just examples), so, when deflated, the > size can reach from 1.25 to 3.33 times the size of compressed text size. > From my own experience (these are simple observations only), the range of > compression is in between 10% and 100% (no compression could be applied) > from the uncompressed data size. That means, the range of deflated data > size is from 1 to 10 times the size of compressed data. Again, these > numbers are for the sake of discussion and may not apply for your case. > > You can find more info about zlib from: > http://zlib.net/ > http://en.wikipedia.org/wiki/Zlib > http://en.wikipedia.org/wiki/LZ77 (LZ77 only) > > CGS > > > > > On Thu, Aug 2, 2012 at 11:18 PM, Zabrane Mickael wrote: > >> Need help on this please! >> >> On Aug 2, 2012, at 5:19 PM, Zabrane Mickael wrote: >> >> > Hi guys, >> > >> > I'm playing a bit with the zlib module today. >> > >> > Let say I wan to uncompress a GzipData binary with zlib:inflate/2: >> > InflatedData = zlib:inflate(Z, GzipData). >> > >> > In case of success, I want to know how many bytes from GzipData were >> used >> > internally to get the InflatedData? >> > >> > Regards, >> > Zabrane >> > >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy.heater@REDACTED Fri Aug 3 14:40:33 2012 From: jeremy.heater@REDACTED (Jeremy Heater) Date: Fri, 3 Aug 2012 14:40:33 +0200 Subject: [erlang-questions] Why does compile:forms spawn a linked process? Message-ID: Hello everyone, We've been using compile:forms to create a dynamically generated module, and while testing we realized - after we started receiving unexpected process death notifications {'EXIT', Pid, normal} - that it spawns a linked process. Compile module code in question: > forms(File) -> forms(File, ?DEFAULT_OPTIONS). > > forms(Forms, Opts) when is_list(Opts) -> > do_compile({forms,Forms}, [binary|Opts++env_default_opts()]); > forms(Forms, Opt) when is_atom(Opt) -> > forms(Forms, [Opt|?DEFAULT_OPTIONS]). > > do_compile(Input, Opts0) -> > Opts = expand_opts(Opts0), > Self = self(), > Serv = spawn_link(fun() -> internal(Self, Input, Opts) end), > receive > {Serv,Rep} -> Rep > end. > > internal(Master, Input, Opts) -> > Master ! {self(), try internal(Input, Opts) > catch error:Reason -> {error, Reason} > end}. Is there a particular reason spawn_link is used here instead of calling internal(Input, Opts) directly? For our application it is less practical, but is there some reason justifying its use? Best regards, Jeremy Heater. From zabrane3@REDACTED Fri Aug 3 14:45:31 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 3 Aug 2012 14:45:31 +0200 Subject: [erlang-questions] zlib: how many bytes were used during the uncompression (one call of zlib:inflate/2) In-Reply-To: References: <09D55B88-E90E-4E19-AD78-92BF3DB6F46C@gmail.com> Message-ID: <6122BA08-6374-4D9B-9883-B54EFAEA1912@gmail.com> Hi JD, > Is your question how much of GzipData was used to produce > InflatedData? ie if GzipData is 1kb, how much of that 1kb was actually > needed? Exactly what I'm looking after. > If so, isn't the idea of compression that every single byte in > GzipData was needed to get the original? Yes. Zlib uncompressor needs a buffer of input (compressed data) at a time to get the job done. You don't need to feed it with all the compressed data at once. Chunk by chunk ... is ok. Let me state my problem differently. One of the less know feature of GZIP is this one: GZIP file + GZIP file + ... + GZIP file = GZIP file. The concatenation of multiple GZIPPed files is a valid GZIP file that can be uncompresed with the unix "gzip" command or by simply calling: {ok, H} = file:read(ContatenatedGZIPs, [read, raw, [compressed]), {ok, Data} = file:read(H, 1024). [...] The problem when reading this kind of concatenated GZIP is that you're no longer able to distinguish between your files. You can only read them as one big stream of data (i.e one big file). Let say I have 3 GZIP files concatenated in one: [begin offset GZIP1 = 0] GZIP1 (compressed size = 10 bytes) [begin offset GZIP2 = 10] GZIP2 (compressed size = 5 bytes) [begin offset GZIP3 = 15] GZIP3 (compressed size = 25 bytes) [end offset of GZIP3 = 40] I'm interested on GZIP2 file and need a reliable way to get where its starts from and where it finishs (the compressed offsets [10,15]). Hope my problem is clear now. Help appreciated guys!!! Regards, Zabrane > > JD > > On 2 August 2012 17:19, Zabrane Mickael wrote: >> Hi guys, >> >> I'm playing a bit with the zlib module today. >> >> Let say I wan to uncompress a GzipData binary with zlib:inflate/2: >> InflatedData = zlib:inflate(Z, GzipData). >> >> In case of success, I want to know how many bytes from GzipData were used >> internally to get the InflatedData? >> >> Regards, >> Zabrane >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From kaiduanx@REDACTED Fri Aug 3 19:40:36 2012 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Fri, 3 Aug 2012 13:40:36 -0400 Subject: [erlang-questions] Question about gen_tcp on multiple cores Message-ID: Hi, We all know in Erlang, writing TCP server is quite easy. For example, call gen_tcp:accept from multiple Erlang processes, and for each TCP connection an Erlang process is created to handle the send and receive packets ... However I would like to know more details about the underlying C layer, here I assume kernel_poll is enabled on Linux, and thread means the C thread instead of Erlang process. How Erlang utilizes multiple cores for accepting connections/sending/receiving data? 1) How many threads are created for handling the network operation such as accept, connect, send and receive? 2) Is epoll_wait() called from multiple threads with same epoll set? I looked the inet_drv.c, unfortunately I can't figure out the answer from there. Can someone with the knowledge provide the answer, or more clues? Thanks for help, /Kaiduan From wde@REDACTED Fri Aug 3 22:58:38 2012 From: wde@REDACTED (wde) Date: Fri, 3 Aug 2012 22:58:38 +0200 Subject: [erlang-questions] update erlang shell prompt dynamically Message-ID: <20120803205836.C8D3470000A5@msfrf2113.sfr.fr> Dear Robert, Thank you for your answer. Have a great week end. wde >There is no trick, it can't be done. > >The shell sends off a request to read an erlang expression, this requests contains the prompt. The io-system prints this prompt at the start of every line entered for that input. It is not possible to change the prompt in the middle of a read request. > >The reason it is done this is to be able to sensibly handle concurrent io-requests, each request behaves as an atomic transaction. > >Robert > >----- Original Message ----- >> Hello, >> >> >> I would like to use a custom prompt inside the erlang shell. >> >> To do it, I defined the following environment variable for stdlib : >> {shell_prompt_func,{my_module,my_prompt}}. It works fine ! >> >> In my prompt definition, I set the number of nodes inside the >> cluster. The prompt is updated only when I type something in the >> shell. >> >> ? TOTO[titi@REDACTED](cluster_wait)::> blabla(). >> ** exception error: undefined shell command blabla/0 >> ? TOTO[titi@REDACTED]:3/3::> >> >> >> How could I refresh the prompt when I push "enter" in the shell ? I >> didn't find the trick. >> >> >> Thank you for your help. >> >> >> wde >> >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From steven.charles.davis@REDACTED Sat Aug 4 01:06:29 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Fri, 3 Aug 2012 16:06:29 -0700 (PDT) Subject: [erlang-questions] bug in erlang:size In-Reply-To: References: Message-ID: <19d7d9d8-acdb-4ac3-943e-1944ac8b88e2@googlegroups.com> 3> byte_size(B). 3 On Friday, August 3, 2012 4:12:02 AM UTC-5, Joe Armstrong wrote: > > I think this is a bug: > > $erl > 1> B = <<1:17>>. > <<0,0,1:1>> > 2> size(B) > 2 > > > B is not a binary or tuple to size(B) should give badarg > > /Joe > _______________________________________________ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sat Aug 4 01:35:01 2012 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 03 Aug 2012 16:35:01 -0700 Subject: [erlang-questions] [erlang-bugs] bug in erlang:size In-Reply-To: <19d7d9d8-acdb-4ac3-943e-1944ac8b88e2@googlegroups.com> References: <19d7d9d8-acdb-4ac3-943e-1944ac8b88e2@googlegroups.com> Message-ID: <501C6025.6000107@gmail.com> 4> byte_size(B). 2.125 On 08/03/2012 04:06 PM, Steve Davis wrote: > 3> byte_size(B). > 3 > > > On Friday, August 3, 2012 4:12:02 AM UTC-5, Joe Armstrong wrote: > > I think this is a bug: > > $erl > 1> B = <<1:17>>. > <<0,0,1:1>> > 2> size(B) > 2 > > > B is not a binary or tuple to size(B) should give badarg > > /Joe > _______________________________________________ > > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukasz.kotula@REDACTED Sat Aug 4 06:29:44 2012 From: lukasz.kotula@REDACTED (=?UTF-8?B?xYF1a2FzeiBLb3R1bGE=?=) Date: Sat, 4 Aug 2012 06:29:44 +0200 Subject: [erlang-questions] atom - limited to 255 characters Message-ID: Hi, Few days ago I came across a erlang limitation, atoms cant be larger then 255 characters. I am using asn1ct module to generate hdr,erl files but those generated atoms are to large. Is there any work around ? Best Regards, Lukasz From ok@REDACTED Sat Aug 4 07:29:28 2012 From: ok@REDACTED (Richard O'Keefe) Date: Sat, 4 Aug 2012 17:29:28 +1200 Subject: [erlang-questions] [erlang-bugs] bug in erlang:size In-Reply-To: <501C6025.6000107@gmail.com> References: <19d7d9d8-acdb-4ac3-943e-1944ac8b88e2@googlegroups.com> <501C6025.6000107@gmail.com> Message-ID: <9072C926-A0E9-4278-90B0-3E090F4178D0@cs.otago.ac.nz> On 4/08/2012, at 11:35 AM, Michael Truog wrote: > 4> byte_size(B). > 2.125 I don't think Joe wants to know how to find the size of a binary. He is concerned that size/1 failed to report the error he expected it to. From steven.charles.davis@REDACTED Sat Aug 4 15:00:54 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sat, 4 Aug 2012 06:00:54 -0700 (PDT) Subject: [erlang-questions] [erlang-bugs] bug in erlang:size In-Reply-To: <9072C926-A0E9-4278-90B0-3E090F4178D0@cs.otago.ac.nz> References: <19d7d9d8-acdb-4ac3-943e-1944ac8b88e2@googlegroups.com> <501C6025.6000107@gmail.com> <9072C926-A0E9-4278-90B0-3E090F4178D0@cs.otago.ac.nz> Message-ID: <093b35cf-1467-45ff-94f8-82ab4e0b56ac@googlegroups.com> Yes, indeed. I was pointing out that byte_size also gives a curious answer for bits. I believe that Michael was reinforcing what the expected answer would be as I don't believe 4> is actual repl output. On Saturday, August 4, 2012 12:29:28 AM UTC-5, Richard O'Keefe wrote: > > > On 4/08/2012, at 11:35 AM, Michael Truog wrote: > > > 4> byte_size(B). > > 2.125 > > I don't think Joe wants to know how to find the size of a > binary. He is concerned that size/1 failed to report the > error he expected it to. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Sun Aug 5 06:33:35 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sun, 5 Aug 2012 07:33:35 +0300 Subject: [erlang-questions] timer, bug !? Message-ID: <275D5DFA-9AD8-47C8-A932-B56E0FF23C38@GMAIL.COM> Hello, I am getting a wired eunit test failure. I am sorry, the code is complicated and it cannot be share here. In nutshell, I do have a gen_server that uses timer:send_after(...) to send timeout message to itself, often it uses timeout equals to 0. If some other message arrives before timeout, then timer is cancelled by timer:cancel(...) Sometimes, with probability 1/5 - 1/6 timeout message is duplicated, see log below. However, if I switch to erlang:send_after and erlang:cancel_timer the issue disappears. Doe anyone seen a similar effect? Why timer duplicates message? 07:21:51.683 [error] make timer: <0.183.0> T=0 07:21:51.684 [error] timeout <0.183.0> ... 07:21:51.693 [error] make timer: <0.183.0> T=0 ... 07:21:51.695 [error] cancel timer: <0.183.0> 07:21:51.696 [error] make timer: <0.183.0> T = 0 07:21:51.696 [error] timeout <0.183.0> 07:21:51.697 [error] make timer: <0.183.0> T = 0 07:21:51.698 [error] cancel timer: <0.183.0> 07:21:51.699 [error] make timer: <0.183.0> T = 0 07:21:51.699 [error] timeout <0.183.0> 07:21:51.700 [error] make timer: <0.183.0> T = 0 07:21:51.701 [error] timeout <0.183.0> 07:21:51.701 [error] timeout <0.183.0> Best Regards, Dmitry From oribrost@REDACTED Sun Aug 5 19:40:10 2012 From: oribrost@REDACTED (ori brost) Date: Sun, 5 Aug 2012 20:40:10 +0300 Subject: [erlang-questions] beam.smp is killed by oom Message-ID: We are load testing a server we implemented in Erlang. Memory usage for beam.smp stays doesn't go above 30% memory until we get to roughly 93000 connections. During what appears to be roughly 5 seconds, beam.smp is killed by Linux's oom-killer (I'm assuming that during these 5 seconds, mem usage spikes up). Any idea how we can trace this problem? Some traces from our system: This is what beam.smp prints shortly before it is oom-killed: http://pastebin.com/s4sJqXwY This is dmesg output: http://pastebin.com/wFUw84yj -------------- next part -------------- An HTML attachment was scrubbed... URL: From oribrost@REDACTED Sun Aug 5 19:48:12 2012 From: oribrost@REDACTED (ori brost) Date: Sun, 5 Aug 2012 20:48:12 +0300 Subject: [erlang-questions] beam.smp is killed by oom In-Reply-To: References: Message-ID: We suspect this may be because of os_mon (as it prints some messages before oom-killer kills beam.smp) On Sun, Aug 5, 2012 at 8:40 PM, ori brost wrote: > We are load testing a server we implemented in Erlang. Memory usage for > beam.smp stays doesn't go above 30% memory until we get to roughly 93000 > connections. During what appears to be roughly 5 seconds, beam.smp is > killed by Linux's oom-killer (I'm assuming that during these 5 seconds, mem > usage spikes up). Any idea how we can trace this problem? > > Some traces from our system: > > This is what beam.smp prints shortly before it is oom-killed: > http://pastebin.com/s4sJqXwY > > This is dmesg output: > http://pastebin.com/wFUw84yj > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eriksoe@REDACTED Sun Aug 5 21:00:11 2012 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Sun, 5 Aug 2012 21:00:11 +0200 Subject: [erlang-questions] timer, bug !? In-Reply-To: <275D5DFA-9AD8-47C8-A932-B56E0FF23C38@GMAIL.COM> References: <275D5DFA-9AD8-47C8-A932-B56E0FF23C38@GMAIL.COM> Message-ID: Have you considered that by the time you call cancel, the timeout message may already be on its way to your process? One solution is to make your timeout messages versioned, and check the version at reception time. Another is to clear the inbox of pending timeout messages just after cancel returns. Den 05/08/2012 06.33 skrev "Dmitry Kolesnikov" : > Hello, > > I am getting a wired eunit test failure. I am sorry, the code is > complicated and it cannot be share here. In nutshell, I do have a > gen_server that uses timer:send_after(...) to send timeout message to > itself, often it uses timeout equals to 0. If some other message arrives > before timeout, then timer is cancelled by timer:cancel(...) > > Sometimes, with probability 1/5 - 1/6 timeout message is duplicated, see > log below. However, if I switch to erlang:send_after and > erlang:cancel_timer the issue disappears. Doe anyone seen a similar effect? > Why timer duplicates message? > > > 07:21:51.683 [error] make timer: <0.183.0> T=0 > 07:21:51.684 [error] timeout <0.183.0> > ... > 07:21:51.693 [error] make timer: <0.183.0> T=0 > ... > 07:21:51.695 [error] cancel timer: <0.183.0> > 07:21:51.696 [error] make timer: <0.183.0> T = 0 > 07:21:51.696 [error] timeout <0.183.0> > 07:21:51.697 [error] make timer: <0.183.0> T = 0 > 07:21:51.698 [error] cancel timer: <0.183.0> > 07:21:51.699 [error] make timer: <0.183.0> T = 0 > 07:21:51.699 [error] timeout <0.183.0> > 07:21:51.700 [error] make timer: <0.183.0> T = 0 > 07:21:51.701 [error] timeout <0.183.0> > 07:21:51.701 [error] timeout <0.183.0> > > Best Regards, Dmitry > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth.lundin@REDACTED Mon Aug 6 00:08:48 2012 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Mon, 6 Aug 2012 00:08:48 +0200 Subject: [erlang-questions] atom - limited to 255 characters In-Reply-To: References: Message-ID: We are aware of the problem with the ASN.1 compiler when generating code for certain asn1 modules containing very long names. This is because these names and combinations of names are mapped to records ( which have names as atoms). Until we find and implement another way of mapping these names in the asn1 compiler the easiest workaround is to change the names inside the asn1 module source to shorter abbreviations. Changes like that will not impact the representaion in BER or PER. Kenneth, Erlang/OTP at Ericsson Den 4 aug 2012 06:30 skrev "?ukasz Kotula" : > Hi, > > Few days ago I came across a erlang limitation, atoms cant be larger then > 255 characters. > I am using asn1ct module to generate hdr,erl files but those generated > atoms are to large. > Is there any work around ? > > Best Regards, > Lukasz > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ghenry@REDACTED Mon Aug 6 00:34:12 2012 From: ghenry@REDACTED (Gavin Henry) Date: Sun, 5 Aug 2012 23:34:12 +0100 Subject: [erlang-questions] Adding StartTLS support to the newly merged eldap? In-Reply-To: References: Message-ID: <2687536686187890495@unknownmsgid> Hi all, I'd contacted Torbj?rn T?rnkvist about sponsoring StartTLS support in eldap and he informed me it's now part of core. I'm no at any type of level to do this myself right now (I'm very new to Erlang and am just reading Programming Erlang by Joe Armstrong), but wondered if we could sponsor the addition as we'd like it for ejabberd. I'm also ghenry@REDACTED so can help on that side of things for testing etc. Hi, I presume no one is interested is this project then? Thanks, Gavin. -------------- next part -------------- An HTML attachment was scrubbed... URL: From overminddl1@REDACTED Mon Aug 6 00:51:45 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Sun, 5 Aug 2012 16:51:45 -0600 Subject: [erlang-questions] Adding StartTLS support to the newly merged eldap? In-Reply-To: <2687536686187890495@unknownmsgid> References: <2687536686187890495@unknownmsgid> Message-ID: On Sun, Aug 5, 2012 at 4:34 PM, Gavin Henry wrote: > Hi all, > > > I'd contacted Torbj?rn T?rnkvist about sponsoring StartTLS support in > > eldap and he informed me it's now part of core. > > > I'm no at any type of level to do this myself right now (I'm very new > > to Erlang and am just reading Programming Erlang by Joe Armstrong), > > but wondered if we could sponsor the addition as we'd like it for > > ejabberd. I'm also ghenry@REDACTED so can help on that side of > > things for testing etc. > > > Hi, > > I presume no one is interested is this project then? Interested yes, but likely many of us do not currently have the skills. Perhaps just post again on occasion and see if anyone catches this that does have the skills and is interested. From maruthavanan_s@REDACTED Mon Aug 6 11:30:20 2012 From: maruthavanan_s@REDACTED (Maruthavanan Subbarayan) Date: Mon, 6 Aug 2012 05:30:20 -0400 Subject: [erlang-questions] Partial disk based and partial memory based terms Message-ID: Hi, I am building an application that would receive requests on bulk and process at back-end. I wrote an code where requested would be inserted into mnesia and the would be processed later.But ended up in a situation where started receiving enormous requests and processing of the requests took more time. So gradually, I faced couple of times where erlang got crashed due to insufficient memory allocation on RAM. There were around 300K records while it got crashed. Is there any workaround where I can keep only 10-20% of the total records in RAM and remaining in disk at any point of time. I have Gigs of disk space and only Megs of RAM. I am looking this as an alternative because It would also give memory for other process in the machine without occupying it unnecessarily. Thanks in advance,Marutha -------------- next part -------------- An HTML attachment was scrubbed... URL: From bgustavsson@REDACTED Mon Aug 6 12:23:16 2012 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 6 Aug 2012 12:23:16 +0200 Subject: [erlang-questions] Why does compile:forms spawn a linked process? In-Reply-To: References: Message-ID: On Fri, Aug 3, 2012 at 2:40 PM, Jeremy Heater wrote: > Is there a particular reason spawn_link is used here instead of > calling internal(Input, Opts) directly? For our application it is less > practical, but is there some reason justifying its use? Yes, there are several good reasons for running the compilation in a temporary process. There is no good reason for leaving an EXIT message in the caller's message queue, however, so in R15B02 the temporary process will be monitored instead of linked. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From ali.sabil@REDACTED Mon Aug 6 12:49:19 2012 From: ali.sabil@REDACTED (Ali Sabil) Date: Mon, 6 Aug 2012 12:49:19 +0200 Subject: [erlang-questions] Partial disk based and partial memory based terms In-Reply-To: References: Message-ID: On Mon, Aug 6, 2012 at 11:30 AM, Maruthavanan Subbarayan wrote: > Hi, > > I am building an application that would receive requests on bulk and process > at back-end. I wrote an code where requested would be inserted into mnesia > and the would be processed later.But ended up in a situation where started > receiving enormous requests and processing of the requests took more time. > So gradually, I faced couple of times where erlang got crashed due to > insufficient memory allocation on RAM. There were around 300K records while > it got crashed. > > Is there any workaround where I can keep only 10-20% of the total records in > RAM and remaining in disk at any point of time. I have Gigs of disk space > and only Megs of RAM. I am looking this as an alternative because It would > also give memory for other process in the machine without occupying it > unnecessarily. > > Thanks in advance, > Marutha > If you use a 64bits node and add configure your OS to have plenty of swap space, it shouldn't crash. From ghenry@REDACTED Mon Aug 6 15:02:06 2012 From: ghenry@REDACTED (Gavin Henry) Date: Mon, 6 Aug 2012 14:02:06 +0100 Subject: [erlang-questions] Adding StartTLS support to the newly merged eldap? In-Reply-To: References: <2687536686187890495@unknownmsgid> Message-ID: >> Hi, >> >> I presume no one is interested is this project then? > > Interested yes, but likely many of us do not currently have the > skills. Perhaps just post again on occasion and see if anyone catches > this that does have the skills and is interested. OK, thanks. Will do, but will carry on learning! Gavin. -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry@REDACTED Open Source. Open Solutions(tm). http://www.suretecsystems.com/ Suretec Systems is a limited company registered in Scotland. Registered number: SC258005. Registered office: 24 Cormack Park, Rothienorman, Inverurie, Aberdeenshire, AB51 8GL. Subject to disclaimer at http://www.suretecgroup.com/disclaimer.html Do you know we have our own VoIP provider called SureVoIP? See http://www.surevoip.co.uk Did you see our API? http://www.surevoip.co.uk/api From rory@REDACTED Mon Aug 6 17:41:18 2012 From: rory@REDACTED (Rory Byrne) Date: Mon, 6 Aug 2012 17:41:18 +0200 Subject: [erlang-questions] Adding StartTLS support to the newly merged eldap? In-Reply-To: <2687536686187890495@unknownmsgid> References: <2687536686187890495@unknownmsgid> Message-ID: <20120806154118.GB13487@almeida.jinsky.com> Hi Gavin, I just took a quick look at this and it seems that ejabberd is using a very different version of eldap than the one that comes with Erlang/OTP. It looks like they might have integrated a version of eldap way back and have made numerous modifications to it. This might not work out so bad for you as I imagine that it will be easier to get a STARTTLS patch accepted in ejabberd than in OTP. Then again, it might be that ejabberd plans to start using the OTP version of eldap whenever it's available. You might want to ask them about this. Do you just want STARTTLS support or do you also want the client to be able to use TLS credentials to authenticate itself to the server? The reason I ask is that neither versions of eldap seem to have support for SASL binding at the moment, so that would need to be added also. Rory On Sun, Aug 05, 2012 at 11:34:12PM +0100, Gavin Henry wrote: > Hi all, > > > I'd contacted Torbj?rn T?rnkvist about sponsoring StartTLS support in > > eldap and he informed me it's now part of core. > > > I'm no at any type of level to do this myself right now (I'm very new > > to Erlang and am just reading Programming Erlang by Joe Armstrong), > > but wondered if we could sponsor the addition as we'd like it for > > ejabberd. I'm also ghenry@REDACTED so can help on that side of > > things for testing etc. > > > Hi, > > I presume no one is interested is this project then? > > Thanks, > > Gavin. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ghenry@REDACTED Mon Aug 6 18:14:48 2012 From: ghenry@REDACTED (Gavin Henry) Date: Mon, 6 Aug 2012 17:14:48 +0100 Subject: [erlang-questions] Adding StartTLS support to the newly merged eldap? In-Reply-To: <20120806154118.GB13487@almeida.jinsky.com> References: <2687536686187890495@unknownmsgid> <20120806154118.GB13487@almeida.jinsky.com> Message-ID: <-5580585911951025444@unknownmsgid> Hi Rory (sorry for top posting. My mail client sucks), That's what I thought. I'm not sure why though or whether it can be pulled out again. In our situation we're not using cliebt certificate based TLS with SASL EXTERNAL. But it should be added at the same time rather than an incomplete patch. Thanks. > Hi Gavin, > > I just took a quick look at this and it seems that ejabberd is using > a very different version of eldap than the one that comes with > Erlang/OTP. It looks like they might have integrated a version of > eldap way back and have made numerous modifications to it. > > This might not work out so bad for you as I imagine that it will > be easier to get a STARTTLS patch accepted in ejabberd than in OTP. > Then again, it might be that ejabberd plans to start using the OTP > version of eldap whenever it's available. You might want to ask > them about this. > > Do you just want STARTTLS support or do you also want the client > to be able to use TLS credentials to authenticate itself to the > server? The reason I ask is that neither versions of eldap seem > to have support for SASL binding at the moment, so that would > need to be added also. > > Rory > > > On Sun, Aug 05, 2012 at 11:34:12PM +0100, Gavin Henry wrote: >> Hi all, >> >> >> I'd contacted Torbj?rn T?rnkvist about sponsoring StartTLS support in >> >> eldap and he informed me it's now part of core. >> >> >> I'm no at any type of level to do this myself right now (I'm very new >> >> to Erlang and am just reading Programming Erlang by Joe Armstrong), >> >> but wondered if we could sponsor the addition as we'd like it for >> >> ejabberd. I'm also ghenry@REDACTED so can help on that side of >> >> things for testing etc. >> >> >> Hi, >> >> I presume no one is interested is this project then? >> >> Thanks, >> >> Gavin. > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From piotr.koper@REDACTED Mon Aug 6 18:36:21 2012 From: piotr.koper@REDACTED (Piotr Koper) Date: Mon, 6 Aug 2012 18:36:21 +0200 Subject: [erlang-questions] More Erlang Users' groups In-Reply-To: References: Message-ID: On Wed, Jul 4, 2012 at 11:56 AM, Ahmed Omar wrote: > And actually latest ones were created upon request of members from these > areas. They are still subgroups under one group though. > > Hi, New subgroup - EUG-POLAND, please join. Thanks Ahmed. Cheers, Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Mon Aug 6 18:36:51 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 6 Aug 2012 18:36:51 +0200 Subject: [erlang-questions] Warning: NOT OPTIMIZED: sub binary is used or returned Message-ID: <84C83563-0075-4308-8B24-09327F1CF24D@gmail.com> Hi guys, I'm communicating with linked-in driver using the following code: control(Port, Command, Data) -> case port_control(Port, Command, Data) of <<0, Result/binary>> -> Result; % <- LINE 468 handle a good case <<1, Error/binary>> -> {error, binary_to_term(Error)} % <- LINE 469 handle error case end. When compiling, I got this 02 warnings: $ make Recompile: src/pimco.erl src/pimco.erl:468: Warning: NOT OPTIMIZED: sub binary is used or returned src/pimco.erl469: Warning: NOT OPTIMIZED: sub binary used by erlang:binary_to_term/1 make[2]: Nothing to be done for `all'. Is there a way to get rid of these warnings and avoid a full binary copies? Hints? N.B: my box R14B04/R15B01 on OSX/Linux. Regards, Zabrane From rory@REDACTED Mon Aug 6 19:12:20 2012 From: rory@REDACTED (Rory Byrne) Date: Mon, 6 Aug 2012 19:12:20 +0200 Subject: [erlang-questions] Adding StartTLS support to Ejabberd? In-Reply-To: <-5580585911951025444@unknownmsgid> References: <2687536686187890495@unknownmsgid> <20120806154118.GB13487@almeida.jinsky.com> <-5580585911951025444@unknownmsgid> Message-ID: <20120806171220.GC13487@almeida.jinsky.com> On Mon, Aug 06, 2012 at 05:14:48PM +0100, Gavin Henry wrote: > Hi Rory (sorry for top posting. My mail client sucks), Ha, it's been so long since I replied to anything on a list that I'd forgot the rules! I've changed the subject of this email in the hope that someone from the ejabberd community might offer some advice. > That's what I thought. I'm not sure why though or whether it can be > pulled out again. Well, it looks like the current release of ejabberd is supported running on Erlang/OTP versions from R10B9 to R15B, and eldap was only added to the R15B01 release. Also, it looks like the 3.0.0 alpha and beta versions of ejabberd are being supported on Erlang/OTP R12B5 and above, so they won't be dropping their own eldap implementation any time soon. However, they might accept a patch that uses the OTP version of eldap when it's available. Ultimately it would be more beneficial for everyone if the STARTTLS and SASL functionality were added to the OTP version. Certainly if I was writing a patch, I'd much rather it was for the OTP version. There might be another problem with adding this functionality to the ejabberd eldap. I'm not 100% sure about this, but I think that the ssl application that was used in Erlang before R14B (the old default ssl application), did not have the ability to start running SSL over an already established TCP connection. I think this was actually one of the main reasons for the big rewrite of the ssl application in recent years. Basically what this means is that it will not be possible (or just wise) to use STARTTLS when using versions of Erlang/OTP before R14B. Any patch to the ejabberd eldap would need to offer this functionality only when a suitable version of Erlang was running. However, the main downside to patching the OTP version - assuming it even gets accepted - is that you will only have the STARTTLS functionality if you are running ejabberd on top of some future version of Erlang (or a patched R15B01+ version). Would this suit you? > In our situation we're not using cliebt certificate based TLS with > SASL EXTERNAL. But it should be added at the same time rather than an > incomplete patch. Yeah, I reckon it's worth having. From micael.erneborg@REDACTED Mon Aug 6 20:36:28 2012 From: micael.erneborg@REDACTED (micern) Date: Mon, 6 Aug 2012 11:36:28 -0700 (PDT) Subject: [erlang-questions] WX Failed loading "wxe_driver" Message-ID: <1344278188447-4655138.post@n4.nabble.com> Hello! I'm trying to learn Erlang, so I apologize if this is a trivial problem I should have solved myself. I'm failing when using WX, trying to open a window for my application. Yep, Windows 7 seams to be an odd choice of OS, but that is what I've got. Doing this in the monitor fails: -------------------------------- 1> Wx = wx:new(). =ERROR REPORT==== 6-Aug-2012::19:42:42 === WX Failed loading "wxe_driver"@"c:/PROGRA~1/ERL59~1.1/lib/wx-0.99.2/priv" ** exception error: {load_driver,"Det g?r inte att hitta den angivna modulen."} in function wxe_server:start/0 (wxe_server.erl, line 64) in call from wx:new/1 (wx.erl, line 107) 2> Wx. * 1: variable 'Wx' is unbound The corresponding code in an .erl file fails too: ----------------------------------------------- -module(minimal). -include_lib("wx/include/wx.hrl"). -export([start/0]). -compile(export_all). start() -> Wx = wx:new(), Frame = wx:batch(fun() -> create_window(Wx) end), wxWindow:show(Frame), loop(Frame), wx:destroy(), ok. (more code, loop, create_window, etc). ... 7> c(minimal). {ok,minimal} 8> minimal:start(). =ERROR REPORT==== 6-Aug-2012::19:49:23 === WX Failed loading "wxe_driver"@"c:/PROGRA~1/ERL59~1.1/lib/wx-0.99.2/priv" ** exception error: {load_driver,"Det g?r inte att hitta den angivna modulen."} in function wxe_server:start/0 (wxe_server.erl, line 64) in call from wx:new/1 (wx.erl, line 107) in call from minimal:start/0 (minimal.erl, line 16) yep, it bombs @ Wx = wx:new(), ... Starting the debugger get a similar warning, even if the window actually opens. ------------------------------------------------ 2> debugger:start(). =ERROR REPORT==== 6-Aug-2012::19:47:08 === WX Failed loading "wxe_driver"@"c:/PROGRA~1/ERL59~1.1/lib/wx-0.99.2/priv" {ok,<0.40.0>} Digging in Erlang library files on my computer... ------------------------------------------------- Directory .../erl5.9.1/lib/wx-0.99.2 exists. wxe_driver@REDACTED:/PROGRA~1/ERL59~1.1/lib/wx-0.99.2/priv exists. Swedish... ---------- "Det g?r inte att hitta den angivna modulen" = "Not possible to find the module" WHY?!?!?! AND WHAT TO DO?!?!?! /micael -- View this message in context: http://erlang.2086793.n4.nabble.com/WX-Failed-loading-wxe-driver-tp4655138.html Sent from the Erlang Questions mailing list archive at Nabble.com. From cookchrisd@REDACTED Mon Aug 6 21:52:24 2012 From: cookchrisd@REDACTED (Chris Cook) Date: Mon, 6 Aug 2012 20:52:24 +0100 Subject: [erlang-questions] Starting slave with erl_boot_server giving bad_return Message-ID: Good Evening List, I have been playing around trying to get a slave starting with erl_boot_server, using a testapp, in a release state using reltools. I start master with following: ./erl -id master -name '[name]@[host]' -setcookie test -run erl_boot_server start [ip_addr] all starts fine, run: erl_boot_server:add_slave('[slave_ip_addr]') and then try to start the slave: erl -id slave -name '[name]@[host]' -setcookie test -loader inet -loader_debug -hosts [master_ip] -boot /home/chriscook/testapp/rel/bin/start this gives me Erlang R14B02 (erts-5.8.3) [source] [rq:1] [async-threads:0] [kernel-poll:false] =INFO REPORT==== 6-Aug-2012::20:34:41 === application: testapp exited: {bad_return, {{testapp_app,start,[normal,[]]}, {'EXIT', {undef, [{testapp_app,start,[normal,[]]}, {application_master,start_it_old,4}]}}}} type: permanent but when I run the start.boot script like on the machine all the code resides on: ./erl -id master -name '[name]@[host]' -setcookie test -boot start everything start correctly. Can anyone help? The only other information I've come across when googling for problems like this is http://erlang.2086793.n4.nabble.com/Working-with-erl-boot-server-td2107906.html from 2008 with no replies any help would be much appreciated. Regards Chris Cook. From ghenry@REDACTED Mon Aug 6 22:24:53 2012 From: ghenry@REDACTED (Gavin Henry) Date: Mon, 6 Aug 2012 21:24:53 +0100 Subject: [erlang-questions] Adding StartTLS support to eldap (for use Ejabberd) Message-ID: > On Mon, Aug 06, 2012 at 05:14:48PM +0100, Gavin Henry wrote: >> Hi Rory (sorry for top posting. My mail client sucks), > > Ha, it's been so long since I replied to anything on a list that > I'd forgot the rules! This list is tame, as on others we'd both have been nailed! > I've changed the subject of this email in the hope that someone > from the ejabberd community might offer some advice. Ha, naughty! I've created a new thread as it was going off a bit. >> That's what I thought. I'm not sure why though or whether it can be >> pulled out again. > > Well, it looks like the current release of ejabberd is supported > running on Erlang/OTP versions from R10B9 to R15B, and eldap was > only added to the R15B01 release. Also, it looks like the 3.0.0 > alpha and beta versions of ejabberd are being supported on > Erlang/OTP R12B5 and above, so they won't be dropping their own > eldap implementation any time soon. Yes, unfortunately most people that develop LDAP libs forget StartTLS and just go for LDAP over SSL which isn't a standard and is deprecated. StartTLS is a standard - http://www.rfc-editor.org/rfc/rfc4513.txt Then again, the whole point of TLS and *getting it right* is to validate certificates which a lot of libs don't offer or admins don't bother with. RFC 4513 is what needs to be read for this work and it's sponsorship, which also covers the SASL side too. > However, they might accept a patch that uses the OTP version of > eldap when it's available. Ultimately it would be more beneficial > for everyone if the STARTTLS and SASL functionality were added to > the OTP version. Certainly if I was writing a patch, I'd much > rather it was for the OTP version. My company would not sponsor the Ejabberd version of eldap. What would be the point, that helps no one and would be a shortsighted piece of sponsorship on our part :-) > There might be another problem with adding this functionality to > the ejabberd eldap. I'm not 100% sure about this, but I think > that the ssl application that was used in Erlang before R14B (the > old default ssl application), did not have the ability to start > running SSL over an already established TCP connection. I think > this was actually one of the main reasons for the big rewrite of > the ssl application in recent years. Basically what this means is > that it will not be possible (or just wise) to use STARTTLS when > using versions of Erlang/OTP before R14B. Any patch to the ejabberd > eldap would need to offer this functionality only when a suitable > version of Erlang was running. Understood. I'm not up to date on the history. > However, the main downside to patching the OTP version - assuming > it even gets accepted - is that you will only have the STARTTLS > functionality if you are running ejabberd on top of some future > version of Erlang (or a patched R15B01+ version). Would this suit > you? If the patch is done right then I'm sure it wouldn't be too difficult to add initial support to the ejabberd eldap version at the same time, with a caveat that no further support will be provided and the ejabberd team would need to either bring in OTP eldap or maintain it themselves. >> In our situation we're not using cliebt certificate based TLS with >> SASL EXTERNAL. But it should be added at the same time rather than an >> incomplete patch. > > Yeah, I reckon it's worth having. It would certainly add excellent support for enterprises using eldap. Some of our OpenLDAP support customers will *only* use SASL EXTERNAL with X.509 certificate based auth. Thanks, Gavin. -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry@REDACTED Open Source. Open Solutions(tm). http://www.suretecsystems.com/ Suretec Systems is a limited company registered in Scotland. Registered number: SC258005. Registered office: 24 Cormack Park, Rothienorman, Inverurie, Aberdeenshire, AB51 8GL. Subject to disclaimer at http://www.suretecgroup.com/disclaimer.html Do you know we have our own VoIP provider called SureVoIP? See http://www.surevoip.co.uk Did you see our API? http://www.surevoip.co.uk/api From erlang@REDACTED Mon Aug 6 22:33:27 2012 From: erlang@REDACTED (Stefan Marr) Date: Mon, 6 Aug 2012 22:33:27 +0200 Subject: [erlang-questions] Deadline Extended to Aug. 10th, RACES'12: Relaxing Synchronization for Multicore and Manycore Scalability, SPLASH'12 Workshop Message-ID: Deadline Extension: The coincidence of the OOPSLA camera-ready deadline and the RACES'12 deadline has prompted requests for an extension. Accordingly, we are extending the RACES'12 deadline to August 10th, this Friday. We look forward to a great workshop, with a chance to exchange ideas and glean insights. RACES'12 Organizers. ============================================================================== Call for Participation R A C E S 2 0 1 2 Relaxing Synchronization for Multicore and Manycore Scalability Workshop Co-located with SPLASH in Tucson, Arizona Sunday, October 21 Extended Submission deadline: Friday, August 10, 11:59PM PT http://soft.vub.ac.be/races/ ============================================================================== Massively-parallel systems are coming: core counts keep rising - whether conventional cores as in multicore and manycore systems, or specialized cores as in GPUs. Conventional wisdom has been to utilize this parallelism by reducing synchronization to the minimum required to preserve determinism - in particular, by eliminating data races. However, Amdahl's law implies that on highly-parallel systems even a small amount of synchronization that introduces serialization will limit scaling. Thus, we are forced to confront the trade-off between synchronization and the ability of an implementation to scale performance with the number of processors: synchronization inherently limits parallelism. This workshop focuses on harnessing parallelism by limiting synchronization, even to the point where programs will compute inconsistent or approximate rather than exact answers. Organizers: Andrew P. Black, Portland State University Theo D'Hondt, Vrije Universiteit Brussel Doug Kimelman, IBM Thomas J. Watson Research Center Martin Rinard, MIT CSAIL David Ungar, IBM Thomas J. Watson Research Center Theme and Topics ---------------- A new school of thought is arising: one that accepts and even embraces nondeterminism (including data races), and in return is able to dramatically reduce synchronization, or even eliminate it completely. However, this approach requires that we leave the realm of the certain and enter the realm of the merely probable. How can we cast aside the security of correctness, the logic of a proof, and adopt a new way of thinking, where answers are good enough but not certain, and where many processors work together in parallel without quite knowing the states that the others are in? We may need some amount of synchronization, but how much? Or better yet, how little? What mental tools and linguistic devices can we give programmers to help them adapt to this challenge? This workshop focuses on these questions and related ones: harnessing parallelism by limiting synchronization, even to the point where programs will compute inconsistent or approximate rather than exact answers. This workshop aims to bring together researchers who, in the quest for scalability, have been exploring the limits of how much synchronization can be avoided. We invite submissions on any topic related to the theme of the workshop, pro or con. We want to hear from those who have experimented with formalisms, algorithms, data structures, programming languages, and mental models that push the limits. In addition, we hope to hear from a few voices with wilder ideas: those who may not have reduced their notions to practice yet, but who have thoughts that can inspire us as we head towards this yet-uncertain future. For example, biology may yield fruitful insights. The ideal presentation for this workshop will focus on a grand idea, but will be backed by some experimental result. Submission ---------- Authors are invited to submit short position papers, technical papers, or experience reports. Submissions may range from a single paragraph to as long as desired, but the committee can only commit to reading one full page. Nonetheless, we expect that in many cases reviewers will read farther than that. Submissions should be formatted according to the ACM SIG Proceedings style at http://www.acm.org/sigs/publications/proceedings-templates and should be submitted via EasyChair at http://www.easychair.org/conferences/?conf=races2012 in PDF format. PLEASE NOTE: All submissions (except for those retracted by their authors) will be posted on the workshop website, along with reviews, which will be signed by the reviewers, and a rating assigned by the program committee. Further, the submissions to be presented at the workshop will be selected by a vote of all registered attendees. As well, submissions to be published in an official proceedings will be selected by the program committee. Please see the sections below concerning the rationale and details for this process. Program Committee ----------------- Andrew P. Black, Portland State University Yvonne Coady, University of Victoria Tom Van Cutsem, Vrije Universiteit Brussel Theo D'Hondt, Vrije Universiteit Brussel Phil Howard, Portland State University Doug Kimelman, IBM Thomas J. Watson Research Center Eddie Kohler, Harvard SEAS Jim Larus, Microsoft Research Stefan Marr, Vrije Universiteit Brussel Tim Mattson, Intel Paul McKenney, IBM Hannes Payer, University of Salzburg Dan Prenner, IBM Lakshmi Renganarayana, IBM David Ungar, IBM Thomas J. Watson Research Center Martin Vechev, ETH Zurich Important Dates --------------- August 10 Submission deadline. August 29 Reviews sent to authors. September 3 Last date for retraction by authors. September 4 Papers, reviews, ratings posted on web site. Voting opens. September 11 Voting closes. September 14 Notification of papers accepted for presentation and/or publication. August 21 SPLASH early registration deadline. October 21 Workshop. mid-November Camera-ready copy due for papers selected for proceedings. Goals and Outcomes ------------------ We will consider the workshop a success if attendees come away with new insights into fundamental principles, and new ideas for algorithms, data structures, programming languages, and mental models, leading to improving scaling by limiting synchronization, even to the point where programs will compute inconsistent or approximate rather than exact answers. The goal of this workshop is both to influence current programming practice and to initiate the coalescence of a new research community giving rise to a new subfield within the general area of concurrent and parallel programming. Results generated by the workshop will be made persistent via the workshop website and possibly via the ACM Digital Library. The RACES 2012 Review Process and Workshop Presentation Selection Process ========================================================================= David Ungar IBM Thomas J. Watson Research Center PC Chair for Workshop Presentations Technology has changed the economic tradeoffs that once shaped the reviewing process. It has become cheap and easy to share submissions, reviews and the preferences of the attendees. What remains scarce is the number of hours in a day, and as a consequence the time we have in our workshop in which to learn and share with each other. I believe that this change in the balance of factors affords us the opportunity to significantly improve the review and selection processes. Sadly, all too often, those who spend their precious time attending a workshop are not served as well they could be with respect to enlightenment, thought provoking discussions, and being challenged by new ideas. The fault lies not in the people who generously donate their time to serve on program committees and do external reviews. Rather, the fault lies in the process itself. The very notion of acceptance by committee forces us to boil a rich stew of reactions, insights, and opinions, down to a single carrot. As a result, it is common for PC members to come away from a meeting feeling that either some fraud will be perpetrated on the audience by a fundamentally flawed paper, or, more often, feeling that a sin of omission will be committed on the audience by the suppression of a significant but controversial new idea. Sometimes instead of a carrot we get a lump of gristle. There are other, lesser, flaws in this process. Although reviewer anonymity protects negative reviewers from resentment and reprisal, all too often it prevents an open debate that would promote mutual understanding. Further, in some cases anonymity allows a reviewer to cast aspersions on authors without being accountable. Finally, we fail to take maximal advantage of the time and effort spent in creating insightful reviews when we withhold them from the audience. Attendees and readers could benefit from expert reactions as they try to glean the wisdom embedded in the authors' papers. In this workshop, we have an opportunity to try a different process, one that we hope will serve all parties better: All reviews will be signed, all submissions and reviews will be posted on the web (unless an author chooses to retract a submission), and the attendees will be the ones selecting which papers will be presented. Here are the details: --------------------- At least three committee members will review each submission, and each review will be signed. Once all the reviews for a submission are in, they will be sent to the author, who can decide to retract the paper if so desired. Then, all submissions (except any that are retracted) will be posted on the workshop website, along with all reviews and a net score determined for each submission by the program committee. At this point, prior to the workshop, all registered attendees will be invited to read the submissions and the reviews, and vote on which of the papers they want to see presented. Of course, an attendee who so wishes will be free to merely vote according to the recommendation of the PC, or to not vote and to accept the wisdom of the rest of the attendees. But the important point remains: it will be those who will be spending the time in the room who get to decide how that time is spent. Please note that a submission being posted on the workshop website and/or presented at the workshop are not intended to constitute prior publication for purposes of publishing in other workshops, major conferences, or journals. This process is a grand experiment, designed to exploit the technologies we Computer Scientists have created, in order to better serve the advancement of Computer Science. We hope that its potential excites you as much as it excites us! The RACES 2012 Published Proceedings Paper Selection Process ============================================================ Theo D'Hondt Vrije Universiteit Brussel PC Chair for Proceedings Papers We understand that many submitters may want to publish their paper in an official proceedings in addition to having it posted on the workshop website. In order to satisfy that desire, we will publish a proceedings via the ACM Digital Library. To satisfy ACM DL selectivity requirements, a separate and more conventional process will be employed for selecting papers to be included in the published proceedings: Even though all submissions will be posted on the workshop website (unless retracted by the author), the program committee will select a smaller number of papers to be included in the published proceedings based on the signed and posted reviews. Authors of the selected papers will be asked to submit revised and extended papers mid-November, taking into account the reviews and the publisher's guidelines. Page limits for the revised and extended papers to be included in the published proceedings are anticipated to be 10 pages for research papers, and 5 pages for position papers. Please note that inclusion in the ACM Digital Library published proceedings may well be considered to be a prior publication for purposes of publication in other workshops, major conferences, or journals. For that reason, authors may choose to decline to have their submission included in the published proceedings, even if it was presented at the workshop. For questions please contact: races@REDACTED For updates, follow us on Twitter: @races_workshop ============================================================================== From torben.lehoff@REDACTED Mon Aug 6 22:58:38 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Mon, 06 Aug 2012 22:58:38 +0200 Subject: [erlang-questions] timer, bug !? In-Reply-To: <275D5DFA-9AD8-47C8-A932-B56E0FF23C38@GMAIL.COM> References: <275D5DFA-9AD8-47C8-A932-B56E0FF23C38@GMAIL.COM> Message-ID: <50202FFE.2090209@gmail.com> Hi Dmitry, I cannot say for sure, but 0ms timers does not seem to be a super idea regardless of what you are trying to do. Anyway, you have to remember that the timer module starts a server that becomes a bottleneck for all timers. The timers from the erlang module does not have this problem. You may want to have a look at the timer module I have created [1], if not for anything else, then to read about the pros and cons of the different timer options in Erlang. Cheers, ___ /orben [1] https://github.com/lehoff/chronos On 2012-08-05 06:33, Dmitry Kolesnikov wrote: > Hello, > > I am getting a wired eunit test failure. I am sorry, the code is complicated and it cannot be share here. In nutshell, I do have a gen_server that uses timer:send_after(...) to send timeout message to itself, often it uses timeout equals to 0. If some other message arrives before timeout, then timer is cancelled by timer:cancel(...) > > Sometimes, with probability 1/5 - 1/6 timeout message is duplicated, see log below. However, if I switch to erlang:send_after and erlang:cancel_timer the issue disappears. Doe anyone seen a similar effect? Why timer duplicates message? > > > 07:21:51.683 [error] make timer: <0.183.0> T=0 > 07:21:51.684 [error] timeout <0.183.0> > ... > 07:21:51.693 [error] make timer: <0.183.0> T=0 > ... > 07:21:51.695 [error] cancel timer: <0.183.0> > 07:21:51.696 [error] make timer: <0.183.0> T = 0 > 07:21:51.696 [error] timeout <0.183.0> > 07:21:51.697 [error] make timer: <0.183.0> T = 0 > 07:21:51.698 [error] cancel timer: <0.183.0> > 07:21:51.699 [error] make timer: <0.183.0> T = 0 > 07:21:51.699 [error] timeout <0.183.0> > 07:21:51.700 [error] make timer: <0.183.0> T = 0 > 07:21:51.701 [error] timeout <0.183.0> > 07:21:51.701 [error] timeout <0.183.0> > > Best Regards, Dmitry > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- http://www.linkedin.com/in/torbenhoffmann From steven.charles.davis@REDACTED Tue Aug 7 01:22:14 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 6 Aug 2012 16:22:14 -0700 (PDT) Subject: [erlang-questions] WX Failed loading "wxe_driver" In-Reply-To: <1344278188447-4655138.post@n4.nabble.com> References: <1344278188447-4655138.post@n4.nabble.com> Message-ID: Pretty sure this a crazy "Program Files" path thing. Windows still isn't too smart. Your easiest bet is to reinstall erlang in a path without spaces in the name. When I was on windows I would choose to install to e.g.: "C:\Erlang-R15B02" ...rather than into "C:\Program Files\...". I ran multiple versions like this and I strongly suspect that the space in the path is the issue for the failed driver load (yes, even though it says "PROGRA~1"). HTH /s On Monday, August 6, 2012 1:36:28 PM UTC-5, micern wrote: > > Hello! > > I'm trying to learn Erlang, so I apologize if this is a trivial problem I > should have solved myself. > > I'm failing when using WX, trying to open a window for my application. > > Yep, Windows 7 seams to be an odd choice of OS, but that is what I've got. > > Doing this in the monitor fails: > -------------------------------- > 1> Wx = wx:new(). > > =ERROR REPORT==== 6-Aug-2012::19:42:42 === > WX Failed loading "wxe_driver"@"c:/PROGRA~1/ERL59~1.1/lib/wx-0.99.2/priv" > ** exception error: {load_driver,"Det g?r inte att hitta den angivna > modulen."} > in function wxe_server:start/0 (wxe_server.erl, line 64) > in call from wx:new/1 (wx.erl, line 107) > > 2> Wx. > * 1: variable 'Wx' is unbound > > > The corresponding code in an .erl file fails too: > ----------------------------------------------- > -module(minimal). > > -include_lib("wx/include/wx.hrl"). > > -export([start/0]). > -compile(export_all). > > start() -> > Wx = wx:new(), > Frame = wx:batch(fun() -> create_window(Wx) end), > wxWindow:show(Frame), > loop(Frame), > wx:destroy(), > ok. > > (more code, loop, create_window, etc). > > ... > > 7> c(minimal). > {ok,minimal} > 8> minimal:start(). > > =ERROR REPORT==== 6-Aug-2012::19:49:23 === > WX Failed loading "wxe_driver"@"c:/PROGRA~1/ERL59~1.1/lib/wx-0.99.2/priv" > ** exception error: {load_driver,"Det g?r inte att hitta den angivna > modulen."} > in function wxe_server:start/0 (wxe_server.erl, line 64) > in call from wx:new/1 (wx.erl, line 107) > in call from minimal:start/0 (minimal.erl, line 16) > > yep, it bombs @ Wx = wx:new(), ... > > > Starting the debugger get a similar warning, even if the window actually > opens. > ------------------------------------------------ > 2> debugger:start(). > > =ERROR REPORT==== 6-Aug-2012::19:47:08 === > WX Failed loading "wxe_driver"@"c:/PROGRA~1/ERL59~1.1/lib/wx-0.99.2/priv" > {ok,<0.40.0>} > > > Digging in Erlang library files on my computer... > ------------------------------------------------- > Directory .../erl5.9.1/lib/wx-0.99.2 exists. > wxe_driver@REDACTED:/PROGRA~1/ERL59~1.1/lib/wx-0.99.2/priv exists. > > > Swedish... > ---------- > "Det g?r inte att hitta den angivna modulen" = "Not possible to find the > module" > > > WHY?!?!?! AND WHAT TO DO?!?!?! > > /micael > > -- > View this message in context: > http://erlang.2086793.n4.nabble.com/WX-Failed-loading-wxe-driver-tp4655138.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From moxford@REDACTED Tue Aug 7 02:04:36 2012 From: moxford@REDACTED (Mike Oxford) Date: Mon, 6 Aug 2012 17:04:36 -0700 Subject: [erlang-questions] Mnesia ETS crash Message-ID: =ERROR REPORT==== 18-Jun-2012::15:01:49 === Error in process <0.1743.0> on node '' with exit value: {badarg,[{ets,safe_fixtable,[px_channel_log,false]},{mnesia_lib,db_fixtable,3},{mnesia_loader,cleanup_tab_copier,3},{mnesia_loader,send_table,3},{mnesia_controller,send_and_re ply,2}]} Logging shows it was then restarted and {inconsistent_database, starting_partitioned_network... mnesia is Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:24:24] [rq:24] [async-threads:64] [hipe] [kernel-poll:true] so it shouldn't fall victim to the safe_fixtable/next issues from R13x This has happened to us twice now, both times exactly the same error. System runs other ram_only nodes as well with no issues; this one is a disc_ table with light load. Thoughts? -mox -------------- next part -------------- An HTML attachment was scrubbed... URL: From rory@REDACTED Tue Aug 7 02:50:03 2012 From: rory@REDACTED (Rory Byrne) Date: Tue, 7 Aug 2012 02:50:03 +0200 Subject: [erlang-questions] Adding StartTLS support to eldap (for use Ejabberd) In-Reply-To: References: Message-ID: <20120807005003.GD13487@almeida.jinsky.com> On Mon, Aug 06, 2012 at 09:24:53PM +0100, Gavin Henry wrote: > > > I've changed the subject of this email in the hope that someone > > from the ejabberd community might offer some advice. > > Ha, naughty! I've created a new thread as it was going off a bit. Good thinking! > Yes, unfortunately most people that develop LDAP libs forget StartTLS and > just go for LDAP over SSL which isn't a standard and is deprecated. StartTLS > is a standard - http://www.rfc-editor.org/rfc/rfc4513.txt Well, adding support for LDAP over SSL is pretty simple and we all know how lazy programmers are! :-) Both methods have their merits and they're both going to be around a long time, so supporting both makes lots of sense. > Then again, the whole point of TLS and *getting it right* is to > validate certificates > which a lot of libs don't offer or admins don't bother with. Actually, it looks like eldap is currently hard-coded not to verify the server's certificate when using LDAP over SSL, so this would need to be updated as well. > RFC 4513 is what needs to be read for this work and it's sponsorship, which > also covers the SASL side too. And probably parts of RFC 4511 too. > > However, they might accept a patch that uses the OTP version of > > eldap when it's available. Ultimately it would be more beneficial > > for everyone if the STARTTLS and SASL functionality were added to > > the OTP version. Certainly if I was writing a patch, I'd much > > rather it was for the OTP version. > > My company would not sponsor the Ejabberd version of eldap. What would > be the point, > that helps no one and would be a shortsighted piece of sponsorship on > our part :-) Actually, the more I look at the ejabberd eldap version, the more I get the feeling that there is no chance that they will opt to use the OTP version as it stands. One of the first things they did with the eldap code was to rewrite it be asynchronous so it can have multiple ongoing requests to the LDAP server at the same time. The OTP eldap, on the other hand, is synchronous - it blocks on each request to the server waiting for a response. Clearly in the context of a large multi-user XMPP server, having asynchronous communication with your directory server is something you'd want. I suspect there's a few other useful changes that they've made that they (or you) wouldn't be willing to give up. > If the patch is done right then I'm sure it wouldn't be too difficult > to add initial > support to the ejabberd eldap version at the same time, with a caveat that > no further support will be provided and the ejabberd team would need > to either bring > in OTP eldap or maintain it themselves. I wish this were the case. Unfortunately the two sets of code are vastly different - at least to my eyes. I can't see how a patch for OTP eldap would be of any use against the ejabberd version. Basically, you should consider them two unrelated projects, at a code level that's what they are. That said, if you wrote a patch for one of them, you'd have a good idea of what you're doing when writing a patch for the other one. > >> In our situation we're not using cliebt certificate based TLS with > >> SASL EXTERNAL. But it should be added at the same time rather than an > >> incomplete patch. > > > > Yeah, I reckon it's worth having. > > It would certainly add excellent support for enterprises using eldap. > Some of our OpenLDAP > support customers will *only* use SASL EXTERNAL with X.509 certificate based > auth. Yeah, I can well imagine. If you're going to put the money and effort into a PKI, you'd be mad not to use it! Rory From moxford@REDACTED Tue Aug 7 03:30:54 2012 From: moxford@REDACTED (Mike Oxford) Date: Mon, 6 Aug 2012 18:30:54 -0700 Subject: [erlang-questions] Mnesia node_down Message-ID: How long is it between "network severed" and "node_down logged"? Does it use the net tick, or something else like socket status? Thanks! -mox -------------- next part -------------- An HTML attachment was scrubbed... URL: From bgustavsson@REDACTED Tue Aug 7 08:24:05 2012 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 7 Aug 2012 08:24:05 +0200 Subject: [erlang-questions] Warning: NOT OPTIMIZED: sub binary is used or returned In-Reply-To: <84C83563-0075-4308-8B24-09327F1CF24D@gmail.com> References: <84C83563-0075-4308-8B24-09327F1CF24D@gmail.com> Message-ID: On Mon, Aug 6, 2012 at 6:36 PM, Zabrane Mickael wrote: > Hi guys, > > I'm communicating with linked-in driver using the following code: > > control(Port, Command, Data) -> > case port_control(Port, Command, Data) of > <<0, Result/binary>> -> Result; % <- LINE 468 handle a good case > <<1, Error/binary>> -> {error, binary_to_term(Error)} % <- LINE 469 handle error case > end. > > > When compiling, I got this 02 warnings: > > $ make > Recompile: src/pimco.erl > src/pimco.erl:468: Warning: NOT OPTIMIZED: sub binary is used or returned > src/pimco.erl469: Warning: NOT OPTIMIZED: sub binary used by erlang:binary_to_term/1 > make[2]: Nothing to be done for `all'. > > Is there a way to get rid of these warnings and avoid a full binary copies? Yes, you can get rid of the warnings by *not* using the bin_opt_info compiler option. No, there is nothing to avoid here. The warning says that a sub binary is created. A sub binary is small term that references a part of a binary. Sub binaries is usually an efficient way to reference just a part of a binary. So there is *no* copying of the binary data going on in your example. The only time you have to worry about sub binaries being created (and the reason for the optimizations and compiler warnings) is when sub binaries are being created repeatedly in a loop, for example in: b2l(<>) -> [H|b2l(T)]; b2l(<<>>) -> []. In this case, the compiler will optimize the generated code to avoid creating a sub binary. Instead of creating a new sub binary in every iteration of the loop, a match context will be created once and kept through the entire loop. That will be beneficial in two ways: 1) Less garbage will be produced (by not building a sub binary that will become garbage almost immediately). 2) Matching will be faster when the match context is already available. To summarize, don't worry about sub binaries being created unless it happens repeatedly in a tight loop. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From dangud@REDACTED Tue Aug 7 08:43:35 2012 From: dangud@REDACTED (Dan Gudmundsson) Date: Tue, 7 Aug 2012 08:43:35 +0200 Subject: [erlang-questions] Mnesia node_down In-Reply-To: References: Message-ID: mnesia reports node_down when a link a breaks between the mnesia_monitor processes. Thus it's when erlang distribution discovers that the node is unreachable. /Dan On Tue, Aug 7, 2012 at 3:30 AM, Mike Oxford wrote: > How long is it between "network severed" and "node_down logged"? > Does it use the net tick, or something else like socket status? > > Thanks! > > -mox > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From eriksoe@REDACTED Tue Aug 7 09:16:17 2012 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Tue, 7 Aug 2012 09:16:17 +0200 Subject: [erlang-questions] Partial disk based and partial memory based terms In-Reply-To: References: Message-ID: Sounds like what you need is a queueing system. Have you considered using e.g. RabbitMQ? It provides persistent queues. Den 06/08/2012 11.30 skrev "Maruthavanan Subbarayan" < maruthavanan_s@REDACTED>: > Hi, > > I am building an application that would receive requests on bulk and > process at back-end. I wrote an code where requested would be inserted > into mnesia and the would be processed later.But ended up in a situation > where started receiving enormous requests and processing of the requests > took more time. So gradually, I faced couple of times where erlang got > crashed due to insufficient memory allocation on RAM. There were around > 300K records while it got crashed. > > Is there any workaround where I can keep only 10-20% of the total records > in RAM and remaining in disk at any point of time. I have Gigs of disk > space and only Megs of RAM. I am looking this as an alternative because It > would also give memory for other process in the machine without occupying > it unnecessarily. > > Thanks in advance, > Marutha > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gustav.simonsson@REDACTED Tue Aug 7 11:14:53 2012 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Tue, 07 Aug 2012 10:14:53 +0100 (BST) Subject: [erlang-questions] beam.smp is killed by oom In-Reply-To: Message-ID: <133ebc58-9dd2-4684-9449-2234f314250e@knuth> If you disable the Linux oom-killer and allow the BEAM to consume memory until it crashes when a memory allocator cannot allocate more memory, you should get a Erlang dump [1] which will contain process information such as heap size and message queue size for your Erlang processes. You can then (hopefully) find certain processes which had large message queues and/or had large heaps. Another way, if you are doing load testing where you can control the number of connections, is to gradually increase the number of connections while looking at message queue length, heap size and other process properties using the observer app [2]. 1. http://www.erlang.org/doc/apps/erts/crash_dump.html 2. http://www.erlang.org/doc/apps/observer/observer_ug.html // Gustav Sent from my PC ----- Original Message ----- > From: "ori brost" > To: erlang-questions@REDACTED > Sent: Sunday, 5 August, 2012 7:48:12 PM > Subject: Re: [erlang-questions] beam.smp is killed by oom > > > > We suspect this may be because of os_mon (as it prints some messages > before oom-killer kills beam.smp) > > > On Sun, Aug 5, 2012 at 8:40 PM, ori brost < oribrost@REDACTED > > wrote: > > > > We are load testing a server we implemented in Erlang. Memory usage > for beam.smp stays doesn't go above 30% memory until we get to > roughly 93000 connections. During what appears to be roughly 5 > seconds, beam.smp is killed by Linux's oom-killer (I'm assuming that > during these 5 seconds, mem usage spikes up). Any idea how we can > trace this problem? > > > Some traces from our system: > > > This is what beam.smp prints shortly before it is oom-killed: > http://pastebin.com/s4sJqXwY > > > This is dmesg output: > http://pastebin.com/wFUw84yj > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From max.lapshin@REDACTED Tue Aug 7 11:48:28 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 7 Aug 2012 13:48:28 +0400 Subject: [erlang-questions] beam.smp is killed by oom In-Reply-To: <133ebc58-9dd2-4684-9449-2234f314250e@knuth> References: <133ebc58-9dd2-4684-9449-2234f314250e@knuth> Message-ID: In 99,99% cases when I get erlang crashed by oom, it is error logger of failing process. When you have a big gen_server state (more than 10 MB) and your server is crashing due to error message, last message and last reason are going to be dumped. gen_server:format_status doesn't help at all, because reason is going to be dumped. 10 MB of state takes gigabyte or more of memory to dump as an error reason. From zabrane3@REDACTED Tue Aug 7 14:03:31 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 7 Aug 2012 14:03:31 +0200 Subject: [erlang-questions] Warning: NOT OPTIMIZED: sub binary is used or returned In-Reply-To: References: <84C83563-0075-4308-8B24-09327F1CF24D@gmail.com> Message-ID: <9F1661F5-A910-4346-B308-E87C368B06B6@gmail.com> Hi Bj?rn, On Aug 7, 2012, at 8:24 AM, Bj?rn Gustavsson wrote: > On Mon, Aug 6, 2012 at 6:36 PM, Zabrane Mickael wrote: >> Hi guys, >> >> I'm communicating with linked-in driver using the following code: >> >> control(Port, Command, Data) -> >> case port_control(Port, Command, Data) of >> <<0, Result/binary>> -> Result; % <- LINE 468 handle a good case >> <<1, Error/binary>> -> {error, binary_to_term(Error)} % <- LINE 469 handle error case >> end. >> >> >> When compiling, I got this 02 warnings: >> >> $ make >> Recompile: src/pimco.erl >> src/pimco.erl:468: Warning: NOT OPTIMIZED: sub binary is used or returned >> src/pimco.erl469: Warning: NOT OPTIMIZED: sub binary used by erlang:binary_to_term/1 >> make[2]: Nothing to be done for `all'. >> >> Is there a way to get rid of these warnings and avoid a full binary copies? > > Yes, you can get rid of the warnings by *not* using the bin_opt_info compiler > option. ;-) > No, there is nothing to avoid here. The warning says that > a sub binary is created. A sub binary is small term that > references a part of a binary. Sub binaries is usually an > efficient way to reference just a part of a binary. > > So there is *no* copying of the binary data going on in > your example. Crystal clear. This version is funny because the compiler now throws only one WARNING instead of two: $ make src/pimco.erl:492: Warning: NOT OPTIMIZED: sub binary is used or returned control(Port, Command, Data) -> control(port_control(Port, Command, Data)). control(<<0, Result/binary>>) -> control(Result, true); control(<<1, Error/binary>>) -> control(Error, false). control(<>, false) -> % <--- LINE 492 {error, binary_to_term(Error)}; control(<>, true) -> Result. Anyway, thanks for your explanations Bj?rn. Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From moxford@REDACTED Tue Aug 7 20:25:30 2012 From: moxford@REDACTED (Mike Oxford) Date: Tue, 7 Aug 2012 11:25:30 -0700 Subject: [erlang-questions] Mnesia ETS crash In-Reply-To: References: Message-ID: Further information: 1) The table is question does exist, has existed, and has been running for a while. 2) safe_fixtable fails with badarg if the table does not exist. #1 runs for a good long while (month?) and then suddenly #2 pops up. Why does mnesia 'lose' the table? Here's more context ===== ALIVE Tue Jul 31 17:22:07 PDT 2012 ===== ALIVE Tue Jul 31 17:37:07 PDT 2012 ===== ALIVE Tue Jul 31 17:52:07 PDT 2012 ===== Tue Jul 31 18:04:17 PDT 2012 =ERROR REPORT==== 31-Jul-2012::18:04:17 === Error in process <0.15759.10> on node '' with exit value: {badarg, [{ets,safe_fixtable,[px_channel_log,false]},{mnesia_lib,db_fixtable,3}, {mnesia_loader,cleanup_tab_copier,3},{mnesia_loader,send_table,3},{mnesia_controller,send_and_reply,2}]} ===== ===== LOGGING STARTED Tue Jul 31 18:04:19 PDT 2012 ===== Exec: /usr/local/chat-game-gwm/current/erts-5.8.4/bin/erlexec -boot /usr/local/chat-game-gwm/current/releases/1/chat -mode embedded -config /usr/local/chat-game-gwm/current/etc/app.config -args_file /usr/local/chat-game-gwm/current/etc/vm.args -- console Root: /usr/local/chat-game-gwm/current Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:24:24] [rq:24] [async-threads:64] [hipe] [kernel-poll:true] =ERROR REPORT==== 31-Jul-2012::18:04:22 === Mnesia('gwm_game@'): ** ERROR ** mnesia_event got {inconsistent_database, starting_partitioned_network, 'gwm_game@'} DEBUG: mnesia_helpers:118 - "{ \"Connecting to cluster: \" , NodeList }" On Mon, Aug 6, 2012 at 5:04 PM, Mike Oxford wrote: > =ERROR REPORT==== 18-Jun-2012::15:01:49 === > Error in process <0.1743.0> on node '' with exit value: > {badarg,[{ets,safe_fixtable,[px_channel_log,false]},{mnesia_lib,db_fixtable,3},{mnesia_loader,cleanup_tab_copier,3},{mnesia_loader,send_table,3},{mnesia_controller,send_and_re > ply,2}]} > > Logging shows it was then restarted and {inconsistent_database, > starting_partitioned_network... > > mnesia is Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:24:24] [rq:24] > [async-threads:64] [hipe] [kernel-poll:true] so it shouldn't fall victim to > the safe_fixtable/next issues from R13x > > This has happened to us twice now, both times exactly the same error. > System runs other ram_only nodes as well with no issues; this one is a > disc_ table with light load. > > Thoughts? > > -mox > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ghenry@REDACTED Tue Aug 7 21:27:54 2012 From: ghenry@REDACTED (Gavin Henry) Date: Tue, 7 Aug 2012 20:27:54 +0100 Subject: [erlang-questions] Adding StartTLS support to eldap (for use Ejabberd) In-Reply-To: <20120807005003.GD13487@almeida.jinsky.com> References: <20120807005003.GD13487@almeida.jinsky.com> Message-ID: > On Mon, Aug 06, 2012 at 09:24:53PM +0100, Gavin Henry wrote: >> >> > I've changed the subject of this email in the hope that someone >> > from the ejabberd community might offer some advice. >> >> Ha, naughty! I've created a new thread as it was going off a bit. > > Good thinking! > >> Yes, unfortunately most people that develop LDAP libs forget StartTLS and >> just go for LDAP over SSL which isn't a standard and is deprecated. StartTLS >> is a standard - http://www.rfc-editor.org/rfc/rfc4513.txt > > Well, adding support for LDAP over SSL is pretty simple and we all > know how lazy programmers are! :-) > > Both methods have their merits and they're both going to be around > a long time, so supporting both makes lots of sense. Yeah, true. But many systems don't enable port 686. >> Then again, the whole point of TLS and *getting it right* is to >> validate certificates >> which a lot of libs don't offer or admins don't bother with. > > Actually, it looks like eldap is currently hard-coded not to verify > the server's certificate when using LDAP over SSL, so this would > need to be updated as well. OK. >> RFC 4513 is what needs to be read for this work and it's sponsorship, which >> also covers the SASL side too. > > And probably parts of RFC 4511 too. Yep, but they should know this one :-) >> > However, they might accept a patch that uses the OTP version of >> > eldap when it's available. Ultimately it would be more beneficial >> > for everyone if the STARTTLS and SASL functionality were added to >> > the OTP version. Certainly if I was writing a patch, I'd much >> > rather it was for the OTP version. >> >> My company would not sponsor the Ejabberd version of eldap. What would >> be the point, >> that helps no one and would be a shortsighted piece of sponsorship on >> our part :-) > > Actually, the more I look at the ejabberd eldap version, the more I > get the feeling that there is no chance that they will opt to use > the OTP version as it stands. One of the first things they did with > the eldap code was to rewrite it be asynchronous so it can have multiple > ongoing requests to the LDAP server at the same time. The OTP eldap, > on the other hand, is synchronous - it blocks on each request to > the server waiting for a response. Clearly in the context of a large > multi-user XMPP server, having asynchronous communication with your > directory server is something you'd want. Yeah, that's kind of a deal breaker then. > I suspect there's a few other useful changes that they've made that > they (or you) wouldn't be willing to give up. True. >> If the patch is done right then I'm sure it wouldn't be too difficult >> to add initial >> support to the ejabberd eldap version at the same time, with a caveat that >> no further support will be provided and the ejabberd team would need >> to either bring >> in OTP eldap or maintain it themselves. > > I wish this were the case. Unfortunately the two sets of code are vastly > different - at least to my eyes. I can't see how a patch for OTP eldap > would be of any use against the ejabberd version. Basically, you should > consider them two unrelated projects, at a code level that's what they > are. That said, if you wrote a patch for one of them, you'd have a good > idea of what you're doing when writing a patch for the other one. I'm beginning to see this now. >> >> In our situation we're not using cliebt certificate based TLS with >> >> SASL EXTERNAL. But it should be added at the same time rather than an >> >> incomplete patch. >> > >> > Yeah, I reckon it's worth having. >> >> It would certainly add excellent support for enterprises using eldap. >> Some of our OpenLDAP >> support customers will *only* use SASL EXTERNAL with X.509 certificate based >> auth. > > Yeah, I can well imagine. If you're going to put the money and effort > into a PKI, you'd be mad not to use it! Of course :-) -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry@REDACTED Open Source. Open Solutions(tm). http://www.suretecsystems.com/ Suretec Systems is a limited company registered in Scotland. Registered number: SC258005. Registered office: 24 Cormack Park, Rothienorman, Inverurie, Aberdeenshire, AB51 8GL. Subject to disclaimer at http://www.suretecgroup.com/disclaimer.html Do you know we have our own VoIP provider called SureVoIP? See http://www.surevoip.co.uk Did you see our API? http://www.surevoip.co.uk/api From micael.erneborg@REDACTED Tue Aug 7 22:16:39 2012 From: micael.erneborg@REDACTED (micern) Date: Tue, 7 Aug 2012 13:16:39 -0700 (PDT) Subject: [erlang-questions] WX Failed loading "wxe_driver" In-Reply-To: <1344278188447-4655138.post@n4.nabble.com> References: <1344278188447-4655138.post@n4.nabble.com> Message-ID: <1344370599399-4655139.post@n4.nabble.com> Hi again! I was recommended to reinstall Erlang using a shorter path name, to avoid path names with a blank, as in 'Program Files'. I reinstalled Erlang to c:\Erlang and edited the Path variable to include c:\Erlang\bin and c:\Erlang\lib (I assume ...\lib should be there too). The wxw_driver do exist in c:/Erlang/lib/wx-0.99.2/priv. But it fails as before. See the error reports below. Any other ideas? /micael starting the debugger --------------------- 3> debugger:start(). =ERROR REPORT==== 7-Aug-2012::21:35:40 === WX Failed loading "wxe_driver"@"c:/Erlang/lib/wx-0.99.2/priv" {ok,<0.37.0>} Wx -- 4> Wx = wx:new(). =ERROR REPORT==== 7-Aug-2012::21:42:40 === WX Failed loading "wxe_driver"@"c:/Erlang/lib/wx-0.99.2/priv" ** exception error: {load_driver,"Det g?r inte att hitta den angivna modulen."} in function wxe_server:start/0 (wxe_server.erl, line 64) in call from wx:new/1 (wx.erl, line 107) -- View this message in context: http://erlang.2086793.n4.nabble.com/WX-Failed-loading-wxe-driver-tp4655138p4655139.html Sent from the Erlang Questions mailing list archive at Nabble.com. From erlang@REDACTED Tue Aug 7 22:41:20 2012 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 7 Aug 2012 22:41:20 +0200 Subject: [erlang-questions] announce: midi and coca native driver (start of) Message-ID: Hello I have make a start on a native (ie CoreMidi) driver for the mac and a native nibless interpreted driver for cocoa. The code is at: http://github.com/joearms/cocoa The midi driver has been tested using VMPK (a virtual keyboard) and with a korg nanoKEY2 (great fun). The coca driver pops up a native window with button and text fields and implements a simple dynamic method call. There is a small amount of documentation - enough to get started but not publishable quality. I am a complete beginner as regards objective-C and cocoa so I have no idea if my program works by accident or design. The method I use in the cocoa driver uses various rather nasty pointer casting but appears to work. Where do I want to go with this? - my goal is to write something like GarageBand so I intend to write interfaces to the audio units and so on also to provide tools for music notation and composition. All feedback is welcome. /Joe From Jrosenblum@REDACTED Tue Aug 7 22:52:43 2012 From: Jrosenblum@REDACTED (Jr0) Date: Tue, 7 Aug 2012 13:52:43 -0700 (PDT) Subject: [erlang-questions] Looking for RDBMS driver with Transaction Support. In-Reply-To: References: Message-ID: <1344372763331-4655140.post@n4.nabble.com> I know that the Odbc library has a commit/2 and commit/3, but I'm not sure if I am interpreting its use correctly. If I create a session with the {auto_commit, off} attribute and then do a series of separate sql_query/2 SELECT statements (not batched statements, but separate invocations of sql_query/2), followed by a committ/2, are the selects all done within a single transaction guaranteeing ACID properties? I am using Oracle, by the way -- View this message in context: http://erlang.2086793.n4.nabble.com/Looking-for-RDBMS-driver-with-Transaction-Support-tp3490646p4655140.html Sent from the Erlang Questions mailing list archive at Nabble.com. From nx@REDACTED Tue Aug 7 22:55:39 2012 From: nx@REDACTED (nx) Date: Tue, 7 Aug 2012 16:55:39 -0400 Subject: [erlang-questions] Looking for RDBMS driver with Transaction Support. In-Reply-To: References: Message-ID: erlang-mysql-driver has transactions support: https://github.com/dizzyd/erlang-mysql-driver On Mon, May 2, 2011 at 2:24 PM, Alex Arnon wrote: > Hi All, > > Is there an RDBMS driver in the Erlang universe whose API supports > transactions? > The docs for odbc, epgsql and emysql do not specify any kind of transaction > support. > > Thanks in advance! > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From puzza007@REDACTED Tue Aug 7 23:06:19 2012 From: puzza007@REDACTED (Paul Oliver) Date: Tue, 7 Aug 2012 17:06:19 -0400 Subject: [erlang-questions] Looking for RDBMS driver with Transaction Support. In-Reply-To: References: Message-ID: Related: https://github.com/smarkets/epgsql_connpool On Thu, May 5, 2011 at 3:40 AM, Alex Arnon wrote: > Oh, excellent! Thank you very much! > > > > On Tue, May 3, 2011 at 3:39 PM, mabrek wrote: > >> On Mon, May 2, 2011 at 10:24 PM, Alex Arnon wrote: >> > Hi All, >> > >> > Is there an RDBMS driver in the Erlang universe whose API supports >> > transactions? >> > The docs for odbc, epgsql and emysql do not specify any kind of >> transaction >> > support. >> >> epgsql does have simple transaction support, take a look at function >> with_transaction/2 at >> https://github.com/wg/epgsql/blob/master/src/pgsql.erl >> >> Regards, >> Anton Lebedevich >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave@REDACTED Tue Aug 7 23:08:30 2012 From: dave@REDACTED (Dave Cottlehuber) Date: Tue, 7 Aug 2012 23:08:30 +0200 Subject: [erlang-questions] WX Failed loading "wxe_driver" In-Reply-To: References: <1344278188447-4655138.post@n4.nabble.com> <1344370599399-4655139.post@n4.nabble.com> Message-ID: On 7 August 2012 22:16, micern wrote: > > > Hi again! > > I was recommended to reinstall Erlang using a shorter path name, to avoid > path names with a blank, as in 'Program Files'. > > I reinstalled Erlang to c:\Erlang and edited the Path variable to include > c:\Erlang\bin and > c:\Erlang\lib (I assume ...\lib should be there too). > > The wxw_driver do exist in c:/Erlang/lib/wx-0.99.2/priv. > > But it fails as before. See the error reports below. > > Any other ideas? > > /micael Could be you have 8.3 filenames disabled for 'performance reasons'. It fits the initial case, but I'm not sure it would explain the 2nd c:\Erlang\ scenario. Does "pushd c:\PROGRA~1\ERL59~1.1\lib\wx-0.99.2\priv" work for you? The other option is that you're missing some MSVC runtime dependency for the DLL. Check with depends.exe (you'll need to google/download it) but its self explanatory really. IIRC this error can also occur. My final next step would be to run procexp from sysinternals.com and get a list of all activities referring to wx in filepath. This will likely clarify where erlang is searching and failing to find the DLL. A+ Dave From dmkolesnikov@REDACTED Wed Aug 8 06:08:33 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Wed, 8 Aug 2012 07:08:33 +0300 Subject: [erlang-questions] WX Failed loading "wxe_driver" In-Reply-To: <1344370599399-4655139.post@n4.nabble.com> References: <1344278188447-4655138.post@n4.nabble.com> <1344370599399-4655139.post@n4.nabble.com> Message-ID: <-8233372969433830276@unknownmsgid> Most likely you have an issue with dependencies. Could you check what libraries are imported by driver and validate that system has corresponding ones? Best Regards, Dmitry >-|-|-*> On 7.8.2012, at 23.16, micern wrote: > > > Hi again! > > I was recommended to reinstall Erlang using a shorter path name, to avoid > path names with a blank, as in 'Program Files'. > > I reinstalled Erlang to c:\Erlang and edited the Path variable to include > c:\Erlang\bin and > c:\Erlang\lib (I assume ...\lib should be there too). > > The wxw_driver do exist in c:/Erlang/lib/wx-0.99.2/priv. > > But it fails as before. See the error reports below. > > Any other ideas? > > /micael > > > starting the debugger > --------------------- > > 3> debugger:start(). > > =ERROR REPORT==== 7-Aug-2012::21:35:40 === > WX Failed loading "wxe_driver"@"c:/Erlang/lib/wx-0.99.2/priv" > {ok,<0.37.0>} > > > Wx > -- > 4> Wx = wx:new(). > > =ERROR REPORT==== 7-Aug-2012::21:42:40 === > WX Failed loading "wxe_driver"@"c:/Erlang/lib/wx-0.99.2/priv" > ** exception error: {load_driver,"Det g?r inte att hitta den angivna > modulen."} > in function wxe_server:start/0 (wxe_server.erl, line 64) > in call from wx:new/1 (wx.erl, line 107) > > > > > -- > View this message in context: http://erlang.2086793.n4.nabble.com/WX-Failed-loading-wxe-driver-tp4655138p4655139.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Wed Aug 8 09:34:23 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 8 Aug 2012 09:34:23 +0200 Subject: [erlang-questions] Midi and core audio in Erlang Message-ID: Dear list, I've been playing with the mac midi/core audio frameworks and now feel reasonably happy that I understand enough to get started. So far there are (as far as I know) only two attempts to use Erlang for real-time midi control http://code.google.com/p/midi-erl/ and http://github.com/joearms/cocoa My current goal is to make something like impromptu (see http://impromptu.moso.com.au/) Impromptu is midi + graphics + scheme I'd like to make midi + graphics + erlang I searched this list for people who are interested in this, but many of the mails are old so I have no idea who is currently interested. Could anybody who is interested in this topic please send me a short mail Cheers /Joe From lukas@REDACTED Wed Aug 8 11:00:21 2012 From: lukas@REDACTED (Lukas Larsson) Date: Wed, 8 Aug 2012 11:00:21 +0200 Subject: [erlang-questions] prim_inet socket incompatible with file:sendfile/2/5 In-Reply-To: <3152847A-9DA4-45FC-A16A-4D016A9D059E@gmail.com> References: <3152847A-9DA4-45FC-A16A-4D016A9D059E@gmail.com> Message-ID: Hi! There are a couple of reasons why this might not work. Could you show me a small example of what you are trying to do? Lukas On Fri, Jul 27, 2012 at 6:57 PM, Zabrane Mickael wrote: > Hi, > > Is there any reason why a client socket created from prim_inet > do not work with the new file:sendfile fonction? > > I got {error, badarg}. > > Any undocumented function to solves this? > > Regards, > Zabrane > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From lukas@REDACTED Wed Aug 8 11:46:07 2012 From: lukas@REDACTED (Lukas Larsson) Date: Wed, 8 Aug 2012 11:46:07 +0200 Subject: [erlang-questions] Sending messages to a process from the port driver from a separate thread In-Reply-To: <06139A918ACCA041BF46A0F36940C7FA39D04C06@exch-mbx1.msk.trd.ru> References: <06139A918ACCA041BF46A0F36940C7FA39D04C06@exch-mbx1.msk.trd.ru> Message-ID: Hi! I'm no expert on the code in question, but after a brief look in io.c:driver_deliver_term it would seem that you can do it from not only scheduler threads, given that you are using the SMP emulator. Lukas On Wed, Aug 1, 2012 at 8:47 AM, Zhemzhitsky Sergey wrote: > Hi erlang gurus, > > > > Is it legal to send messages to any erlang process by means of > driver_send_term from a separate thread started by erl_drv_thread_create? > > I.e. can I send messages to the process not from the port driver callback, > but at any time from the function specified when invoking > erl_drv_thread_create? > > > > Best Regards, > > Sergey > > > > _______________________________________________________ > > > > The information contained in this message may be privileged and conf > idential and protected from disclosure. If you are not the original intended > recipient, you are hereby notified that any review, retransmission, > dissemination, or other use of, or taking of any action in reliance upon, > this information is prohibited. If you have received this communication in > error, please notify the sender immediately by replying to this message and > delete it from your computer. Thank you for your cooperation. Troika Dialog, > Russia. > > If you need assistance please contact our Contact Center (+7495) 258 0500 or > go to www.troika.ru/eng/Contacts/system.wbp > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mrtndimitrov@REDACTED Wed Aug 8 13:01:21 2012 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Wed, 08 Aug 2012 14:01:21 +0300 Subject: [erlang-questions] OtpErlangDouble/OtpErlangFloat Message-ID: <50224701.4020302@gmail.com> Hello, I have some strings in Java representing floating point numbers. I want to transfer them to Erlang. So I create an object of type OtpErlangFloat. The problem is that is rounds the number funnily. For example, String s = "22.4"; float f = Float.parseFloat(s); System.out.println(f); // 22.4 OtpErlangFloat ef = new OtpErlangFloat(f); System.out.println(ef.toString()); // 22.399999618530273 Then in Erlang: mochinum:digits(Ef). "22.399999618530273" but mochinum:digits(22.4). "22.4" Is there a way to keep the precision when constructing OtpErlangDouble/OtpErlangFloat? Thanks in advance, Martin From essen@REDACTED Wed Aug 8 14:00:06 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 08 Aug 2012 14:00:06 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.6.1 Message-ID: <502254C6.1040407@ninenines.eu> Hello, Tagged a new version before starting to break API compatibility for the upcoming big release. This version contains a few fixes that do not break compatibility. I do not plan to make another release before 0.8. Changes are: * Add hello_world, rest_hello_world, chunked_hello_world, echo_get, echo_post and static examples. * Add support for the "Expect: 100-continue" header. * Keep the original 'Host' header value instead of modifying it. * Fix use of parsed headers cache. * REST: fix the matching of charsets. * REST: allow <<"type/subtype">> format for content_types_accepted. * Improve typespecs. I urge everyone to get out of master for a little while because the switch to Ranch will break your project until you make the necessary changes. Thanks for your understanding. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From zabrane3@REDACTED Wed Aug 8 16:10:44 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 8 Aug 2012 16:10:44 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.6.1 In-Reply-To: <502254C6.1040407@ninenines.eu> References: <502254C6.1040407@ninenines.eu> Message-ID: <3F3CC0C2-B801-48D9-BA35-B2D0693E89EE@gmail.com> Hi Lo?c, Congrats for this new release. Did it include the parsing of HTTP headers without the decode_packet call? Regards, Zabrane On Aug 8, 2012, at 2:00 PM, Lo?c Hoguin wrote: > Hello, > > Tagged a new version before starting to break API compatibility for the upcoming big release. This version contains a few fixes that do not break compatibility. I do not plan to make another release before 0.8. > > Changes are: > > * Add hello_world, rest_hello_world, chunked_hello_world, > echo_get, echo_post and static examples. > > * Add support for the "Expect: 100-continue" header. > > * Keep the original 'Host' header value instead of modifying it. > > * Fix use of parsed headers cache. > > * REST: fix the matching of charsets. > > * REST: allow <<"type/subtype">> format for content_types_accepted. > > * Improve typespecs. > > I urge everyone to get out of master for a little while because the switch to Ranch will break your project until you make the necessary changes. > > Thanks for your understanding. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Wed Aug 8 16:14:50 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 08 Aug 2012 16:14:50 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.6.1 In-Reply-To: <3F3CC0C2-B801-48D9-BA35-B2D0693E89EE@gmail.com> References: <502254C6.1040407@ninenines.eu> <3F3CC0C2-B801-48D9-BA35-B2D0693E89EE@gmail.com> Message-ID: <5022745A.2080302@ninenines.eu> Nope that one comes after 0.8. It breaks API compatibility because the point is most importantly to remove the inconsistencies of decode_packet like the header name being either atom() or binary() and such. On 08/08/2012 04:10 PM, Zabrane Mickael wrote: > Hi Lo?c, > > Congrats for this new release. > Did it include the parsing of HTTP headers without the decode_packet call? > > Regards, > Zabrane > > On Aug 8, 2012, at 2:00 PM, Lo?c Hoguin wrote: > >> Hello, >> >> Tagged a new version before starting to break API compatibility for the upcoming big release. This version contains a few fixes that do not break compatibility. I do not plan to make another release before 0.8. >> >> Changes are: >> >> * Add hello_world, rest_hello_world, chunked_hello_world, >> echo_get, echo_post and static examples. >> >> * Add support for the "Expect: 100-continue" header. >> >> * Keep the original 'Host' header value instead of modifying it. >> >> * Fix use of parsed headers cache. >> >> * REST: fix the matching of charsets. >> >> * REST: allow <<"type/subtype">> format for content_types_accepted. >> >> * Improve typespecs. >> >> I urge everyone to get out of master for a little while because the switch to Ranch will break your project until you make the necessary changes. >> >> Thanks for your understanding. >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From max.lapshin@REDACTED Wed Aug 8 16:52:04 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 8 Aug 2012 18:52:04 +0400 Subject: [erlang-questions] [ANN] Cowboy 0.6.1 In-Reply-To: <5022745A.2080302@ninenines.eu> References: <502254C6.1040407@ninenines.eu> <3F3CC0C2-B801-48D9-BA35-B2D0693E89EE@gmail.com> <5022745A.2080302@ninenines.eu> Message-ID: What is better: rewrite parsing in erlang, either add translation of atoms to binaries? Isn't parsing via decode_packet very fast? On Wed, Aug 8, 2012 at 6:14 PM, Lo?c Hoguin wrote: > Nope that one comes after 0.8. It breaks API compatibility because the point > is most importantly to remove the inconsistencies of decode_packet like the > header name being either atom() or binary() and such. > > > On 08/08/2012 04:10 PM, Zabrane Mickael wrote: >> >> Hi Lo?c, >> >> Congrats for this new release. >> Did it include the parsing of HTTP headers without the decode_packet call? >> >> Regards, >> Zabrane >> >> On Aug 8, 2012, at 2:00 PM, Lo?c Hoguin wrote: >> >>> Hello, >>> >>> Tagged a new version before starting to break API compatibility for the >>> upcoming big release. This version contains a few fixes that do not break >>> compatibility. I do not plan to make another release before 0.8. >>> >>> Changes are: >>> >>> * Add hello_world, rest_hello_world, chunked_hello_world, >>> echo_get, echo_post and static examples. >>> >>> * Add support for the "Expect: 100-continue" header. >>> >>> * Keep the original 'Host' header value instead of modifying it. >>> >>> * Fix use of parsed headers cache. >>> >>> * REST: fix the matching of charsets. >>> >>> * REST: allow <<"type/subtype">> format for content_types_accepted. >>> >>> * Improve typespecs. >>> >>> I urge everyone to get out of master for a little while because the >>> switch to Ranch will break your project until you make the necessary >>> changes. >>> >>> Thanks for your understanding. >>> >>> -- >>> Lo?c Hoguin >>> Erlang Cowboy >>> Nine Nines >>> http://ninenines.eu >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Wed Aug 8 17:39:32 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 08 Aug 2012 17:39:32 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.6.1 In-Reply-To: References: <502254C6.1040407@ninenines.eu> <3F3CC0C2-B801-48D9-BA35-B2D0693E89EE@gmail.com> <5022745A.2080302@ninenines.eu> Message-ID: <50228834.9000601@ninenines.eu> decode_packet would be a lot faster if it wasn't doing most things it's actually doing that make it more problematic for me to handle requests. Problems include: * using both atoms and binaries for header names * for binary header names, changing the case but only for name length < 22 characters, and with a weird behavior if it contains "--" * returning tuples for everything (forces me to handle all the different errors at the same level, and makes it harder to read) * not allowing me to fail early (see previous) I can simplify the code a lot by not using it, all without sacrificing performance because I can still use binary BIFs instead. The only thing that requires a little Erlang code is parsing the request line. On 08/08/2012 04:52 PM, Max Lapshin wrote: > What is better: rewrite parsing in erlang, either add translation of > atoms to binaries? > Isn't parsing via decode_packet very fast? > > On Wed, Aug 8, 2012 at 6:14 PM, Lo?c Hoguin wrote: >> Nope that one comes after 0.8. It breaks API compatibility because the point >> is most importantly to remove the inconsistencies of decode_packet like the >> header name being either atom() or binary() and such. >> >> >> On 08/08/2012 04:10 PM, Zabrane Mickael wrote: >>> >>> Hi Lo?c, >>> >>> Congrats for this new release. >>> Did it include the parsing of HTTP headers without the decode_packet call? >>> >>> Regards, >>> Zabrane >>> >>> On Aug 8, 2012, at 2:00 PM, Lo?c Hoguin wrote: >>> >>>> Hello, >>>> >>>> Tagged a new version before starting to break API compatibility for the >>>> upcoming big release. This version contains a few fixes that do not break >>>> compatibility. I do not plan to make another release before 0.8. >>>> >>>> Changes are: >>>> >>>> * Add hello_world, rest_hello_world, chunked_hello_world, >>>> echo_get, echo_post and static examples. >>>> >>>> * Add support for the "Expect: 100-continue" header. >>>> >>>> * Keep the original 'Host' header value instead of modifying it. >>>> >>>> * Fix use of parsed headers cache. >>>> >>>> * REST: fix the matching of charsets. >>>> >>>> * REST: allow <<"type/subtype">> format for content_types_accepted. >>>> >>>> * Improve typespecs. >>>> >>>> I urge everyone to get out of master for a little while because the >>>> switch to Ranch will break your project until you make the necessary >>>> changes. >>>> >>>> Thanks for your understanding. >>>> >>>> -- >>>> Lo?c Hoguin >>>> Erlang Cowboy >>>> Nine Nines >>>> http://ninenines.eu >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From max.lapshin@REDACTED Wed Aug 8 17:44:56 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 8 Aug 2012 19:44:56 +0400 Subject: [erlang-questions] [ANN] Cowboy 0.6.1 In-Reply-To: <50228834.9000601@ninenines.eu> References: <502254C6.1040407@ninenines.eu> <3F3CC0C2-B801-48D9-BA35-B2D0693E89EE@gmail.com> <5022745A.2080302@ninenines.eu> <50228834.9000601@ninenines.eu> Message-ID: Ok, I see. On Wed, Aug 8, 2012 at 7:39 PM, Lo?c Hoguin wrote: > decode_packet would be a lot faster if it wasn't doing most things it's > actually doing that make it more problematic for me to handle requests. > Problems include: > > * using both atoms and binaries for header names > * for binary header names, changing the case but only for name length < 22 > characters, and with a weird behavior if it contains "--" > * returning tuples for everything (forces me to handle all the different > errors at the same level, and makes it harder to read) > * not allowing me to fail early (see previous) > > I can simplify the code a lot by not using it, all without sacrificing > performance because I can still use binary BIFs instead. The only thing that > requires a little Erlang code is parsing the request line. > > > On 08/08/2012 04:52 PM, Max Lapshin wrote: >> >> What is better: rewrite parsing in erlang, either add translation of >> atoms to binaries? >> Isn't parsing via decode_packet very fast? >> >> On Wed, Aug 8, 2012 at 6:14 PM, Lo?c Hoguin wrote: >>> >>> Nope that one comes after 0.8. It breaks API compatibility because the >>> point >>> is most importantly to remove the inconsistencies of decode_packet like >>> the >>> header name being either atom() or binary() and such. >>> >>> >>> On 08/08/2012 04:10 PM, Zabrane Mickael wrote: >>>> >>>> >>>> Hi Lo?c, >>>> >>>> Congrats for this new release. >>>> Did it include the parsing of HTTP headers without the decode_packet >>>> call? >>>> >>>> Regards, >>>> Zabrane >>>> >>>> On Aug 8, 2012, at 2:00 PM, Lo?c Hoguin wrote: >>>> >>>>> Hello, >>>>> >>>>> Tagged a new version before starting to break API compatibility for the >>>>> upcoming big release. This version contains a few fixes that do not >>>>> break >>>>> compatibility. I do not plan to make another release before 0.8. >>>>> >>>>> Changes are: >>>>> >>>>> * Add hello_world, rest_hello_world, chunked_hello_world, >>>>> echo_get, echo_post and static examples. >>>>> >>>>> * Add support for the "Expect: 100-continue" header. >>>>> >>>>> * Keep the original 'Host' header value instead of modifying it. >>>>> >>>>> * Fix use of parsed headers cache. >>>>> >>>>> * REST: fix the matching of charsets. >>>>> >>>>> * REST: allow <<"type/subtype">> format for content_types_accepted. >>>>> >>>>> * Improve typespecs. >>>>> >>>>> I urge everyone to get out of master for a little while because the >>>>> switch to Ranch will break your project until you make the necessary >>>>> changes. >>>>> >>>>> Thanks for your understanding. >>>>> >>>>> -- >>>>> Lo?c Hoguin >>>>> Erlang Cowboy >>>>> Nine Nines >>>>> http://ninenines.eu >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> >>>> >>>> >>> >>> >>> -- >>> Lo?c Hoguin >>> Erlang Cowboy >>> Nine Nines >>> http://ninenines.eu >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu From ok@REDACTED Thu Aug 9 00:41:36 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 9 Aug 2012 10:41:36 +1200 Subject: [erlang-questions] OtpErlangDouble/OtpErlangFloat In-Reply-To: <50224701.4020302@gmail.com> References: <50224701.4020302@gmail.com> Message-ID: <7530DF94-FC61-42C6-9E5B-EE2604F1630C@cs.otago.ac.nz> On 8/08/2012, at 11:01 PM, Martin Dimitrov wrote: > Hello, > > I have some strings in Java representing floating point numbers. I want > to transfer them to Erlang. So I create an object of type > OtpErlangFloat. The problem is that is rounds the number funnily. For > example, > > String s = "22.4"; > float f = Float.parseFloat(s); Here you asked for a SINGLE-PRECISION floating point number. I presume you did try Prelude> 22.4 - 22.399999618530273 3.8146972514141453e-7 and you did notice that the difference is about the size of single precision epsilon? Well that's why. You asked for a number with 24 bits of precision that was close to 22.4, and 22.399999618530273 was indeed the answer you got. > System.out.println(f); // 22.4 PrintStream.println(float) calls PrintStream.print(float) calls String.valueOf(float) calls Float.toString(float) whose documentation says (for this value of m) tha it will be the integer part, then a dot, then "as any, but only as many, more digits as are needed to uniquely distinguish the argument value from adjacent values of type 'float'". So the value that is printed is the _least_ precise decimal representation that cannot be mistaken for a different single-precision (24 bits of precision) float. If you convert the same value to a double in Java and print that in Java, you will get, well, try it: L% cat NotErlangsFault.java public class NotErlangsFault { public static void main(String[] args) { final String s = "22.4"; final float f = Float.parseFloat(s); final double d = (double)f; System.out.println(s + " converts to float " + f + " which as double is " + d); } } L% java NotErlangsFault 22.4 converts to float 22.4 which as double is 22.399999618530273 See? Nothing to do with Erlang at all. > OtpErlangFloat ef = new OtpErlangFloat(f); > System.out.println(ef.toString()); // 22.399999618530273 And this tells us that an OtpErlangFloat contains a Java double, which is *more* precise than the float you gave it. > > Then in Erlang: > > mochinum:digits(Ef). > "22.399999618530273" And this is the right answer. > > but > > mochinum:digits(22.4). > "22.4" > > Is there a way to keep the precision when constructing > OtpErlangDouble/OtpErlangFloat? No precision whatsoever was lost in that step. The precision was lost when you first converted a String to a float. The value in the float really *is* much better approximated by 22.399999618530273 than by 22.4. From rory@REDACTED Thu Aug 9 01:03:30 2012 From: rory@REDACTED (Rory Byrne) Date: Thu, 9 Aug 2012 01:03:30 +0200 Subject: [erlang-questions] Adding StartTLS support to eldap (for use Ejabberd) In-Reply-To: References: <20120807005003.GD13487@almeida.jinsky.com> Message-ID: <20120808230330.GG13487@almeida.jinsky.com> On Tue, Aug 07, 2012 at 08:27:54PM +0100, Gavin Henry wrote: > > >> RFC 4513 is what needs to be read for this work and it's sponsorship, which > >> also covers the SASL side too. > > > > And probably parts of RFC 4511 too. > > Yep, but they should know this one :-) Yeah, yeah, of course they do :-) > > Actually, the more I look at the ejabberd eldap version, the more I > > get the feeling that there is no chance that they will opt to use > > the OTP version as it stands. One of the first things they did with > > the eldap code was to rewrite it be asynchronous so it can have multiple > > ongoing requests to the LDAP server at the same time. The OTP eldap, > > on the other hand, is synchronous - it blocks on each request to > > the server waiting for a response. Clearly in the context of a large > > multi-user XMPP server, having asynchronous communication with your > > directory server is something you'd want. > > Yeah, that's kind of a deal breaker then. > > > I suspect there's a few other useful changes that they've made that > > they (or you) wouldn't be willing to give up. > > True. Adding some of these features to OTP eldap wouldn't be too difficult, but I'd guess that only a subset of these changes would get accepted. And at the end of the day, I can't see ejabberd giving up their current flexibility where they can make any changes to their LDAP code as and when they need to. Anyway, it was good of you guys to offer to sponsor this. Hopefully you'll get hooked on Erlang yourself and stick around a while. The OTP eldap module is nice clean code and you could have a lot of fun working on it - once you find your feet that is. I'm sure it'll be looking a lot more enterprise-ready when you get your mits on it! :-) Rory From overminddl1@REDACTED Thu Aug 9 03:40:00 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Wed, 8 Aug 2012 19:40:00 -0600 Subject: [erlang-questions] Adding StartTLS support to eldap (for use Ejabberd) In-Reply-To: <20120808230330.GG13487@almeida.jinsky.com> References: <20120807005003.GD13487@almeida.jinsky.com> <20120808230330.GG13487@almeida.jinsky.com> Message-ID: On Wed, Aug 8, 2012 at 5:03 PM, Rory Byrne wrote: > On Tue, Aug 07, 2012 at 08:27:54PM +0100, Gavin Henry wrote: >> >> >> RFC 4513 is what needs to be read for this work and it's sponsorship, which >> >> also covers the SASL side too. >> > >> > And probably parts of RFC 4511 too. >> >> Yep, but they should know this one :-) > > Yeah, yeah, of course they do :-) > >> > Actually, the more I look at the ejabberd eldap version, the more I >> > get the feeling that there is no chance that they will opt to use >> > the OTP version as it stands. One of the first things they did with >> > the eldap code was to rewrite it be asynchronous so it can have multiple >> > ongoing requests to the LDAP server at the same time. The OTP eldap, >> > on the other hand, is synchronous - it blocks on each request to >> > the server waiting for a response. Clearly in the context of a large >> > multi-user XMPP server, having asynchronous communication with your >> > directory server is something you'd want. >> >> Yeah, that's kind of a deal breaker then. >> >> > I suspect there's a few other useful changes that they've made that >> > they (or you) wouldn't be willing to give up. >> >> True. > > Adding some of these features to OTP eldap wouldn't be too difficult, > but I'd guess that only a subset of these changes would get accepted. > And at the end of the day, I can't see ejabberd giving up their current > flexibility where they can make any changes to their LDAP code as and > when they need to. > > Anyway, it was good of you guys to offer to sponsor this. Hopefully > you'll get hooked on Erlang yourself and stick around a while. The OTP > eldap module is nice clean code and you could have a lot of fun working > on it - once you find your feet that is. I'm sure it'll be looking a lot > more enterprise-ready when you get your mits on it! :-) I, personally, would certainly not mind StartTLS support and asynchronous and concurrent connections being added to the OTP version... *hint*. From cavaluo@REDACTED Thu Aug 9 05:07:48 2012 From: cavaluo@REDACTED (cavaluo) Date: Thu, 9 Aug 2012 11:07:48 +0800 Subject: [erlang-questions] problems with creating table named 'user' in ets Message-ID: Hi, when i create a new ets table named 'user',result returned 'ok',and also i can do any operaton on it,but i can't find in The Table Visualizer,why? (cavaluo@REDACTED)15> ets:new(user,[set,public,named_table]). user (cavaluo@REDACTED)18> ets:info(user). [{compressed,false}, {memory,316}, {owner,<0.37.0>}, {heir,none}, {name,user}, {size,0}, {node,'cavaluo@REDACTED'}, {named_table,true}, {type,set}, {keypos,1}, {protection,public}] (cavaluo@REDACTED)20> ets:insert(user,{test,1,2,3}). true (cavaluo@REDACTED)21> ets:lookup(user,test). [{test,1,2,3}] -------------- next part -------------- An HTML attachment was scrubbed... URL: From jai@REDACTED Thu Aug 9 05:23:24 2012 From: jai@REDACTED (Jai Gupta) Date: Thu, 9 Aug 2012 08:53:24 +0530 Subject: [erlang-questions] Sock.js and Cowboy - Example Message-ID: I am new to both Sock.js and Erlang/Cowboy. I am trying to run a example. I have already install Erlang on Mac. $ erl Erlang R15B01 (erts-5.9.1) [source] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.1 (abort with ^G) 1> I am trying to run a simple echo example from my app. ====rebar.config==== %% -*- erlang -*- {erl_opts, [debug_info]}. %, fail_on_warning]}. {deps, [ {sockjs, ".*", {git, "git://github.com/sockjs/sockjs-erlang.git", {branch, "master"}}} ] }. ================ Note that dependency is in following way. My app has dependency on Sockjs which has dependency on Cowboy which has dependency on Proper. $ ./rebar get-deps After above command I get 3 folders in my deps folder. cowboy, proper & sockjs $ ./rebar compile I am getting following error. $ ./deps/sockjs/examples/cowboy_echo.erl escript: exception error: undefined function sockjs_handler:init_state/4 in function cowboy_echo:main/1 (./deps/sockjs/examples/cowboy_echo.erl, line 17) in call from escript:run/2 (escript.erl, line 727) in call from escript:start/1 (escript.erl, line 277) in call from init:start_it/1 in call from init:start_em/1 I have also tried coping cowboy_echo.erl to ./src/ folder but it gives same error. Are there any pointers that can help me? Note: I can successfully run this example if I directly use sockjs-erlang rebar.config but I want to run it on my app which describes sockjs as dependency. Jai -------------- next part -------------- An HTML attachment was scrubbed... URL: From aleksandr.vin@REDACTED Thu Aug 9 09:45:02 2012 From: aleksandr.vin@REDACTED (Aleksandr Vinokurov) Date: Thu, 9 Aug 2012 11:45:02 +0400 Subject: [erlang-questions] Why io::format() type is not an iolist? Message-ID: Hello all I've found the subject and did not understand the root of such restriction: why io:format/2 can't understand iolist for a Format arg? WBR, -- ????????? ????????? +7 (921) 982-21-43 @aleksandrvin -------------- next part -------------- An HTML attachment was scrubbed... URL: From ghenry@REDACTED Thu Aug 9 12:23:20 2012 From: ghenry@REDACTED (Gavin Henry) Date: Thu, 9 Aug 2012 11:23:20 +0100 Subject: [erlang-questions] Adding StartTLS support to eldap (for use Ejabberd) In-Reply-To: <20120808230330.GG13487@almeida.jinsky.com> References: <20120807005003.GD13487@almeida.jinsky.com> <20120808230330.GG13487@almeida.jinsky.com> Message-ID: >> True. > > Adding some of these features to OTP eldap wouldn't be too difficult, > but I'd guess that only a subset of these changes would get accepted. > And at the end of the day, I can't see ejabberd giving up their current > flexibility where they can make any changes to their LDAP code as and > when they need to. > > Anyway, it was good of you guys to offer to sponsor this. Hopefully > you'll get hooked on Erlang yourself and stick around a while. The OTP > eldap module is nice clean code and you could have a lot of fun working > on it - once you find your feet that is. I'm sure it'll be looking a lot > more enterprise-ready when you get your mits on it! :-) OK, better get learning! -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry@REDACTED Open Source. Open Solutions(tm). http://www.suretecsystems.com/ Suretec Systems is a limited company registered in Scotland. Registered number: SC258005. Registered office: 24 Cormack Park, Rothienorman, Inverurie, Aberdeenshire, AB51 8GL. Subject to disclaimer at http://www.suretecgroup.com/disclaimer.html Do you know we have our own VoIP provider called SureVoIP? See http://www.surevoip.co.uk Did you see our API? http://www.surevoip.co.uk/api From ingela.andin@REDACTED Thu Aug 9 15:09:51 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 9 Aug 2012 15:09:51 +0200 Subject: [erlang-questions] SSL key / password problems In-Reply-To: <20120723202637.GA25591@bittwiddlers.com> References: <20120723202637.GA25591@bittwiddlers.com> Message-ID: Hi! 2012/7/23, Matthew Harrell : > > I searched but all the information I found was very dated and didn't seem > to help any. Most of these examples are just a few modifications on what > is found here > > http://www.erlang.org/doc/apps/ssl/using_ssl.html > > First, only TLS v1 is supported at the moment in R15B01, right? Not 1.1 or > 1.2? Correct, but support for TLS 1.1 and TLS 1.2 will be released in a near future. > > I have a key pair with protected with the password "password" called > client.crt and client.key. When I try to start up a client connection > using that pair I get > > ssl:start(). > {ok, Socket} = ssl:connect("localhost", > 9950, > [{certfile, "client.crt"}, > {keyfile, "client.key"}], > infinity). > ** exception error: no match of right hand side value {error,ekeyfile} > > which is fine because it can't open the private key. But when I try > > {ok, Socket} = ssl:connect("localhost", > 9950, > [{certfile, "example/client.crt"}, > {keyfile, "example/client.key"}, > {password, "password"}], > infinity). > ** exception error: no match of right hand side value {error,ekeyfile} > > I get the same error. What am I doing wrong? Isn't that the point of the > password option? When I try things with openssl like the following it > works fine > > openssl s_client -cert example/client.crt -key example/client.key \ > -CAfile example/ca.pem -pass pass:password -state -connect > 127.0.0.1:9950 The password option should be working fine. You have different paths in your examples could that be it? (I will look further into this to make sure that nothing is broken, seems we only have proper tests for this on public key level and not on ssl level) > Also if I try to load the CA files I get messages about them not being > decoded properly > > {ok, Socket} = ssl:connect("localhost", > 9950, > [{cacertfile, > "/etc/ssl/certs/ca-certificates.crt"}, > {certfile, "example/client.crt"}, > {keyfile, "example/client.key"}, > {password, "password"}], > infinity). > > =INFO REPORT==== 23-Jul-2012::15:36:56 === > SSL WARNING: Ignoring a CA cert as it could not be correctly decoded. > > I get the same message on my own ca.crt file with it's one key but thought > I > would try the system one to see whether it differed > > > Finally, on the server side if I do the following using server keys > (without > passwords) and the openssl client line above > > ssl:start(). > {ok, ListenSocket} = ssl:listen ( 9950, [{active, true}, > {reuseaddr, true}, > {keyfile, "example/server.key"}, > {certfile, > "example/server.crt"}, > {backlog, 30}] ). > {ok, Socket} = ssl:transport_accept ( ListenSocket ). > ssl:ssl_accept ( Socket ). > ssl:setopts ( Socket, [{active, true}] ). > > then an SSL connection seems to start up fine according to the messages on > the openssl side. If I change this to > > ssl:start(). > {ok, ListenSocket} = ssl:listen ( 9950, [{active, true}, > {reuseaddr, true}, > {verify, verify_peer}, > {depth, 2}, > {cacertfile, "example/ca.pem"}, > {keyfile, "example/server.key"}, > {certfile, > "example/server.crt"}, > {backlog, 30}] ). > {ok, Socket} = ssl:transport_accept ( ListenSocket ). > ssl:ssl_accept ( Socket ). > ssl:setopts ( Socket, [{active, true}] ). > > where example/ca.pem is the one CA certificate I get > > =INFO REPORT==== 23-Jul-2012::16:12:59 === > SSL WARNING: Ignoring a CA cert as it could not be correctly decoded. > > ** exception exit: {{{badmatch, > {error, > {asn1, > {'Type not compatible with table constraint', > {{component,'Type'}, > {value,{5,<<>>}}, > {unique_name_and_value,id,{1,3,14,3,2,29}}}}}}}, > [{public_key,pkix_decode_cert,2, > [{file,"public_key.erl"},{line,215}]}, > {ssl_certificate,trusted_cert_and_path,3, > [{file,"ssl_certificate.erl"},{line,58}]}, > {ssl_handshake,certify,7, > [{file,"ssl_handshake.erl"},{line,216}]}, > {ssl_connection,certify,2, > [{file,"ssl_connection.erl"},{line,514}]}, > {ssl_connection,next_state,4, > [{file,"ssl_connection.erl"},{line,1929}]}, > > {gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,494}]}, > {proc_lib,init_p_do_apply,3, > [{file,"proc_lib.erl"},{line,227}]}]}, > {gen_fsm,sync_send_all_state_event, > [<0.50.0>,start,infinity]}} > in function gen_fsm:sync_send_all_state_event/3 (gen_fsm.erl, line > 240) > in call from ssl_connection:sync_send_all_state_event/3 > (ssl_connection.erl, line 1195) > in call from ssl_connection:handshake/2 (ssl_connection.erl, line 167) > > What does that mean? A few old certificates that are part of atleast some linux-distributions breaks the ASN-1 specs for x509-certificates, which means that the erlang asn-1 application can not decode them and hence the erlang ssl-application can not use them, but if the CA-file has many certs all correctly encoded certs will be understood. Regards Ingela Erlang/OTP team Ericsson AB From mrtndimitrov@REDACTED Thu Aug 9 15:12:12 2012 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Thu, 09 Aug 2012 16:12:12 +0300 Subject: [erlang-questions] OtpErlangDouble/OtpErlangFloat In-Reply-To: <7530DF94-FC61-42C6-9E5B-EE2604F1630C@cs.otago.ac.nz> References: <50224701.4020302@gmail.com> <7530DF94-FC61-42C6-9E5B-EE2604F1630C@cs.otago.ac.nz> Message-ID: <5023B72C.6000607@gmail.com> Thanks for the explanation. I know it is not Erlang to be blamed. On 8/9/2012 1:41 AM, Richard O'Keefe wrote: > On 8/08/2012, at 11:01 PM, Martin Dimitrov wrote: > >> Hello, >> >> I have some strings in Java representing floating point numbers. I want >> to transfer them to Erlang. So I create an object of type >> OtpErlangFloat. The problem is that is rounds the number funnily. For >> example, >> >> String s = "22.4"; >> float f = Float.parseFloat(s); > Here you asked for a SINGLE-PRECISION floating point number. > I presume you did try > > Prelude> 22.4 - 22.399999618530273 > 3.8146972514141453e-7 > > and you did notice that the difference is about the size of > single precision epsilon? > Well that's why. You asked for a number with 24 bits of > precision that was close to 22.4, and 22.399999618530273 > was indeed the answer you got. > >> System.out.println(f); // 22.4 > PrintStream.println(float) calls > PrintStream.print(float) calls > String.valueOf(float) calls > Float.toString(float) > > whose documentation says (for this value of m) tha > it will be the integer part, then a dot, then "as > any, but only as many, more digits as are needed to > uniquely distinguish the argument value from adjacent > values of type 'float'". > > So the value that is printed is the _least_ precise > decimal representation that cannot be mistaken for > a different single-precision (24 bits of precision) > float. > > If you convert the same value to a double in Java and > print that in Java, you will get, well, try it: > > L% cat NotErlangsFault.java > public class NotErlangsFault { > public static void main(String[] args) { > final String s = "22.4"; > final float f = Float.parseFloat(s); > final double d = (double)f; > > System.out.println(s + " converts to float " + f + > " which as double is " + d); > } > } > > L% java NotErlangsFault > 22.4 converts to float 22.4 which as double is 22.399999618530273 > > See? Nothing to do with Erlang at all. > >> OtpErlangFloat ef = new OtpErlangFloat(f); >> System.out.println(ef.toString()); // 22.399999618530273 > And this tells us that an OtpErlangFloat contains a Java double, > which is *more* precise than the float you gave it. >> Then in Erlang: >> >> mochinum:digits(Ef). >> "22.399999618530273" > And this is the right answer. >> but >> >> mochinum:digits(22.4). >> "22.4" >> >> Is there a way to keep the precision when constructing >> OtpErlangDouble/OtpErlangFloat? > No precision whatsoever was lost in that step. > > The precision was lost when you first converted a String > to a float. The value in the float really *is* much > better approximated by 22.399999618530273 than by 22.4. > From dscvlt@REDACTED Thu Aug 9 15:32:56 2012 From: dscvlt@REDACTED (Ashley Holman) Date: Thu, 9 Aug 2012 23:02:56 +0930 Subject: [erlang-questions] Execution delays Message-ID: Hi, (code attached) I've just began learning Erlang over the last few days and so far a really enjoyed programming in it. As a learning exercise, I decided to implement the classic "factorial" function: fac(N) = N*fac(N-1); F(0) = 1. For calculating large factorials, this takes a long time (fac(200000) takes 42 secs on my laptop). So, I made some optimisations (using condense_terms/1 and shuffle/1 - but you can basically ignore those). However, it still was only using 100% of 1 cpu core, and I have 2 cores, so this was a good opportunity to learn an important feature of Erlang, spawning processes! I partitioned the list into 3 sublists and spawned a process to multiply each sublist. eg. [pseudo code] Proc1([1,2,3]) * Proc2([4, 5, 6]) * Proc3(7, 8, 9) should give me the result of fac(9) using 3 processes. This appears to be working but I'm completely confused about certain delays in execution I'm seeing - I'm not sure where it is spending some of its time in the code. I put in some debugging, and it appears that the child processes (fac_multi_actor) are finishing their calculations and returning the results, but the receiver (fac_multi_collect) doesn't return for several seconds later. The whole point of this was to try to optimise it, so I would like to know where these few seconds are going (~30% of execution time). fac:fac(200000) is taking ~9.5s using the three processes, but based on the debugging it looks like it should be completing in 6. Something else hangs around using 100% of one core for several seconds later. I've attached the code in case anyone is able to take a look at this behaviour. PS. I realise it will take some effort for someone to actually bother reading the code and trying to reproduce my problem, so no problem if I don't get any answers. In case anyone feels like helping though it would be much appreciated because I'm a bit confused at the moment! Thanks Ash -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: fac.erl Type: application/octet-stream Size: 2429 bytes Desc: not available URL: From dmercer@REDACTED Thu Aug 9 15:38:11 2012 From: dmercer@REDACTED (David Mercer) Date: Thu, 9 Aug 2012 08:38:11 -0500 Subject: [erlang-questions] Execution delays In-Reply-To: References: Message-ID: <00e401cd7634$38ec2730$aac47590$@com> I did not look at your code, but given your description of the symptoms, and that you have 2 cores working on 3 sublists, and that you observe one of those cores pegging for a few seconds after the other one completes, one hypothesis worth testing is that only 2 of the 3 sublists are complete and that the third is still processing (on one core). From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Ashley Holman Sent: Thursday, August 09, 2012 8:33 AM To: erlang-questions@REDACTED Subject: [erlang-questions] Execution delays Hi, (code attached) I've just began learning Erlang over the last few days and so far a really enjoyed programming in it. As a learning exercise, I decided to implement the classic "factorial" function: fac(N) = N*fac(N-1); F(0) = 1. For calculating large factorials, this takes a long time (fac(200000) takes 42 secs on my laptop). So, I made some optimisations (using condense_terms/1 and shuffle/1 - but you can basically ignore those). However, it still was only using 100% of 1 cpu core, and I have 2 cores, so this was a good opportunity to learn an important feature of Erlang, spawning processes! I partitioned the list into 3 sublists and spawned a process to multiply each sublist. eg. [pseudo code] Proc1([1,2,3]) * Proc2([4, 5, 6]) * Proc3(7, 8, 9) should give me the result of fac(9) using 3 processes. This appears to be working but I'm completely confused about certain delays in execution I'm seeing - I'm not sure where it is spending some of its time in the code. I put in some debugging, and it appears that the child processes (fac_multi_actor) are finishing their calculations and returning the results, but the receiver (fac_multi_collect) doesn't return for several seconds later. The whole point of this was to try to optimise it, so I would like to know where these few seconds are going (~30% of execution time). fac:fac(200000) is taking ~9.5s using the three processes, but based on the debugging it looks like it should be completing in 6. Something else hangs around using 100% of one core for several seconds later. I've attached the code in case anyone is able to take a look at this behaviour. PS. I realise it will take some effort for someone to actually bother reading the code and trying to reproduce my problem, so no problem if I don't get any answers. In case anyone feels like helping though it would be much appreciated because I'm a bit confused at the moment! Thanks Ash -------------- next part -------------- An HTML attachment was scrubbed... URL: From dscvlt@REDACTED Thu Aug 9 15:45:30 2012 From: dscvlt@REDACTED (Ashley Holman) Date: Thu, 9 Aug 2012 23:15:30 +0930 Subject: [erlang-questions] Execution delays In-Reply-To: References: Message-ID: Argh, silly me! I think I just figured out what's happening. It's the final multiplication in fac_multi_collect/2 that's having to run after the third process returns. It is multiplying some pretty massive numbers so that would explain it. This raises a more general question though - are there some profiling tools out there so that I can more easily find out where the time is going and when things are executing? Thanks very much. On Thu, Aug 9, 2012 at 11:02 PM, Ashley Holman wrote: > Hi, > > (code attached) > > I've just began learning Erlang over the last few days and so far a really > enjoyed programming in it. As a learning exercise, I decided to implement > the classic "factorial" function: fac(N) = N*fac(N-1); F(0) = 1. For > calculating large factorials, this takes a long time (fac(200000) takes 42 > secs on my laptop). > > So, I made some optimisations (using condense_terms/1 and shuffle/1 - but > you can basically ignore those). However, it still was only using 100% of > 1 cpu core, and I have 2 cores, so this was a good opportunity to learn an > important feature of Erlang, spawning processes! I partitioned the list > into 3 sublists and spawned a process to multiply each sublist. eg. > [pseudo code] Proc1([1,2,3]) * Proc2([4, 5, 6]) * Proc3(7, 8, 9) should > give me the result of fac(9) using 3 processes. > > This appears to be working but I'm completely confused about certain > delays in execution I'm seeing - I'm not sure where it is spending some of > its time in the code. I put in some debugging, and it appears that the > child processes (fac_multi_actor) are finishing their calculations and > returning the results, but the receiver (fac_multi_collect) doesn't return > for several seconds later. > > The whole point of this was to try to optimise it, so I would like to know > where these few seconds are going (~30% of execution time). > > fac:fac(200000) is taking ~9.5s using the three processes, but based on > the debugging it looks like it should be completing in 6. Something else > hangs around using 100% of one core for several seconds later. I've > attached the code in case anyone is able to take a look at this behaviour. > > PS. I realise it will take some effort for someone to actually bother > reading the code and trying to reproduce my problem, so no problem if I > don't get any answers. In case anyone feels like helping though it would > be much appreciated because I'm a bit confused at the moment! > > Thanks > Ash > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.leopold@REDACTED Thu Aug 9 16:08:10 2012 From: ulf.leopold@REDACTED (Ulf Leopold) Date: Thu, 9 Aug 2012 16:08:10 +0200 Subject: [erlang-questions] Unexpected {#Ref<0.0.0.110>, {error, closed}} message on passive SSL socket in R15 Message-ID: Hi! When using an SSL socket in passive mode I receive a {#Ref<0.0.0.110>,{error,closed}} message after the connection is closed. I only see this message in R15 but not in R14B04. The example below can be used to reproduce the problem. I this expected behavior? Thanks, Ulf -module(test1). -export([f/0]). f() -> ssl:start(), {ok, S} = ssl:connect("www.google.com", 443, [{active, false}], 10000), ok = ssl:send(S, "GET / HTTP1.1\r\n\r\n"), get_all(S), ok = ssl:close(S), receive {_, {error,closed}} = X -> io:format("got a: ~p~n", [X]) after 1000 -> io:format("no extra message~n") end. get_all(S) -> {ok, D} = ssl:recv(S, 0), get_all(S, [D]). get_all(S, Acc) -> case ssl:recv(S, 0) of {error, closed} -> lists:append(lists:reverse(Acc)); {ok, D} -> get_all(S, [D | Acc]) end. Example run: Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [kernel-poll:false] Eshell V5.8.5 (abort with ^G) 1> c(test1). {ok,test1} 2> test1:f(). no extra message Erlang R15B (erts-5.9) [source] [64-bit] [smp:8:8] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9 (abort with ^G) 1> c(test1). {ok,test1} 2> test1:f(). got a: {#Ref<0.0.0.110>,{error,closed}} -------------- next part -------------- An HTML attachment was scrubbed... URL: From jared.kofron@REDACTED Thu Aug 9 16:17:28 2012 From: jared.kofron@REDACTED (Jared Kofron) Date: Thu, 9 Aug 2012 07:17:28 -0700 Subject: [erlang-questions] Execution delays In-Reply-To: References: Message-ID: Hi Ashley, You might give fprof a whirl (http://www.erlang.org/doc/man/fprof.html). JK On Thu, Aug 9, 2012 at 6:45 AM, Ashley Holman wrote: > Argh, silly me! I think I just figured out what's happening. It's the > final multiplication in fac_multi_collect/2 that's having to run after the > third process returns. It is multiplying some pretty massive numbers so > that would explain it. > > This raises a more general question though - are there some profiling > tools out there so that I can more easily find out where the time is going > and when things are executing? > > Thanks very much. > > > On Thu, Aug 9, 2012 at 11:02 PM, Ashley Holman wrote: > >> Hi, >> >> (code attached) >> >> I've just began learning Erlang over the last few days and so far a >> really enjoyed programming in it. As a learning exercise, I decided to >> implement the classic "factorial" function: fac(N) = N*fac(N-1); F(0) = 1. >> For calculating large factorials, this takes a long time (fac(200000) >> takes 42 secs on my laptop). >> >> So, I made some optimisations (using condense_terms/1 and shuffle/1 - but >> you can basically ignore those). However, it still was only using 100% of >> 1 cpu core, and I have 2 cores, so this was a good opportunity to learn an >> important feature of Erlang, spawning processes! I partitioned the list >> into 3 sublists and spawned a process to multiply each sublist. eg. >> [pseudo code] Proc1([1,2,3]) * Proc2([4, 5, 6]) * Proc3(7, 8, 9) should >> give me the result of fac(9) using 3 processes. >> >> This appears to be working but I'm completely confused about certain >> delays in execution I'm seeing - I'm not sure where it is spending some of >> its time in the code. I put in some debugging, and it appears that the >> child processes (fac_multi_actor) are finishing their calculations and >> returning the results, but the receiver (fac_multi_collect) doesn't return >> for several seconds later. >> >> The whole point of this was to try to optimise it, so I would like to >> know where these few seconds are going (~30% of execution time). >> >> fac:fac(200000) is taking ~9.5s using the three processes, but based on >> the debugging it looks like it should be completing in 6. Something else >> hangs around using 100% of one core for several seconds later. I've >> attached the code in case anyone is able to take a look at this behaviour. >> >> PS. I realise it will take some effort for someone to actually bother >> reading the code and trying to reproduce my problem, so no problem if I >> don't get any answers. In case anyone feels like helping though it would >> be much appreciated because I'm a bit confused at the moment! >> >> Thanks >> Ash >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Thu Aug 9 19:07:06 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Thu, 9 Aug 2012 14:07:06 -0300 Subject: [erlang-questions] user_drv cannot properly print new lines Message-ID: Hello everyone, If I run the following command: $ erl -noinput -s user_drv And then try to send any io message to the user process, the message appears skewed: 1> io:format(user, "foo\nbar\nbaz", []). foo bar bazok However, sending the message to io makes it appear correctly: 2> io:format(standard_io, "foo\nbar\nbaz", []). foo bar bazok This becomes a major problem when printing error logger messages to tty, which become completely unreadable. I have found out this issue when starting an Elixir shell with user_drv. The issue also appears with LFE and any other time user_drv is started manually. Does anyone know a fix to this problem or how I could debug it further? * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Thu Aug 9 19:17:44 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Thu, 9 Aug 2012 14:17:44 -0300 Subject: [erlang-questions] user_drv cannot properly print new lines In-Reply-To: References: Message-ID: More info: Running on Mac OS Snow Leopard, Erlang R15B or R15B01. user_drv:start() is spawning user:start() and not 'tty -c -e'. * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * On Thu, Aug 9, 2012 at 2:07 PM, Jos? Valim wrote: > Hello everyone, > > If I run the following command: > > $ erl -noinput -s user_drv > > > And then try to send any io message to the user process, the message > appears skewed: > > 1> io:format(user, "foo\nbar\nbaz", []). > > foo > bar > bazok > > > However, sending the message to io makes it appear correctly: > > 2> io:format(standard_io, "foo\nbar\nbaz", []). > foo > bar > bazok > > > This becomes a major problem when printing error logger messages to tty, > which become completely unreadable. > > I have found out this issue when starting an Elixir shell with user_drv. > The issue also appears with LFE and any other time user_drv is started > manually. > > Does anyone know a fix to this problem or how I could debug it further? > > * > Jos? Valim > www.plataformatec.com.br > Founder and Lead Developer > * > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ashwin.mangale@REDACTED Thu Aug 9 19:24:09 2012 From: ashwin.mangale@REDACTED (Ashwin Mangale) Date: Thu, 9 Aug 2012 22:54:09 +0530 Subject: [erlang-questions] user_drv cannot properly print new lines In-Reply-To: References: Message-ID: Hi, Instead of: io:format(user, "foo\nbar\nbaz", []). Try: io:format(user, "foo\r\nbar\r\nbaz", []). This should give the proper formatted output. Regards, Ashwin On Thu, Aug 9, 2012 at 10:47 PM, Jos? Valim wrote: > More info: > > Running on Mac OS Snow Leopard, Erlang R15B or R15B01. > > user_drv:start() is spawning user:start() and not 'tty -c -e'. > > > > * > Jos? Valim > www.plataformatec.com.br > Founder and Lead Developer > * > > > On Thu, Aug 9, 2012 at 2:07 PM, Jos? Valim wrote: > >> Hello everyone, >> >> If I run the following command: >> >> $ erl -noinput -s user_drv >> >> >> And then try to send any io message to the user process, the message >> appears skewed: >> >> 1> io:format(user, "foo\nbar\nbaz", []). >> >> foo >> bar >> bazok >> >> >> However, sending the message to io makes it appear correctly: >> >> 2> io:format(standard_io, "foo\nbar\nbaz", []). >> foo >> bar >> bazok >> >> >> This becomes a major problem when printing error logger messages to tty, >> which become completely unreadable. >> >> I have found out this issue when starting an Elixir shell with user_drv. >> The issue also appears with LFE and any other time user_drv is started >> manually. >> >> Does anyone know a fix to this problem or how I could debug it further? >> >> * >> Jos? Valim >> www.plataformatec.com.br >> Founder and Lead Developer >> * >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Thu Aug 9 19:27:11 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Thu, 9 Aug 2012 14:27:11 -0300 Subject: [erlang-questions] user_drv cannot properly print new lines In-Reply-To: References: Message-ID: Yes, it does, thanks! However, error_logger, lager and everything else just sends \n. So while I could control my output, I cannot control such tools. Maybe something in user_drv or user should convert \n to \r\n? * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * On Thu, Aug 9, 2012 at 2:24 PM, Ashwin Mangale wrote: > Hi, > > Instead of: > > io:format(user, "foo\nbar\nbaz", []). > > Try: > io:format(user, "foo\r\nbar\r\nbaz", []). > > > This should give the proper formatted output. > > Regards, > Ashwin > > On Thu, Aug 9, 2012 at 10:47 PM, Jos? Valim wrote: > >> More info: >> >> Running on Mac OS Snow Leopard, Erlang R15B or R15B01. >> >> user_drv:start() is spawning user:start() and not 'tty -c -e'. >> >> >> >> * >> Jos? Valim >> www.plataformatec.com.br >> Founder and Lead Developer >> * >> >> >> On Thu, Aug 9, 2012 at 2:07 PM, Jos? Valim wrote: >> >>> Hello everyone, >>> >>> If I run the following command: >>> >>> $ erl -noinput -s user_drv >>> >>> >>> And then try to send any io message to the user process, the message >>> appears skewed: >>> >>> 1> io:format(user, "foo\nbar\nbaz", []). >>> >>> foo >>> bar >>> bazok >>> >>> >>> However, sending the message to io makes it appear correctly: >>> >>> 2> io:format(standard_io, "foo\nbar\nbaz", []). >>> foo >>> bar >>> bazok >>> >>> >>> This becomes a major problem when printing error logger messages to tty, >>> which become completely unreadable. >>> >>> I have found out this issue when starting an Elixir shell with user_drv. >>> The issue also appears with LFE and any other time user_drv is started >>> manually. >>> >>> Does anyone know a fix to this problem or how I could debug it further? >>> >>> * >>> Jos? Valim >>> www.plataformatec.com.br >>> Founder and Lead Developer >>> * >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Thu Aug 9 20:27:40 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Thu, 9 Aug 2012 15:27:40 -0300 Subject: [erlang-questions] user_drv cannot properly print new lines In-Reply-To: References: Message-ID: More information: When starting erlang with -noshell, process_info(whereis(user)) is inside user:server_loop. Otherwise, it is inside group:server_loop. Everything else appears to be the same. Also, if we invoke io:format(user, ...)as in the previous e-mail when starting with -noshell, everything works fine. The message just becomes skewed with -noshell when user_drv:start() is invoked. I still could not detect what is causing the side effect. * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * On Thu, Aug 9, 2012 at 2:27 PM, Jos? Valim wrote: > Yes, it does, thanks! > > However, error_logger, lager and everything else just sends \n. > So while I could control my output, I cannot control such tools. > Maybe something in user_drv or user should convert \n to \r\n? > > > > * > Jos? Valim > www.plataformatec.com.br > Founder and Lead Developer > * > > > On Thu, Aug 9, 2012 at 2:24 PM, Ashwin Mangale wrote: > >> Hi, >> >> Instead of: >> >> io:format(user, "foo\nbar\nbaz", []). >> >> Try: >> io:format(user, "foo\r\nbar\r\nbaz", []). >> >> >> This should give the proper formatted output. >> >> Regards, >> Ashwin >> >> On Thu, Aug 9, 2012 at 10:47 PM, Jos? Valim wrote: >> >>> More info: >>> >>> Running on Mac OS Snow Leopard, Erlang R15B or R15B01. >>> >>> user_drv:start() is spawning user:start() and not 'tty -c -e'. >>> >>> >>> >>> * >>> Jos? Valim >>> www.plataformatec.com.br >>> Founder and Lead Developer >>> * >>> >>> >>> On Thu, Aug 9, 2012 at 2:07 PM, Jos? Valim wrote: >>> >>>> Hello everyone, >>>> >>>> If I run the following command: >>>> >>>> $ erl -noinput -s user_drv >>>> >>>> >>>> And then try to send any io message to the user process, the message >>>> appears skewed: >>>> >>>> 1> io:format(user, "foo\nbar\nbaz", []). >>>> >>>> foo >>>> bar >>>> bazok >>>> >>>> >>>> However, sending the message to io makes it appear correctly: >>>> >>>> 2> io:format(standard_io, "foo\nbar\nbaz", []). >>>> foo >>>> bar >>>> bazok >>>> >>>> >>>> This becomes a major problem when printing error logger messages to >>>> tty, which become completely unreadable. >>>> >>>> I have found out this issue when starting an Elixir shell with user_drv. >>>> The issue also appears with LFE and any other time user_drv is started >>>> manually. >>>> >>>> Does anyone know a fix to this problem or how I could debug it further? >>>> >>>> * >>>> Jos? Valim >>>> www.plataformatec.com.br >>>> Founder and Lead Developer >>>> * >>>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From garret.smith@REDACTED Thu Aug 9 21:00:53 2012 From: garret.smith@REDACTED (Garret Smith) Date: Thu, 9 Aug 2012 12:00:53 -0700 Subject: [erlang-questions] problems with creating table named 'user' in ets In-Reply-To: References: Message-ID: On Wed, Aug 8, 2012 at 8:07 PM, cavaluo wrote: > Hi, > when i create a new ets table named 'user',result returned 'ok',and also > i can do any operaton on it,but i can't find in The Table Visualizer,why? > > (cavaluo@REDACTED)15> ets:new(user,[set,public,named_table]). > user > (cavaluo@REDACTED)18> ets:info(user). > [{compressed,false}, > {memory,316}, > {owner,<0.37.0>}, > {heir,none}, > {name,user}, > {size,0}, > {node,'cavaluo@REDACTED'}, > {named_table,true}, > {type,set}, > {keypos,1}, > {protection,public}] > > (cavaluo@REDACTED)20> ets:insert(user,{test,1,2,3}). > true > (cavaluo@REDACTED)21> ets:lookup(user,test). > [{test,1,2,3}] > > I cannot see the table in the Table View either. I am not sure why. However, I can see the table through observer: > observer:start(). tv doesn't get much attention any more as observer is superseding it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Thu Aug 9 21:58:31 2012 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 09 Aug 2012 12:58:31 -0700 Subject: [erlang-questions] user_drv cannot properly print new lines In-Reply-To: References: Message-ID: <50241667.6000004@gmail.com> This problem also pops up if a linked-in driver has any usage of stdout. For some reason there is usage of '\r\n' as newlines within the erlang VM somewhere, which seems very awkward when running on unix/linux. On 08/09/2012 10:24 AM, Ashwin Mangale wrote: > Hi, > > Instead of: > io:format(user, "foo\nbar\nbaz", []). > > Try: > io:format(user, "foo\r\nbar\r\nbaz", []). > > > This should give the proper formatted output. > > Regards, > Ashwin > > On Thu, Aug 9, 2012 at 10:47 PM, Jos? Valim > wrote: > > More info: > > Running on Mac OS Snow Leopard, Erlang R15B or R15B01. > > user_drv:start() is spawning user:start() and not 'tty -c -e'. > > > > * > *Jos? Valim* > www.plataformatec.com.br > Founder and Lead Developer > * > > > On Thu, Aug 9, 2012 at 2:07 PM, Jos? Valim > wrote: > > Hello everyone, > > If I run the following command: > > $ erl -noinput -s user_drv > > > And then try to send any io message to the user process, the message appears skewed: > > 1> io:format(user, "foo\nbar\nbaz", []). > > foo > bar > bazok > > > However, sending the message to io makes it appear correctly: > > 2> io:format(standard_io, "foo\nbar\nbaz", []). > foo > bar > bazok > > > This becomes a major problem when printing error logger messages to tty, which become completely unreadable. > > I have found out this issue when starting an Elixir shell with user_drv. > The issue also appears with LFE and any other time user_drv is started manually. > > Does anyone know a fix to this problem or how I could debug it further? > > * > *Jos? Valim* > www.plataformatec.com.br > Founder and Lead Developer > * > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Fri Aug 10 10:51:21 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 10 Aug 2012 10:51:21 +0200 Subject: [erlang-questions] Unexpected {#Ref<0.0.0.110>, {error, closed}} message on passive SSL socket in R15 In-Reply-To: References: Message-ID: Hi! This is a known bug that has been fixed in the upcoming release of the ssl application. Regards Ingela Erlang/OTP team - Ericsson AB 2012/8/9, Ulf Leopold : > Hi! > > When using an SSL socket in passive mode I receive a > {#Ref<0.0.0.110>,{error,closed}} message after the connection is closed. I > only see this message in R15 but not in R14B04. The example below can be > used to reproduce the problem. I this expected behavior? > > Thanks, > > Ulf > > > -module(test1). > -export([f/0]). > > f() -> > ssl:start(), > > {ok, S} = ssl:connect("www.google.com", 443, [{active, false}], 10000), > ok = ssl:send(S, "GET / HTTP1.1\r\n\r\n"), > get_all(S), > ok = ssl:close(S), > > receive > {_, {error,closed}} = X -> io:format("got a: ~p~n", [X]) > after 1000 > -> io:format("no extra message~n") > end. > > get_all(S) -> > {ok, D} = ssl:recv(S, 0), > get_all(S, [D]). > > get_all(S, Acc) -> > case ssl:recv(S, 0) of > {error, closed} -> lists:append(lists:reverse(Acc)); > {ok, D} -> get_all(S, [D | Acc]) > end. > > > Example run: > > Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:8:8] [rq:8] > [async-threads:0] [kernel-poll:false] > > Eshell V5.8.5 (abort with ^G) > 1> c(test1). > {ok,test1} > 2> test1:f(). > no extra message > > > Erlang R15B (erts-5.9) [source] [64-bit] [smp:8:8] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.9 (abort with ^G) > 1> c(test1). > {ok,test1} > 2> test1:f(). > got a: {#Ref<0.0.0.110>,{error,closed}} > From ulf.leopold@REDACTED Fri Aug 10 11:01:44 2012 From: ulf.leopold@REDACTED (Ulf Leopold) Date: Fri, 10 Aug 2012 11:01:44 +0200 Subject: [erlang-questions] Unexpected {#Ref<0.0.0.110>, {error, closed}} message on passive SSL socket in R15 In-Reply-To: References: Message-ID: Great! Thanks, Ulf On Fri, Aug 10, 2012 at 10:51 AM, Ingela Andin wrote: > Hi! > > This is a known bug that has been fixed in the upcoming release of the > ssl application. > > Regards Ingela Erlang/OTP team - Ericsson AB > > 2012/8/9, Ulf Leopold : > > Hi! > > > > When using an SSL socket in passive mode I receive a > > {#Ref<0.0.0.110>,{error,closed}} message after the connection is closed. > I > > only see this message in R15 but not in R14B04. The example below can be > > used to reproduce the problem. I this expected behavior? > > > > Thanks, > > > > Ulf > > > > > > -module(test1). > > -export([f/0]). > > > > f() -> > > ssl:start(), > > > > {ok, S} = ssl:connect("www.google.com", 443, [{active, false}], > 10000), > > ok = ssl:send(S, "GET / HTTP1.1\r\n\r\n"), > > get_all(S), > > ok = ssl:close(S), > > > > receive > > {_, {error,closed}} = X -> io:format("got a: ~p~n", [X]) > > after 1000 > > -> io:format("no extra message~n") > > end. > > > > get_all(S) -> > > {ok, D} = ssl:recv(S, 0), > > get_all(S, [D]). > > > > get_all(S, Acc) -> > > case ssl:recv(S, 0) of > > {error, closed} -> lists:append(lists:reverse(Acc)); > > {ok, D} -> get_all(S, [D | Acc]) > > end. > > > > > > Example run: > > > > Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:8:8] [rq:8] > > [async-threads:0] [kernel-poll:false] > > > > Eshell V5.8.5 (abort with ^G) > > 1> c(test1). > > {ok,test1} > > 2> test1:f(). > > no extra message > > > > > > Erlang R15B (erts-5.9) [source] [64-bit] [smp:8:8] [async-threads:0] > > [hipe] [kernel-poll:false] > > > > Eshell V5.9 (abort with ^G) > > 1> c(test1). > > {ok,test1} > > 2> test1:f(). > > got a: {#Ref<0.0.0.110>,{error,closed}} > > > -- * Ulf Leopold* ------------------------------ Klarna AB Norra Stationsgatan 61 SE-113 43 Stockholm Tel: +468- 120 120 00 Dir: +46 - 70 001 2980 Fax: +468- 120 120 99 Web: www.klarna.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Fri Aug 10 15:07:42 2012 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 10 Aug 2012 14:07:42 +0100 (BST) Subject: [erlang-questions] yecc question In-Reply-To: Message-ID: No, yecc works on the type or class of a token not on its value and there is no provision to look at a tokens value when parsing. Robert ----- Original Message ----- > Hi, let's say that I am parsing a string and that string has a > backreference to something previously seen. > like this : > {key: 'nameA', value: n1}, > {key: 'nameB', value:n3}, > {key: 1, value:n2}, > {key: 2, value:n4} > > here the values of key of 1 and 2 are backreferences to the 'nameA' > and 'nameB'. > Is there a way to lookup the value of reference 1 during parsing, > rather than afterwards ? > > Thanks, > Radu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From tuncer.ayaz@REDACTED Fri Aug 10 19:56:54 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 10 Aug 2012 19:56:54 +0200 Subject: [erlang-questions] Common Test & skip_cases In-Reply-To: References: Message-ID: On Thu, Mar 1, 2012 at 2:22 PM, Fred Hebert wrote: > Hi there mailing list, > > I have a very simple question. I was revisiting Common Test for LYSE's next > chapter, and I remembered that Common Test has the following skip options in > its test specifications: > > {skip_suites, [NodeRefs,] DirRef, Suites, Comment} > {skip_cases, [NodeRefs,] DirRef, Suite, Cases, Comment} > > These two options have their equivalent in tuples keyed by 'suites' and > 'cases'. > However, there is no documented option for 'skip_groups' even though there > is a 'groups' tuple. Is there any special reason for that? Fred, any news on this one? From max.lapshin@REDACTED Fri Aug 10 23:10:19 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 11 Aug 2012 01:10:19 +0400 Subject: [erlang-questions] stockdb: stock exchange quotes database in erlang Message-ID: Hi. I want to announce our library stockdb: https://github.com/maxlapshin/stockdb It is a specialized database, that can store stock exchange quotes in a compressed append-only file, with indexes and auto-repairing. Level of compression is rather high: out data compressed 100-200 times, comparing to CSV. It is written in erlang with some places, reimplemented in C for speed. It is rather fast: about 80 000 inserts per second and about 200 000 reads per second on moderate hardware. It is an infrastructure code, so we've decided to opensource it. Thanks to my colleague, Danil Zagoskin for doing most part of work. From mononcqc@REDACTED Sat Aug 11 04:20:35 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 10 Aug 2012 22:20:35 -0400 Subject: [erlang-questions] Common Test & skip_cases In-Reply-To: References: Message-ID: <5025C173.4020406@ferd.ca> Yeah, it turns out the code for that one was implemented, but it was neither documented nor enabled. Following the LYSE chapter being made public, I've received an e-mail telling me it had been turned on in later versions. Newer versions should have it enabled. I can't remember which version, but it's documented in the chapter now. On 12-08-10 1:56 PM, Tuncer Ayaz wrote: > On Thu, Mar 1, 2012 at 2:22 PM, Fred Hebert wrote: >> Hi there mailing list, >> >> I have a very simple question. I was revisiting Common Test for LYSE's next >> chapter, and I remembered that Common Test has the following skip options in >> its test specifications: >> >> {skip_suites, [NodeRefs,] DirRef, Suites, Comment} >> {skip_cases, [NodeRefs,] DirRef, Suite, Cases, Comment} >> >> These two options have their equivalent in tuples keyed by 'suites' and >> 'cases'. >> However, there is no documented option for 'skip_groups' even though there >> is a 'groups' tuple. Is there any special reason for that? > Fred, any news on this one? From rory@REDACTED Sun Aug 12 14:08:26 2012 From: rory@REDACTED (Rory Byrne) Date: Sun, 12 Aug 2012 14:08:26 +0200 Subject: [erlang-questions] Adding StartTLS support to eldap (for use Ejabberd) In-Reply-To: References: <20120807005003.GD13487@almeida.jinsky.com> <20120808230330.GG13487@almeida.jinsky.com> Message-ID: <20120812120826.GJ13487@almeida.jinsky.com> On Wed, Aug 08, 2012 at 07:40:00PM -0600, OvermindDL1 wrote: > > > > Adding some of these features to OTP eldap wouldn't be too difficult, > > but I'd guess that only a subset of these changes would get accepted. > > And at the end of the day, I can't see ejabberd giving up their current > > flexibility where they can make any changes to their LDAP code as and > > when they need to. > > > > > > I, personally, would certainly not mind StartTLS support and > asynchronous and concurrent connections being added to the OTP > version... *hint*. > Actually, adding this stuff is probably more involved than I suggested. If I needed this kind of functionality at the moment I'd grab the ejabberd version, dust it off a little, add the STARTTLS functionality, and use that. It's got to be amazingly well tested at this stage! I think it has some code in there to use the ejabberd logging system and probably a couple of other ejabberd-specific things you'll want to strip out or modify, but other than that it's probably good to go. The only potential downside for some is that it's released under the GPLv2, but I doubt this is a problem in most cases. From ghenry@REDACTED Sun Aug 12 19:16:28 2012 From: ghenry@REDACTED (Gavin Henry) Date: Sun, 12 Aug 2012 18:16:28 +0100 Subject: [erlang-questions] Adding StartTLS support to eldap (for use Ejabberd) In-Reply-To: <20120812120826.GJ13487@almeida.jinsky.com> References: <20120807005003.GD13487@almeida.jinsky.com> <20120808230330.GG13487@almeida.jinsky.com> <20120812120826.GJ13487@almeida.jinsky.com> Message-ID: On 12 August 2012 13:08, Rory Byrne wrote: > On Wed, Aug 08, 2012 at 07:40:00PM -0600, OvermindDL1 wrote: >> > >> > Adding some of these features to OTP eldap wouldn't be too difficult, >> > but I'd guess that only a subset of these changes would get accepted. >> > And at the end of the day, I can't see ejabberd giving up their current >> > flexibility where they can make any changes to their LDAP code as and >> > when they need to. >> > >> > >> >> I, personally, would certainly not mind StartTLS support and >> asynchronous and concurrent connections being added to the OTP >> version... *hint*. >> > > Actually, adding this stuff is probably more involved than I suggested. > > If I needed this kind of functionality at the moment I'd grab the ejabberd > version, dust it off a little, add the STARTTLS functionality, and use > that. It's got to be amazingly well tested at this stage! I think it has > some code in there to use the ejabberd logging system and probably a couple > of other ejabberd-specific things you'll want to strip out or modify, but > other than that it's probably good to go. The only potential downside for > some is that it's released under the GPLv2, but I doubt this is a problem > in most cases. I'm not fussed about GPL 2! -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry@REDACTED Open Source. Open Solutions(tm). http://www.suretecsystems.com/ Suretec Systems is a limited company registered in Scotland. Registered number: SC258005. Registered office: 24 Cormack Park, Rothienorman, Inverurie, Aberdeenshire, AB51 8GL. Subject to disclaimer at http://www.suretecgroup.com/disclaimer.html Do you know we have our own VoIP provider called SureVoIP? See http://www.surevoip.co.uk Did you see our API? http://www.surevoip.co.uk/api From zabrane3@REDACTED Mon Aug 13 00:06:57 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 13 Aug 2012 00:06:57 +0200 Subject: [erlang-questions] prim_inet socket incompatible with file:sendfile/2/5 In-Reply-To: References: <3152847A-9DA4-45FC-A16A-4D016A9D059E@gmail.com> Message-ID: <4E6D7BD2-273D-494B-B789-86A73A2A5297@gmail.com> Hi Lukas, Sorry for the late response. Please find attached a minimalistic example. 1> prim_sendfile:start(). and try: curl http://127.0.0.1:8080/ Nothing happens. What am I doing wrong? If I use the standard gen_tcp:accept/1, gen_tcp:recv/2, gen_tcp:send/2 on the socket, sendfile works as expected. But not with a socket handled by prim_inet. Why? Thanks for your help. Regards, Zabrane -------------- next part -------------- A non-text attachment was scrubbed... Name: prim_sendfile.erl Type: application/octet-stream Size: 1831 bytes Desc: not available URL: -------------- next part -------------- On Aug 8, 2012, at 11:00 AM, Lukas Larsson wrote: > Hi! > > There are a couple of reasons why this might not work. Could you show > me a small example of what you are trying to do? > > Lukas > > On Fri, Jul 27, 2012 at 6:57 PM, Zabrane Mickael wrote: >> Hi, >> >> Is there any reason why a client socket created from prim_inet >> do not work with the new file:sendfile fonction? >> >> I got {error, badarg}. >> >> Any undocumented function to solves this? >> >> Regards, >> Zabrane >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From zabrane3@REDACTED Mon Aug 13 00:08:59 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 13 Aug 2012 00:08:59 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.6.1 In-Reply-To: <50228834.9000601@ninenines.eu> References: <502254C6.1040407@ninenines.eu> <3F3CC0C2-B801-48D9-BA35-B2D0693E89EE@gmail.com> <5022745A.2080302@ninenines.eu> <50228834.9000601@ninenines.eu> Message-ID: <91CC3352-2689-44FB-8480-668C9DAF12D9@gmail.com> Hi guys, > I can simplify the code a lot by not using it, all without sacrificing performance because I can still use binary BIFs instead. The only thing that requires a little Erlang code is parsing the request line. That should be very easy too ! Regards, Zabrane From bengt.kleberg@REDACTED Mon Aug 13 07:26:38 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 13 Aug 2012 07:26:38 +0200 Subject: [erlang-questions] link: On Curiosity and its software Message-ID: <1344835598.4929.2.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> Greetings, A small article about the Curiosity rover software similarities to Erlang/OTP. http://jlouisramblings.blogspot.nl/2012/08/getting-25-megalines-of-code-to-behave.html#!/2012/08/getting-25-megalines-of-code-to-behave.html bengt From lukas@REDACTED Mon Aug 13 10:40:26 2012 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 13 Aug 2012 10:40:26 +0200 Subject: [erlang-questions] prim_inet socket incompatible with file:sendfile/2/5 In-Reply-To: <4E6D7BD2-273D-494B-B789-86A73A2A5297@gmail.com> References: <3152847A-9DA4-45FC-A16A-4D016A9D059E@gmail.com> <4E6D7BD2-273D-494B-B789-86A73A2A5297@gmail.com> Message-ID: Hi, When using the prim_inet API you have to be careful with what you remove. In this case you have to call `erlang:port_set_data(CSock,inet_tcp)` after accepting the socket. The port data is used by erlang code to figure out which type of socket the port represents. Lukas On Mon, Aug 13, 2012 at 12:06 AM, Zabrane Mickael wrote: > Hi Lukas, > > Sorry for the late response. > > Please find attached a minimalistic example. > > 1> prim_sendfile:start(). > > and try: > curl http://127.0.0.1:8080/ > > Nothing happens. > What am I doing wrong? > > If I use the standard gen_tcp:accept/1, gen_tcp:recv/2, gen_tcp:send/2 on the socket, sendfile works as expected. > But not with a socket handled by prim_inet. Why? > > Thanks for your help. > > Regards, > Zabrane > > > > > > On Aug 8, 2012, at 11:00 AM, Lukas Larsson wrote: > >> Hi! >> >> There are a couple of reasons why this might not work. Could you show >> me a small example of what you are trying to do? >> >> Lukas >> >> On Fri, Jul 27, 2012 at 6:57 PM, Zabrane Mickael wrote: >>> Hi, >>> >>> Is there any reason why a client socket created from prim_inet >>> do not work with the new file:sendfile fonction? >>> >>> I got {error, badarg}. >>> >>> Any undocumented function to solves this? >>> >>> Regards, >>> Zabrane >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From zabrane3@REDACTED Mon Aug 13 11:06:47 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 13 Aug 2012 11:06:47 +0200 Subject: [erlang-questions] prim_inet socket incompatible with file:sendfile/2/5 In-Reply-To: References: <3152847A-9DA4-45FC-A16A-4D016A9D059E@gmail.com> <4E6D7BD2-273D-494B-B789-86A73A2A5297@gmail.com> Message-ID: <820B3E23-2FE8-4DAE-93F1-18664F90A9D5@gmail.com> Hi Lukas, Perfect now. Anything else I should know avour prim_inet usage? Regards, Zabrane On Aug 13, 2012, at 10:40 AM, Lukas Larsson wrote: > Hi, > > When using the prim_inet API you have to be careful with what you > remove. In this case you have to call > `erlang:port_set_data(CSock,inet_tcp)` after accepting the socket. The > port data is used by erlang code to figure out which type of socket > the port represents. > > Lukas > > On Mon, Aug 13, 2012 at 12:06 AM, Zabrane Mickael wrote: >> Hi Lukas, >> >> Sorry for the late response. >> >> Please find attached a minimalistic example. >> >> 1> prim_sendfile:start(). >> >> and try: >> curl http://127.0.0.1:8080/ >> >> Nothing happens. >> What am I doing wrong? >> >> If I use the standard gen_tcp:accept/1, gen_tcp:recv/2, gen_tcp:send/2 on the socket, sendfile works as expected. >> But not with a socket handled by prim_inet. Why? >> >> Thanks for your help. >> >> Regards, >> Zabrane >> >> >> >> >> >> On Aug 8, 2012, at 11:00 AM, Lukas Larsson wrote: >> >>> Hi! >>> >>> There are a couple of reasons why this might not work. Could you show >>> me a small example of what you are trying to do? >>> >>> Lukas >>> >>> On Fri, Jul 27, 2012 at 6:57 PM, Zabrane Mickael wrote: >>>> Hi, >>>> >>>> Is there any reason why a client socket created from prim_inet >>>> do not work with the new file:sendfile fonction? >>>> >>>> I got {error, badarg}. >>>> >>>> Any undocumented function to solves this? >>>> >>>> Regards, >>>> Zabrane >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From watson.timothy@REDACTED Mon Aug 13 12:52:31 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 13 Aug 2012 11:52:31 +0100 Subject: [erlang-questions] common test, test_server and ct_hooks Message-ID: <4B447EE0-BDF3-43DB-836E-3400A26B2CAE@gmail.com> I'm currently using the ct_hooks mechanism to do some automatic setup/teardown work between suites, groups and test cases. It works great, but I've discovered that under some circumstances, stashing framework specific information in the Config variable doesn't work, because when a test case fails (specifically with a time trap timeout failure) then the configuration set seems to disappear before it arrives in the hook's post_end_per_testcase handler. I've tracked the reason down to 1153 of test_server: %%! This is a temporary fix that keeps Test Server alive during %%! execution of a parallel test case group, when sometimes %%! this clause gets called with EndConf == undefined. See OTP-9594 %%! for more info. EndConf1 = if EndConf == undefined -> [{tc_status,{failed,{Mod,end_per_testcase,Why}}}]; true -> EndConf end, Where can I go and look at OTP-9594, as I can't understand why my test case configuration has disappeared and without this, I'll need to start storing the relevant information in an ets table (or equivalent) in order to work around this issue. Cheers, Tim -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 235 bytes Desc: Message signed with OpenPGP using GPGMail URL: From pekadan@REDACTED Mon Aug 13 14:38:35 2012 From: pekadan@REDACTED (Peter Andersson) Date: Mon, 13 Aug 2012 14:38:35 +0200 Subject: [erlang-questions] common test, test_server and ct_hooks In-Reply-To: <4B447EE0-BDF3-43DB-836E-3400A26B2CAE@gmail.com> References: <4B447EE0-BDF3-43DB-836E-3400A26B2CAE@gmail.com> Message-ID: Hi Tim, When a test case times out, the test server control process kills the test case process, on which the Config data is known (it's last returned by init_per_testcase/2 on the same hanging process). The result is that the latest Config value is unknown when the call to the post_end_per_testcase hook function takes place. This is not a bug as such (and OTP-9594 is a different issue), but it's a limitation for sure. There are 2 possible solutions to this problem: 1. You keep saving the last known Config data (or at least your relevant subset of it) in the hook state. If the test case times out, you can restore the data from there. No need to use an ets table. You will be able to restore the Config data that init_per_testcase/2 was called with, but not the final value that was passed to the test case function. 2. Common Test gets updated so that the control process saves the last known Config for a test case so that it can be restored and passed to the post_end_per_testcase hook call. The result will be the same as #1 above, but it will make it easier to implement the CTH callback module. It might also be possible to have the test case process send the Config data to the control process after init_per_testcase, so that the very latest Config can be saved and restored. Implementation of this can be somewhat tricky because of parallel test case groups - but certainly not impossible! ;-) I'd be very happy (well, sort of) to implement #2 above. Will #1 be an ok workaround for you until #2 is in place? I know it's usually not appropriate to be talking about doing #1 and #2, but in this context you know what I mean! :-) /Peter Ericsson AB, Erlang/OTP 2012/8/13, Tim Watson : > I'm currently using the ct_hooks mechanism to do some automatic > setup/teardown work between suites, groups and test cases. It works great, > but I've discovered that under some circumstances, stashing framework > specific information in the Config variable doesn't work, because when a > test case fails (specifically with a time trap timeout failure) then the > configuration set seems to disappear before it arrives in the hook's > post_end_per_testcase handler. I've tracked the reason down to 1153 of > test_server: > > %%! This is a temporary fix that keeps Test Server alive during > %%! execution of a parallel test case group, when sometimes > %%! this clause gets called with EndConf == undefined. See OTP-9594 > %%! for more info. > EndConf1 = if EndConf == undefined -> > [{tc_status,{failed,{Mod,end_per_testcase,Why}}}]; > true -> > EndConf > end, > > Where can I go and look at OTP-9594, as I can't understand why my test case > configuration has disappeared and without this, I'll need to start storing > the relevant information in an ets table (or equivalent) in order to work > around this issue. > > Cheers, > Tim > > > From majek04@REDACTED Mon Aug 13 16:43:14 2012 From: majek04@REDACTED (Marek Majkowski) Date: Mon, 13 Aug 2012 15:43:14 +0100 Subject: [erlang-questions] Sock.js and Cowboy - Example In-Reply-To: References: Message-ID: Jai, Here's the answer to your question on SockJS mailing list: https://groups.google.com/d/msg/sockjs/m5UtBVcmJ_g/0M92khnAGmgJ Marek On Thu, Aug 9, 2012 at 4:23 AM, Jai Gupta wrote: > I am new to both Sock.js and Erlang/Cowboy. I am trying to run a example. > > I have already install Erlang on Mac. > $ erl > Erlang R15B01 (erts-5.9.1) [source] [smp:4:4] [async-threads:0] [hipe] > [kernel-poll:false] > Eshell V5.9.1 (abort with ^G) > 1> > > I am trying to run a simple echo example from my app. > > ====rebar.config==== > %% -*- erlang -*- > > {erl_opts, [debug_info]}. %, fail_on_warning]}. > > {deps, > [ > {sockjs, ".*", {git, "git://github.com/sockjs/sockjs-erlang.git", > {branch, "master"}}} > ] > }. > ================ > > Note that dependency is in following way. My app has dependency on Sockjs > which has dependency on Cowboy which has dependency on Proper. > $ ./rebar get-deps > After above command I get 3 folders in my deps folder. cowboy, proper & > sockjs > $ ./rebar compile > > I am getting following error. > > $ ./deps/sockjs/examples/cowboy_echo.erl > escript: exception error: undefined function sockjs_handler:init_state/4 > in function cowboy_echo:main/1 (./deps/sockjs/examples/cowboy_echo.erl, > line 17) > in call from escript:run/2 (escript.erl, line 727) > in call from escript:start/1 (escript.erl, line 277) > in call from init:start_it/1 > in call from init:start_em/1 > > I have also tried coping cowboy_echo.erl to ./src/ folder but it gives same > error. > > Are there any pointers that can help me? > > Note: I can successfully run this example if I directly use sockjs-erlang > rebar.config but I want to run it on my app which describes sockjs as > dependency. > > Jai > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From moxford@REDACTED Mon Aug 13 16:55:45 2012 From: moxford@REDACTED (Mike Oxford) Date: Mon, 13 Aug 2012 07:55:45 -0700 Subject: [erlang-questions] Mnesia ETS crash In-Reply-To: References: Message-ID: No one has seen this before? Thanks, -mox On Tue, Aug 7, 2012 at 11:25 AM, Mike Oxford wrote: > Further information: > 1) The table is question does exist, has existed, and has been running > for a while. > 2) safe_fixtable fails with badarg if the table does not exist. > > #1 runs for a good long while (month?) and then suddenly #2 pops up. > Why does mnesia 'lose' the table? > > Here's more context > > > ===== ALIVE Tue Jul 31 17:22:07 PDT 2012 > > ===== ALIVE Tue Jul 31 17:37:07 PDT 2012 > > ===== ALIVE Tue Jul 31 17:52:07 PDT 2012 > > ===== Tue Jul 31 18:04:17 PDT 2012 > > =ERROR REPORT==== 31-Jul-2012::18:04:17 === > Error in process <0.15759.10> on node '' with exit value: > {badarg, > [{ets,safe_fixtable,[px_channel_log,false]},{mnesia_lib,db_fixtable,3}, > > {mnesia_loader,cleanup_tab_copier,3},{mnesia_loader,send_table,3},{mnesia_controller,send_and_reply,2}]} > > > ===== > ===== LOGGING STARTED Tue Jul 31 18:04:19 PDT 2012 > ===== > Exec: /usr/local/chat-game-gwm/current/erts-5.8.4/bin/erlexec -boot > /usr/local/chat-game-gwm/current/releases/1/chat -mode > embedded -config /usr/local/chat-game-gwm/current/etc/app.config > -args_file /usr/local/chat-game-gwm/current/etc/vm.args -- > console > Root: /usr/local/chat-game-gwm/current > Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:24:24] [rq:24] > [async-threads:64] [hipe] [kernel-poll:true] > > > =ERROR REPORT==== 31-Jul-2012::18:04:22 === > Mnesia('gwm_game@'): ** ERROR ** mnesia_event got > {inconsistent_database, starting_partitioned_network, > 'gwm_game@'} > DEBUG: mnesia_helpers:118 - "{ \"Connecting to cluster: \" , NodeList }" > > > > > On Mon, Aug 6, 2012 at 5:04 PM, Mike Oxford wrote: > >> =ERROR REPORT==== 18-Jun-2012::15:01:49 === >> Error in process <0.1743.0> on node '' with exit value: >> {badarg,[{ets,safe_fixtable,[px_channel_log,false]},{mnesia_lib,db_fixtable,3},{mnesia_loader,cleanup_tab_copier,3},{mnesia_loader,send_table,3},{mnesia_controller,send_and_re >> ply,2}]} >> >> Logging shows it was then restarted and {inconsistent_database, >> starting_partitioned_network... >> >> mnesia is Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:24:24] >> [rq:24] [async-threads:64] [hipe] [kernel-poll:true] so it shouldn't fall >> victim to the safe_fixtable/next issues from R13x >> >> This has happened to us twice now, both times exactly the same error. >> System runs other ram_only nodes as well with no issues; this one is a >> disc_ table with light load. >> >> Thoughts? >> >> -mox >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From moxford@REDACTED Mon Aug 13 19:28:01 2012 From: moxford@REDACTED (Mike Oxford) Date: Mon, 13 Aug 2012 10:28:01 -0700 Subject: [erlang-questions] R15B01 for CentOS5 Message-ID: I see the excellent Erlang Solutions has a CentOS6 RPM, but no CentOS5 RPM. Anyone have a good location for a 5 RPM, by chance? Thanks! -mox -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Mon Aug 13 19:52:15 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 13 Aug 2012 19:52:15 +0200 Subject: [erlang-questions] binary:part/2/3 Message-ID: <097E05BA-5618-40D6-A348-2E750EE9B5A9@gmail.com> Hi guys, I'd like to know if the binary returned by "binary:part/2/3" (http://www.erlang.org/doc/man/binary.html#part-2) is always referencing Subject (i.e a sub-binary) or not? The doc is clear about this fact for "binary:split/2/3" but nothing is mentionned for "binary:part/2/3". Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From jai@REDACTED Mon Aug 13 19:59:30 2012 From: jai@REDACTED (Jai Gupta) Date: Mon, 13 Aug 2012 23:29:30 +0530 Subject: [erlang-questions] Sock.js and Cowboy - Example In-Reply-To: References: Message-ID: > Here's the answer to your question on SockJS mailing list: > > https://groups.google.com/d/msg/sockjs/m5UtBVcmJ_g/0M92khnAGmgJ Thank you Marek for your detailed answer. Both Sock.js and Cowboy are great projects and it was good experience of having opportunity of using them. I am trying to write some reasons so that you and can give your suggestions. - We aim to build a benchmarking client for performance analysis. This tool should be able to perform all sorts of actives that any user might do. Both, with SockJS & Cowboy do not have a client which I could use. SocketIO had a client which could be used in Node.js. However, SocketIO port of Erlang is looking as if it is not in active development. - Coming from C++/PHP background Erlang language is looking difficult. Working on Node.js was natural but performance of Node.js is not good. In our benchmark Node.js was performing 40k msg/sec using SockJS module and around 60k msg/sec when using ZeroMQ. Even PHP gives 400K msg/sec with ZeroMQ (although it has leaking memory ). We are trying some basic prototype on other languages. Jai -------------- next part -------------- An HTML attachment was scrubbed... URL: From ronny.meeus@REDACTED Mon Aug 13 21:11:53 2012 From: ronny.meeus@REDACTED (Ronny Meeus) Date: Mon, 13 Aug 2012 21:11:53 +0200 Subject: [erlang-questions] UDP client/server performance Message-ID: Hello I have created a simple UDP based client/server application. The code is available on bitbucket (https://bitbucket.org/meeusr/erl-examples) in the networking directory. The server is just an echo server running in 1 process. The client can behave like a sync client (sending 1 message and waiting for the reply) or like an async client (sending X messages before waiting for all the replies). I see that my PC (Intel(R) Quad code i7 CPU 860 @ 2.80GHz) is able to process something like 7K - 10K messages per second. Both the sync and the async are in the same order of magnitude. Please note that I use the loopback interface. $ erl +sbt db Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.1 (abort with ^G) 1> c(gen_udp_test). gen_udp_test.erl:47: Warning: variable 'Data' is unused gen_udp_test.erl:47: Warning: variable 'Msg' is unused gen_udp_test.erl:62: Warning: variable 'Data' is unused gen_udp_test.erl:62: Warning: variable 'Msg' is unused {ok,gen_udp_test} 2> gen_udp_test:start_server(). true ... 12> gen_udp_test:client_async(5000). stop : 841482 ok 13> gen_udp_test:client_async(6000). stop : 1082679 ok The timings are the number of usec for the processing of the X number of messages (round-trip). If I implement the same application in C (client and server are 2 applications), I see that the application performs an order of magnitude better for the sync solution (86Kpkts/sec) and 100 times better (700Kpkts/sec) for the async version. The code for this is also available in the repo on bitbucket. I know there this test is only looking to a part of the application (transport of the data), but I find the difference rather big. Is this normal or is there something wrong in my Erlang code? Thanks. Regards, Ronny From mjtruog@REDACTED Mon Aug 13 21:12:20 2012 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 13 Aug 2012 12:12:20 -0700 Subject: [erlang-questions] Sock.js and Cowboy - Example In-Reply-To: References: Message-ID: <50295194.4020700@gmail.com> On 08/13/2012 10:59 AM, Jai Gupta wrote: > > Here's the answer to your question on SockJS mailing list: > > https://groups.google.com/d/msg/sockjs/m5UtBVcmJ_g/0M92khnAGmgJ > > > Thank you Marek for your detailed answer. > > Both Sock.js and Cowboy are great projects and it was good experience of having opportunity of using them. > > I am trying to write some reasons so that you and can give your suggestions. > > * We aim to build a benchmarking client for performance analysis. This tool should be able to perform all sorts of actives that any user might do. Both, with SockJS & Cowboy do not have a client which I could use. SocketIO had a client which could be used in Node.js. However, SocketIO port of Erlang is looking as if it is not in active development. > * Coming from C++/PHP background Erlang language is looking difficult. Working on Node.js was natural but performance of Node.js is not good. In our benchmark Node.js was performing 40k msg/sec using SockJS module and around 60k msg/sec when using ZeroMQ. Even PHP gives 400K msg/sec with ZeroMQ (although it has leaking memory ). > > We are trying some basic prototype on other languages. > > It is much better if you use https://github.com/zeromq/erlzmq2 for Erlang<->ZeroMQ integration. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbothma@REDACTED Mon Aug 13 21:42:27 2012 From: jbothma@REDACTED (JD Bothma) Date: Mon, 13 Aug 2012 21:42:27 +0200 Subject: [erlang-questions] UDP client/server performance In-Reply-To: References: Message-ID: I'm curious: What are the numbers like (c versus erlang) when it's between two physical machines? Or between two (vmware or virtualbox, not xen) virtual machines? Or said otherwise: does the fact that it's the same machine, network stack and loopback interface affect the comparison between c and erlang? JD On 13 August 2012 21:11, Ronny Meeus wrote: > Hello > > I have created a simple UDP based client/server application. > The code is available on bitbucket > (https://bitbucket.org/meeusr/erl-examples) in the networking > directory. > > The server is just an echo server running in 1 process. > The client can behave like a sync client (sending 1 message and > waiting for the reply) or like an async client (sending X messages > before waiting for all the replies). > I see that my PC (Intel(R) Quad code i7 CPU 860 @ 2.80GHz) is able to > process something like 7K - 10K messages per second. Both the sync and > the async are in the same order of magnitude. Please note that I use > the loopback interface. > > $ erl +sbt db > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.9.1 (abort with ^G) > 1> c(gen_udp_test). > gen_udp_test.erl:47: Warning: variable 'Data' is unused > gen_udp_test.erl:47: Warning: variable 'Msg' is unused > gen_udp_test.erl:62: Warning: variable 'Data' is unused > gen_udp_test.erl:62: Warning: variable 'Msg' is unused > {ok,gen_udp_test} > 2> gen_udp_test:start_server(). > true > ... > 12> gen_udp_test:client_async(5000). > stop : 841482 > ok > 13> gen_udp_test:client_async(6000). > stop : 1082679 > ok > > The timings are the number of usec for the processing of the X number > of messages (round-trip). > > If I implement the same application in C (client and server are 2 > applications), I see that the application performs an order of > magnitude better for the sync solution (86Kpkts/sec) and 100 times > better (700Kpkts/sec) for the async version. > The code for this is also available in the repo on bitbucket. > > I know there this test is only looking to a part of the application > (transport of the data), but I find the difference rather big. > Is this normal or is there something wrong in my Erlang code? > > Thanks. > Regards, > Ronny > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From moxford@REDACTED Mon Aug 13 21:56:23 2012 From: moxford@REDACTED (Mike Oxford) Date: Mon, 13 Aug 2012 12:56:23 -0700 Subject: [erlang-questions] UDP client/server performance In-Reply-To: References: Message-ID: Erlang will be slower and will consume more memory than C. That's pretty much par for the course due to type munging and layers between you and the hardware, including the VM itself. Some are using libevent through an Erlang port to get around this. You can also try "when I receive a packet, spawn an unlinked process to do the reply" which may unblock things. (Not sure, haven't profiled it myself.) Writing C directly to low-level structures is known to be much faster than the Erlang VM. You can also see this in action at http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-3 -mox On Mon, Aug 13, 2012 at 12:11 PM, Ronny Meeus wrote: > Hello > > I have created a simple UDP based client/server application. > The code is available on bitbucket > (https://bitbucket.org/meeusr/erl-examples) in the networking > directory. > > The server is just an echo server running in 1 process. > The client can behave like a sync client (sending 1 message and > waiting for the reply) or like an async client (sending X messages > before waiting for all the replies). > I see that my PC (Intel(R) Quad code i7 CPU 860 @ 2.80GHz) is able to > process something like 7K - 10K messages per second. Both the sync and > the async are in the same order of magnitude. Please note that I use > the loopback interface. > > $ erl +sbt db > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.9.1 (abort with ^G) > 1> c(gen_udp_test). > gen_udp_test.erl:47: Warning: variable 'Data' is unused > gen_udp_test.erl:47: Warning: variable 'Msg' is unused > gen_udp_test.erl:62: Warning: variable 'Data' is unused > gen_udp_test.erl:62: Warning: variable 'Msg' is unused > {ok,gen_udp_test} > 2> gen_udp_test:start_server(). > true > ... > 12> gen_udp_test:client_async(5000). > stop : 841482 > ok > 13> gen_udp_test:client_async(6000). > stop : 1082679 > ok > > The timings are the number of usec for the processing of the X number > of messages (round-trip). > > If I implement the same application in C (client and server are 2 > applications), I see that the application performs an order of > magnitude better for the sync solution (86Kpkts/sec) and 100 times > better (700Kpkts/sec) for the async version. > The code for this is also available in the repo on bitbucket. > > I know there this test is only looking to a part of the application > (transport of the data), but I find the difference rather big. > Is this normal or is there something wrong in my Erlang code? > > Thanks. > Regards, > Ronny > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ronny.meeus@REDACTED Mon Aug 13 22:29:30 2012 From: ronny.meeus@REDACTED (Ronny Meeus) Date: Mon, 13 Aug 2012 22:29:30 +0200 Subject: [erlang-questions] UDP client/server performance In-Reply-To: References: Message-ID: Hello I did a quick check. - The values between 2 machines are comparable for Erlang. - For the C code there is a difference: the async version drops from 700Kpks/sec to 400Kpks/sec while the sync version drops from 86Kpkts/sec to 10Kpkts/sec. On Mon, Aug 13, 2012 at 9:42 PM, JD Bothma wrote: > I'm curious: > > What are the numbers like (c versus erlang) when it's between two > physical machines? Or between two (vmware or virtualbox, not xen) > virtual machines? > > Or said otherwise: does the fact that it's the same machine, network > stack and loopback interface affect the comparison between c and > erlang? > > JD > > On 13 August 2012 21:11, Ronny Meeus wrote: >> Hello >> >> I have created a simple UDP based client/server application. >> The code is available on bitbucket >> (https://bitbucket.org/meeusr/erl-examples) in the networking >> directory. >> >> The server is just an echo server running in 1 process. >> The client can behave like a sync client (sending 1 message and >> waiting for the reply) or like an async client (sending X messages >> before waiting for all the replies). >> I see that my PC (Intel(R) Quad code i7 CPU 860 @ 2.80GHz) is able to >> process something like 7K - 10K messages per second. Both the sync and >> the async are in the same order of magnitude. Please note that I use >> the loopback interface. >> >> $ erl +sbt db >> Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] >> [async-threads:0] [hipe] [kernel-poll:false] >> >> Eshell V5.9.1 (abort with ^G) >> 1> c(gen_udp_test). >> gen_udp_test.erl:47: Warning: variable 'Data' is unused >> gen_udp_test.erl:47: Warning: variable 'Msg' is unused >> gen_udp_test.erl:62: Warning: variable 'Data' is unused >> gen_udp_test.erl:62: Warning: variable 'Msg' is unused >> {ok,gen_udp_test} >> 2> gen_udp_test:start_server(). >> true >> ... >> 12> gen_udp_test:client_async(5000). >> stop : 841482 >> ok >> 13> gen_udp_test:client_async(6000). >> stop : 1082679 >> ok >> >> The timings are the number of usec for the processing of the X number >> of messages (round-trip). >> >> If I implement the same application in C (client and server are 2 >> applications), I see that the application performs an order of >> magnitude better for the sync solution (86Kpkts/sec) and 100 times >> better (700Kpkts/sec) for the async version. >> The code for this is also available in the repo on bitbucket. >> >> I know there this test is only looking to a part of the application >> (transport of the data), but I find the difference rather big. >> Is this normal or is there something wrong in my Erlang code? >> >> Thanks. >> Regards, >> Ronny >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From thomas.elsgaard@REDACTED Mon Aug 13 23:54:46 2012 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Mon, 13 Aug 2012 23:54:46 +0200 Subject: [erlang-questions] Remote command execution via SSH ? Message-ID: Hi Does anybody have a good simple example of executing commands on a remote server via SSH? Nothing fancy, just something like: ------ Connect to host Execute command '/bin/ls' Read output from command Close connection ------ Best regards Thomas From moxford@REDACTED Tue Aug 14 00:03:19 2012 From: moxford@REDACTED (Mike Oxford) Date: Mon, 13 Aug 2012 15:03:19 -0700 Subject: [erlang-questions] Remote command execution via SSH ? In-Reply-To: References: Message-ID: OT, but I'll answer anyways. :) ssh server.com /bin/ls > output.txt -mox On Mon, Aug 13, 2012 at 2:54 PM, Thomas Elsgaard wrote: > Hi > > Does anybody have a good simple example of executing commands on a > remote server via SSH? > > Nothing fancy, just something like: > > ------ > Connect to host > Execute command '/bin/ls' > Read output from command > Close connection > ------ > > Best regards > > Thomas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.elsgaard@REDACTED Tue Aug 14 00:06:25 2012 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Tue, 14 Aug 2012 00:06:25 +0200 Subject: [erlang-questions] Remote command execution via SSH ? In-Reply-To: References: Message-ID: > ssh server.com /bin/ls > output.txt > > -mox Ha ha got the point.. Okay, then the Erlang'ish way then ;-) ///Thomas From mjtruog@REDACTED Tue Aug 14 00:57:06 2012 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 13 Aug 2012 15:57:06 -0700 Subject: [erlang-questions] Remote command execution via SSH ? In-Reply-To: References: Message-ID: <50298642.9000902@gmail.com> On 08/13/2012 02:54 PM, Thomas Elsgaard wrote: > Hi > > Does anybody have a good simple example of executing commands on a > remote server via SSH? > > Nothing fancy, just something like: > > ------ > Connect to host > Execute command '/bin/ls' > Read output from command > Close connection > ------ > Just do: |slave:start_link('nodename', 'user', "-rsh ssh"). See: http://www.erlang.org/doc/man/slave.html http://stackoverflow.com/questions/2658912/erlang-starting-slave-node Keep in mind that your ssh configuration needs to be set, so you really should test it outside of Erlang before attempting the login within Erlang. Tsung does this for client machines if you wanted to see a production example (|||http://tsung.erlang-projects.org/|). | -------------- next part -------------- An HTML attachment was scrubbed... URL: From jvliwanag@REDACTED Tue Aug 14 03:48:46 2012 From: jvliwanag@REDACTED (Jan Vincent Liwanag) Date: Tue, 14 Aug 2012 09:48:46 +0800 Subject: [erlang-questions] Unit Testing gen_server callbacks Message-ID: Hi, What are your thoughts on unit testing gen_server (or other gen_*) callbacks (handle_call, etc) directly? Is this an ok practice? Or should unit tests test only the public api? On my end, I find testing gen_server callbacks to be rather fragile. Eager to hear out other thoughts. Thanks, Jan Vincent Liwanag jvliwanag@REDACTED +63 (999) 888-0247 -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.arnon@REDACTED Tue Aug 14 07:03:49 2012 From: alex.arnon@REDACTED (Alex Arnon) Date: Tue, 14 Aug 2012 08:03:49 +0300 Subject: [erlang-questions] link: On Curiosity and its software In-Reply-To: <1344835598.4929.2.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> References: <1344835598.4929.2.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> Message-ID: Thank you! On Mon, Aug 13, 2012 at 8:26 AM, Bengt Kleberg wrote: > Greetings, > > A small article about the Curiosity rover software similarities to > Erlang/OTP. > > > http://jlouisramblings.blogspot.nl/2012/08/getting-25-megalines-of-code-to-behave.html#!/2012/08/getting-25-megalines-of-code-to-behave.html > > > bengt > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ronny.meeus@REDACTED Tue Aug 14 08:25:23 2012 From: ronny.meeus@REDACTED (Ronny Meeus) Date: Tue, 14 Aug 2012 08:25:23 +0200 Subject: [erlang-questions] UDP client/server performance In-Reply-To: References: Message-ID: Is there any code available that implements a NIF to have access to UDP sockets? I have found the procket implementation but it looks like it is using a separate process to communicate with the socket. This sounds like overkill to me for this test. Ronny On Mon, Aug 13, 2012 at 10:29 PM, Ronny Meeus wrote: > Hello > > I did a quick check. > - The values between 2 machines are comparable for Erlang. > - For the C code there is a difference: the async version drops from > 700Kpks/sec to 400Kpks/sec while the sync version drops from > 86Kpkts/sec to 10Kpkts/sec. > > > On Mon, Aug 13, 2012 at 9:42 PM, JD Bothma wrote: >> I'm curious: >> >> What are the numbers like (c versus erlang) when it's between two >> physical machines? Or between two (vmware or virtualbox, not xen) >> virtual machines? >> >> Or said otherwise: does the fact that it's the same machine, network >> stack and loopback interface affect the comparison between c and >> erlang? >> >> JD >> >> On 13 August 2012 21:11, Ronny Meeus wrote: >>> Hello >>> >>> I have created a simple UDP based client/server application. >>> The code is available on bitbucket >>> (https://bitbucket.org/meeusr/erl-examples) in the networking >>> directory. >>> >>> The server is just an echo server running in 1 process. >>> The client can behave like a sync client (sending 1 message and >>> waiting for the reply) or like an async client (sending X messages >>> before waiting for all the replies). >>> I see that my PC (Intel(R) Quad code i7 CPU 860 @ 2.80GHz) is able to >>> process something like 7K - 10K messages per second. Both the sync and >>> the async are in the same order of magnitude. Please note that I use >>> the loopback interface. >>> >>> $ erl +sbt db >>> Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] >>> [async-threads:0] [hipe] [kernel-poll:false] >>> >>> Eshell V5.9.1 (abort with ^G) >>> 1> c(gen_udp_test). >>> gen_udp_test.erl:47: Warning: variable 'Data' is unused >>> gen_udp_test.erl:47: Warning: variable 'Msg' is unused >>> gen_udp_test.erl:62: Warning: variable 'Data' is unused >>> gen_udp_test.erl:62: Warning: variable 'Msg' is unused >>> {ok,gen_udp_test} >>> 2> gen_udp_test:start_server(). >>> true >>> ... >>> 12> gen_udp_test:client_async(5000). >>> stop : 841482 >>> ok >>> 13> gen_udp_test:client_async(6000). >>> stop : 1082679 >>> ok >>> >>> The timings are the number of usec for the processing of the X number >>> of messages (round-trip). >>> >>> If I implement the same application in C (client and server are 2 >>> applications), I see that the application performs an order of >>> magnitude better for the sync solution (86Kpkts/sec) and 100 times >>> better (700Kpkts/sec) for the async version. >>> The code for this is also available in the repo on bitbucket. >>> >>> I know there this test is only looking to a part of the application >>> (transport of the data), but I find the difference rather big. >>> Is this normal or is there something wrong in my Erlang code? >>> >>> Thanks. >>> Regards, >>> Ronny >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions From daniel.eliasson@REDACTED Tue Aug 14 08:41:38 2012 From: daniel.eliasson@REDACTED (Daniel Eliasson) Date: Tue, 14 Aug 2012 08:41:38 +0200 Subject: [erlang-questions] Unit Testing gen_server callbacks In-Reply-To: References: Message-ID: It should be easy enough to unit test such things, just run a handle_call with the right arguments, and verify that the output and new state is correct? I don't think there's a need to actually start the gen_server itself and unit test the callbacks by going through gen_server:call, if that's what you mean. In what way do you find it to be fragile? Best, Daniel On 14 August 2012 03:48, Jan Vincent Liwanag wrote: > Hi, > > What are your thoughts on unit testing gen_server (or other gen_*) > callbacks (handle_call, etc) directly? Is this an ok practice? Or should > unit tests test only the public api? > > On my end, I find testing gen_server callbacks to be rather fragile. Eager > to hear out other thoughts. > > Thanks, > > Jan Vincent Liwanag > jvliwanag@REDACTED > +63 (999) 888-0247 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jvliwanag@REDACTED Tue Aug 14 09:57:01 2012 From: jvliwanag@REDACTED (Jan Vincent Liwanag) Date: Tue, 14 Aug 2012 15:57:01 +0800 Subject: [erlang-questions] Unit Testing gen_server callbacks In-Reply-To: References: Message-ID: <93FAAEBD-ECD3-4AF3-A73D-50E5A048B644@gmail.com> Given the following code: adder.erl: start(B) -> gen_server:start(?MODULE, B, []). add(Pid, N) -> gen_server:cast(Pid, {add, N}). get_total(Pid, N) -> gen_server:call(Pid, get_total). init(Total) -> {ok, Total}. handle_call(get_total, _From, Total) -> {reply, Total, Total}. handle_cast({add, N}, Total) -> {noreply, N+Total). It would be better to have a test such that: {ok, P} = adder:start(0), adder:add(P, 5), adder:add(P, 3), adder:add(P, 2), ?assertEqual(10, adder:get_running_total()). Rather than: ?assertEqual({noreply, 12}, handle_cast({add, 5}, 7)). ?assertEqual({reply, 100, 100}, handle_call(get_total, somepid, 100)). For one thing, if I change the gen_server state from an int to a record and fix up the code - the first set of tests should still succeed while the second won't. On Aug 14, 2012, at 2:41 PM, Daniel Eliasson wrote: > It should be easy enough to unit test such things, just run a handle_call with the right arguments, and verify that the output and new state is correct? > > I don't think there's a need to actually start the gen_server itself and unit test the callbacks by going through gen_server:call, if that's what you mean. In what way do you find it to be fragile? > > Best, > Daniel > > On 14 August 2012 03:48, Jan Vincent Liwanag wrote: > Hi, > > What are your thoughts on unit testing gen_server (or other gen_*) callbacks (handle_call, etc) directly? Is this an ok practice? Or should unit tests test only the public api? > > On my end, I find testing gen_server callbacks to be rather fragile. Eager to hear out other thoughts. > > Thanks, > > Jan Vincent Liwanag > jvliwanag@REDACTED > +63 (999) 888-0247 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > Jan Vincent Liwanag jvliwanag@REDACTED +63 (999) 888-0247 -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve@REDACTED Tue Aug 14 10:11:46 2012 From: steve@REDACTED (Steve Strong) Date: Tue, 14 Aug 2012 10:11:46 +0200 Subject: [erlang-questions] Unit Testing gen_server callbacks In-Reply-To: <93FAAEBD-ECD3-4AF3-A73D-50E5A048B644@gmail.com> References: <93FAAEBD-ECD3-4AF3-A73D-50E5A048B644@gmail.com> Message-ID: <84499F8B-EEC5-4A1F-B152-866EBE0F07B1@srstrong.com> I would much prefer the latter - i.e., to test the public interface. Otherwise you will end up with brittle tests that become a maintenance headache. That said, I have had occasions where some functionality was much easier to test through the internal implementation rather than through the public API, so I think it depends to a degree on the context. Cheers, Steve -------------------------- Steve Strong Director, id3as +34 636451137 @srstrong On Aug 14, 2012, at 9:57 AM, Jan Vincent Liwanag wrote: > Given the following code: > > adder.erl: > > start(B) -> gen_server:start(?MODULE, B, []). > add(Pid, N) -> gen_server:cast(Pid, {add, N}). > get_total(Pid, N) -> gen_server:call(Pid, get_total). > > init(Total) -> {ok, Total}. > > handle_call(get_total, _From, Total) -> {reply, Total, Total}. > handle_cast({add, N}, Total) -> {noreply, N+Total). > > > It would be better to have a test such that: > > {ok, P} = adder:start(0), > adder:add(P, 5), > adder:add(P, 3), > adder:add(P, 2), > ?assertEqual(10, adder:get_running_total()). > > Rather than: > > ?assertEqual({noreply, 12}, handle_cast({add, 5}, 7)). > ?assertEqual({reply, 100, 100}, handle_call(get_total, somepid, 100)). > > For one thing, if I change the gen_server state from an int to a record and fix up the code - the first set of tests should still succeed while the second won't. > > On Aug 14, 2012, at 2:41 PM, Daniel Eliasson wrote: > >> It should be easy enough to unit test such things, just run a handle_call with the right arguments, and verify that the output and new state is correct? >> >> I don't think there's a need to actually start the gen_server itself and unit test the callbacks by going through gen_server:call, if that's what you mean. In what way do you find it to be fragile? >> >> Best, >> Daniel >> >> On 14 August 2012 03:48, Jan Vincent Liwanag wrote: >> Hi, >> >> What are your thoughts on unit testing gen_server (or other gen_*) callbacks (handle_call, etc) directly? Is this an ok practice? Or should unit tests test only the public api? >> >> On my end, I find testing gen_server callbacks to be rather fragile. Eager to hear out other thoughts. >> >> Thanks, >> >> Jan Vincent Liwanag >> jvliwanag@REDACTED >> +63 (999) 888-0247 >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > Jan Vincent Liwanag > jvliwanag@REDACTED > +63 (999) 888-0247 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.elsgaard@REDACTED Tue Aug 14 10:39:39 2012 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Tue, 14 Aug 2012 10:39:39 +0200 Subject: [erlang-questions] Remote command execution via SSH ? In-Reply-To: <2979C5BB-D7D2-4705-8299-7DF13BDECBA0@jtendo.com> References: <2979C5BB-D7D2-4705-8299-7DF13BDECBA0@jtendo.com> Message-ID: > 5. Stop the channel process or close the connection if you need to (ssh:close/1). > > When in doubt, try to take a look at source files located in lib/ssh-2.1/src > I found them much more helpful than the ssh application documentation. > Good luck :) > > > -- > AR > Hi Adam Thanks, i will take a look! ///Thomas From vegayours@REDACTED Tue Aug 14 10:41:55 2012 From: vegayours@REDACTED (VegaS) Date: Tue, 14 Aug 2012 14:41:55 +0600 Subject: [erlang-questions] Parametric modules and Dialyzer Message-ID: Hello all! We are using parametric modules in our project and were quite happy with them unless we have started using Dialyzer. Dialyzer fails on parametric modules because functions in parametric modules are expanded with module params argument on compilation stage - so function's specs are unmatched. I know that parametric modules not the official part of Erlang, but I'd like to get both parametric modules and dialyzer working. I have tried to do some work on this: 1. Try to fix specs on parametric module's functions with parse_transform - simply extend specs on this stage, but this won't work, code just won't compile (because of specs for unknown functions) 2. More complicated approach - try to implement parametric modules with parse_transform (create "new" function and extend all module functions and specs). This looks working unless some code like SomeFun = local_fun/1, SomeFun(Arg) and some variations like this local function is assigned to record field. 3. Decided that there no way out besides patching OTP sources or not using parametric modules at all. After looking at OTP sources I have found, that parametric modules are expanded on on "expand_module" compiler pass. And on that stage specs are not expanded. I Have patched this stage with specs expand. All were correct, but dialyzer fails with message like specs were not fixed, so no luck... This happend because dialyzer using "abstract_code" from beam to extract specs and pass "save_abstract_code" is just before "expand_module" pass. Simple changing these passes order broke dialyzer at all. 4. The last attempt were to patch dialyzer - on the stage, when it extracts specs info from beam's abstract_code section. At this place I insert function's specs expanding with last parametric parameter - and it works! So the question is what the rigth way to make both parametric modules and dialyzer working? At my opinion there are no way except OTP sources patching - but what the rigth place for making such patch? I can attach my patch here but it is little ugly (I were tired by previous explorations) and I think I have chosen not rigth place for changing. Thanks, Lihanov Alexandr -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.eliasson@REDACTED Tue Aug 14 10:53:47 2012 From: daniel.eliasson@REDACTED (Daniel Eliasson) Date: Tue, 14 Aug 2012 10:53:47 +0200 Subject: [erlang-questions] Unit Testing gen_server callbacks In-Reply-To: <84499F8B-EEC5-4A1F-B152-866EBE0F07B1@srstrong.com> References: <93FAAEBD-ECD3-4AF3-A73D-50E5A048B644@gmail.com> <84499F8B-EEC5-4A1F-B152-866EBE0F07B1@srstrong.com> Message-ID: I guess it depends a bit on how advanced your server's state is. In the adder example above, it's easy enough to test via the public interface. However, if you want to test edge cases on the state, it can be a lot easier to be able to just pass in the state you want to begin from, than to start up a gen_server and send it the right sequence of commands to bring it to that state. I've been working on tests of a connection pool system, and it's a lot easier to just test that handle_call, handle_info et. al. do the right thing given an input state and a command, than it is to set up the gen_server and provoke the right kind of errors via mocking. Another way is to make the handle_call etc. simply destructure their input and pass on to other functions that actually perform the operations. Then you can unit test those functions separately. This might help avoid some of the brittleness mentioned in regards to changing the formats of calls or state. /Daniel On 14 August 2012 10:11, Steve Strong wrote: > I would much prefer the latter - i.e., to test the public interface. > Otherwise you will end up with brittle tests that become a maintenance > headache. That said, I have had occasions where some functionality was > much easier to test through the internal implementation rather than through > the public API, so I think it depends to a degree on the context. > > Cheers, > > Steve > > -------------------------- > *Steve Strong* > *Director, id3as* > +34 636451137 > @srstrong > > > > On Aug 14, 2012, at 9:57 AM, Jan Vincent Liwanag > wrote: > > Given the following code: > > adder.erl: > > start(B) -> gen_server:start(?MODULE, B, []). > add(Pid, N) -> gen_server:cast(Pid, {add, N}). > get_total(Pid, N) -> gen_server:call(Pid, get_total). > > init(Total) -> {ok, Total}. > > handle_call(get_total, _From, Total) -> {reply, Total, Total}. > handle_cast({add, N}, Total) -> {noreply, N+Total). > > > It would be better to have a test such that: > > {ok, P} = adder:start(0), > adder:add(P, 5), > adder:add(P, 3), > adder:add(P, 2), > ?assertEqual(10, adder:get_running_total()). > > Rather than: > > ?assertEqual({noreply, 12}, handle_cast({add, 5}, 7)). > ?assertEqual({reply, 100, 100}, handle_call(get_total, somepid, 100)). > > For one thing, if I change the gen_server state from an int to a record > and fix up the code - the first set of tests should still succeed while the > second won't. > > On Aug 14, 2012, at 2:41 PM, Daniel Eliasson wrote: > > It should be easy enough to unit test such things, just run a handle_call > with the right arguments, and verify that the output and new state is > correct? > > I don't think there's a need to actually start the gen_server itself and > unit test the callbacks by going through gen_server:call, if that's what > you mean. In what way do you find it to be fragile? > > Best, > Daniel > > On 14 August 2012 03:48, Jan Vincent Liwanag wrote: > >> Hi, >> >> What are your thoughts on unit testing gen_server (or other gen_*) >> callbacks (handle_call, etc) directly? Is this an ok practice? Or should >> unit tests test only the public api? >> >> On my end, I find testing gen_server callbacks to be rather fragile. >> Eager to hear out other thoughts. >> >> Thanks, >> >> Jan Vincent Liwanag >> jvliwanag@REDACTED >> +63 (999) 888-0247 >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > Jan Vincent Liwanag > jvliwanag@REDACTED > +63 (999) 888-0247 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.santos@REDACTED Tue Aug 14 11:40:19 2012 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 14 Aug 2012 05:40:19 -0400 Subject: [erlang-questions] UDP client/server performance In-Reply-To: References: Message-ID: <20120814094019.GA21542@ioctl> On Tue, Aug 14, 2012 at 08:25:23AM +0200, Ronny Meeus wrote: > Is there any code available that implements a NIF to have access to UDP sockets? > I have found the procket implementation but it looks like it is using > a separate process to communicate with the socket. This sounds like > overkill to me for this test. You can get the socket directly using procket:socket/3. See procket:sendto/4 and procket:recvfrom/4 as well for retrieving the struct sockaddr for the peer. -module(udp_srv). -export([s/0]). -define(UINT16(N), N:2/native-unsigned-integer-unit:8). -define(PF_INET, 2). -define(PORT, 6001). s() -> {ok,FD} = procket:socket(inet, dgram, 0), % sockaddr for Linux Sockaddr = <>, ok = procket:bind(FD, Sockaddr), Ref = erlang:open_port({fd, FD, FD}, [stream, binary]), loop(Ref). loop(Ref) -> receive {Ref,{data,Data}} -> error_logger:info_report(Data), loop(Ref) end. From jesper.louis.andersen@REDACTED Tue Aug 14 11:54:30 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 14 Aug 2012 11:54:30 +0200 Subject: [erlang-questions] binary:part/2/3 In-Reply-To: <097E05BA-5618-40D6-A348-2E750EE9B5A9@gmail.com> References: <097E05BA-5618-40D6-A348-2E750EE9B5A9@gmail.com> Message-ID: Try looking at the source. If I remember correctly, binary:split uses binary:part inside. If this is the case, then you can update the documentation as well to reflect the fact. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen On Aug 13, 2012, at 7:52 PM, Zabrane Mickael wrote: > Hi guys, > > I'd like to know if the binary returned by "binary:part/2/3" (http://www.erlang.org/doc/man/binary.html#part-2) > is always referencing Subject (i.e a sub-binary) or not? > > The doc is clear about this fact for "binary:split/2/3" but nothing is mentionned for "binary:part/2/3". > > Regards, > Zabrane > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Tue Aug 14 12:10:29 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 14 Aug 2012 12:10:29 +0200 Subject: [erlang-questions] binary:part/2/3 In-Reply-To: References: <097E05BA-5618-40D6-A348-2E750EE9B5A9@gmail.com> Message-ID: <19E076BE-AD6E-4B28-BDDB-D9ED043C4C76@gmail.com> Hi Jesper, You're right. From "binary.erl" source code: [..] split(Haystack,Needles,Options) -> ... do_split(Haystack,MList,0,Trim) ... do_split(H,[],N,true) when N >= byte_size(H) -> []; do_split(H,[],N,_) -> [binary:part(H,{N,byte_size(H)-N})]; <---- HERE do_split(H,[{A,B}|T],N,Trim) -> case binary:part(H,{N,A-N}) of <---- HERE ... So yes, binary:split uses binary:part for sure. Can OTP guys confirm this fact now? Regards, Zabrane On Aug 14, 2012, at 11:54 AM, Jesper Louis Andersen wrote: > Try looking at the source. If I remember correctly, binary:split uses binary:part inside. If this is the case, then you can update the documentation as well to reflect the fact. > > Jesper Louis Andersen > Erlang Solutions Ltd., Copenhagen > > > > On Aug 13, 2012, at 7:52 PM, Zabrane Mickael wrote: > >> Hi guys, >> >> I'd like to know if the binary returned by "binary:part/2/3" (http://www.erlang.org/doc/man/binary.html#part-2) >> is always referencing Subject (i.e a sub-binary) or not? >> >> The doc is clear about this fact for "binary:split/2/3" but nothing is mentionned for "binary:part/2/3". >> >> Regards, >> Zabrane >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From zabrane3@REDACTED Tue Aug 14 12:32:25 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 14 Aug 2012 12:32:25 +0200 Subject: [erlang-questions] http_uri_parse/1 inconsistent return value + dialyser headache Message-ID: Hi guys, Hi Kostis, http_uri_parse/1 returns inconsistent value since 15RB. I'm trying to build a wrapper around it to make my code works on = {ok, tuple()} | {error, atom()}. http_uri_parse(URL) when is_list(URL) -> case http_uri:parse(URL) of {ok,{Scheme, UserInfo, Host, Port, Path, Query}} -> %% >=R15B http_uri_parse_1(Scheme, UserInfo, Host, Port, Path, Query); {Scheme, UserInfo, Host, Port, Path, Query} -> %% =< R14B <------- LINE 1106 http_uri_parse_1(Scheme, UserInfo, Host, Port, Path, Query); Error -> Error end. http_uri_parse_1(Scheme, UserInfo, Host, Port, Path, Query) -> {ok, {Scheme, UserInfo, Host, Port, Path, Query}}. But Dilayzer isn't happy: Checking whether the PLT /Users/younes/.otp_plt is up-to-date... yes Proceeding with analysis... pimco.erl:1106: The pattern {Scheme, UserInfo, Host, Port, Path, Query} can never match the type {'error','no_scheme' | {_,atom(),_}} Any hint/help to get rid of this warning? Regards Zabrane From zabrane3@REDACTED Tue Aug 14 12:41:31 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 14 Aug 2012 12:41:31 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.6.1 In-Reply-To: <50228834.9000601@ninenines.eu> References: <502254C6.1040407@ninenines.eu> <3F3CC0C2-B801-48D9-BA35-B2D0693E89EE@gmail.com> <5022745A.2080302@ninenines.eu> <50228834.9000601@ninenines.eu> Message-ID: <41123609-B62E-44C9-BCD5-7FCA2BE934FC@gmail.com> Hi Lo?c, Compiling lastest Cowboy from github with R15B01 and running Dialyzer, I got this: cowboy $ make dialyze Checking whether the PLT .cowboy.plt is up-to-date... yes Proceeding with analysis... cowboy_http_protocol.erl:36: Callback info about the cowboy_protocol behaviour is not available Did you noticed that? Regards, Zabrane From desired.mta@REDACTED Tue Aug 14 12:46:28 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Tue, 14 Aug 2012 13:46:28 +0300 Subject: [erlang-questions] http_uri_parse/1 inconsistent return value + dialyser headache In-Reply-To: References: Message-ID: On Tue, Aug 14, 2012 at 1:32 PM, Zabrane Mickael wrote: > Hi guys, Hi Kostis, > > http_uri_parse/1 returns inconsistent value since 15RB. > I'm trying to build a wrapper around it to make my code works on = > Here's what i got so far: > > -spec http_uri_parse([byte()]) -> {ok, tuple()} | {error, atom()}. > http_uri_parse(URL) when is_list(URL) -> > case http_uri:parse(URL) of > {ok,{Scheme, UserInfo, Host, Port, Path, Query}} -> %% >=R15B > http_uri_parse_1(Scheme, UserInfo, Host, Port, Path, Query); > {Scheme, UserInfo, Host, Port, Path, Query} -> %% =< R14B <------- LINE 1106 > http_uri_parse_1(Scheme, UserInfo, Host, Port, Path, Query); > Error -> > Error > end. > > http_uri_parse_1(Scheme, UserInfo, Host, Port, Path, Query) -> > {ok, {Scheme, UserInfo, Host, Port, Path, Query}}. > > But Dilayzer isn't happy: > > Checking whether the PLT /Users/younes/.otp_plt is up-to-date... yes > Proceeding with analysis... > pimco.erl:1106: The pattern {Scheme, UserInfo, Host, Port, Path, Query} can never match the type {'error','no_scheme' | {_,atom(),_}} > > > Any hint/help to get rid of this warning? One way would be to define a constant at compile-time which version of Erlang are you running. Then check for it in source and put the right function/case statement accordingly. rebar -> erl_opts -> platform_define is one of the ways to define that constant. -- Motiejus Jak?tys From essen@REDACTED Tue Aug 14 12:48:23 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 14 Aug 2012 12:48:23 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.6.1 In-Reply-To: <41123609-B62E-44C9-BCD5-7FCA2BE934FC@gmail.com> References: <502254C6.1040407@ninenines.eu> <3F3CC0C2-B801-48D9-BA35-B2D0693E89EE@gmail.com> <5022745A.2080302@ninenines.eu> <50228834.9000601@ninenines.eu> <41123609-B62E-44C9-BCD5-7FCA2BE934FC@gmail.com> Message-ID: <502A2CF7.7080701@ninenines.eu> That's something that got added to R15, but if you use it it can't compile on R14, so since we are dropping R14 in the next version we simply waited before adding the callback specs. On 08/14/2012 12:41 PM, Zabrane Mickael wrote: > Hi Lo?c, > > Compiling lastest Cowboy from github with R15B01 and running Dialyzer, I got this: > > cowboy $ make dialyze > Checking whether the PLT .cowboy.plt is up-to-date... yes > Proceeding with analysis... > cowboy_http_protocol.erl:36: Callback info about the cowboy_protocol behaviour is not available > > > Did you noticed that? > > Regards, > Zabrane > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From desired.mta@REDACTED Tue Aug 14 12:48:49 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Tue, 14 Aug 2012 13:48:49 +0300 Subject: [erlang-questions] [ANN] Cowboy 0.6.1 In-Reply-To: <41123609-B62E-44C9-BCD5-7FCA2BE934FC@gmail.com> References: <502254C6.1040407@ninenines.eu> <3F3CC0C2-B801-48D9-BA35-B2D0693E89EE@gmail.com> <5022745A.2080302@ninenines.eu> <50228834.9000601@ninenines.eu> <41123609-B62E-44C9-BCD5-7FCA2BE934FC@gmail.com> Message-ID: On Tue, Aug 14, 2012 at 1:41 PM, Zabrane Mickael wrote: > Hi Lo?c, > > Compiling lastest Cowboy from github with R15B01 and running Dialyzer, I got this: > > cowboy $ make dialyze > Checking whether the PLT .cowboy.plt is up-to-date... yes > Proceeding with analysis... > cowboy_http_protocol.erl:36: Callback info about the cowboy_protocol behaviour is not available > > > Did you noticed that? This email explains it: http://erlang.org/pipermail/erlang-questions/2012-June/067428.html -- Motiejus Jak?tys From zabrane3@REDACTED Tue Aug 14 12:56:44 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 14 Aug 2012 12:56:44 +0200 Subject: [erlang-questions] http_uri_parse/1 inconsistent return value + dialyser headache In-Reply-To: References: Message-ID: <1E2A89D8-33C4-4D46-A9C7-42AF4EC12A3C@gmail.com> Thanks Motiejus, Yep, that's a way to solve it. Anything else? Regards, Zabrane On Aug 14, 2012, at 12:46 PM, Motiejus Jak?tys wrote: > On Tue, Aug 14, 2012 at 1:32 PM, Zabrane Mickael wrote: >> Hi guys, Hi Kostis, >> >> http_uri_parse/1 returns inconsistent value since 15RB. >> I'm trying to build a wrapper around it to make my code works on => >> Here's what i got so far: >> >> -spec http_uri_parse([byte()]) -> {ok, tuple()} | {error, atom()}. >> http_uri_parse(URL) when is_list(URL) -> >> case http_uri:parse(URL) of >> {ok,{Scheme, UserInfo, Host, Port, Path, Query}} -> %% >=R15B >> http_uri_parse_1(Scheme, UserInfo, Host, Port, Path, Query); >> {Scheme, UserInfo, Host, Port, Path, Query} -> %% =< R14B <------- LINE 1106 >> http_uri_parse_1(Scheme, UserInfo, Host, Port, Path, Query); >> Error -> >> Error >> end. >> >> http_uri_parse_1(Scheme, UserInfo, Host, Port, Path, Query) -> >> {ok, {Scheme, UserInfo, Host, Port, Path, Query}}. >> >> But Dilayzer isn't happy: >> >> Checking whether the PLT /Users/younes/.otp_plt is up-to-date... yes >> Proceeding with analysis... >> pimco.erl:1106: The pattern {Scheme, UserInfo, Host, Port, Path, Query} can never match the type {'error','no_scheme' | {_,atom(),_}} >> >> >> Any hint/help to get rid of this warning? > > One way would be to define a constant at compile-time which version of > Erlang are you running. Then check for it in source and put the right > function/case statement accordingly. > > rebar -> erl_opts -> platform_define is one of the ways to define that constant. > > -- > Motiejus Jak?tys From magnus.henoch@REDACTED Tue Aug 14 13:04:04 2012 From: magnus.henoch@REDACTED (Magnus Henoch) Date: Tue, 14 Aug 2012 12:04:04 +0100 Subject: [erlang-questions] http_uri_parse/1 inconsistent return value + dialyser headache In-Reply-To: (Zabrane Mickael's message of "Tue, 14 Aug 2012 12:32:25 +0200") References: Message-ID: Zabrane Mickael writes: > http_uri_parse/1 returns inconsistent value since 15RB. > I'm trying to build a wrapper around it to make my code works on = > Here's what i got so far: > > -spec http_uri_parse([byte()]) -> {ok, tuple()} | {error, atom()}. > http_uri_parse(URL) when is_list(URL) -> > case http_uri:parse(URL) of > {ok,{Scheme, UserInfo, Host, Port, Path, Query}} -> %% >=R15B > http_uri_parse_1(Scheme, UserInfo, Host, Port, Path, Query); > {Scheme, UserInfo, Host, Port, Path, Query} -> %% =< R14B <------- LINE 1106 > http_uri_parse_1(Scheme, UserInfo, Host, Port, Path, Query); > Error -> > Error > end. > > http_uri_parse_1(Scheme, UserInfo, Host, Port, Path, Query) -> > {ok, {Scheme, UserInfo, Host, Port, Path, Query}}. > > But Dilayzer isn't happy: > > Checking whether the PLT /Users/younes/.otp_plt is up-to-date... yes > Proceeding with analysis... > pimco.erl:1106: The pattern {Scheme, UserInfo, Host, Port, Path, Query} can never match the type {'error','no_scheme' | {_,atom(),_}} > > > Any hint/help to get rid of this warning? You've already been given the nice way to do it, so let me show the hackish way to get rid of the warning. Dialyzer can't infer anything about what a 'catch' expression returns, so if you write this instead: case catch http_uri:parse(URL) of then Dialyzer won't complain. An advanced variant of that is to define a ?catch_for_dialyzer macro that expands to nothing for normal compilation, and to 'catch' when running Dialyzer. Regards, Magnus From desired.mta@REDACTED Tue Aug 14 13:23:20 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Tue, 14 Aug 2012 14:23:20 +0300 Subject: [erlang-questions] http_uri_parse/1 inconsistent return value + dialyser headache In-Reply-To: References: Message-ID: On Tue, Aug 14, 2012 at 2:04 PM, Magnus Henoch wrote: > Zabrane Mickael writes: > > An advanced variant of that is to define a ?catch_for_dialyzer macro > that expands to nothing for normal compilation, and to 'catch' when > running Dialyzer. Interesting. How do you know when you are running in dialyzer (i.e. how would you define that macro)? -- Motiejus Jak?tys From magnus.henoch@REDACTED Tue Aug 14 14:06:06 2012 From: magnus.henoch@REDACTED (Magnus Henoch) Date: Tue, 14 Aug 2012 13:06:06 +0100 Subject: [erlang-questions] http_uri_parse/1 inconsistent return value + dialyser headache In-Reply-To: ("Motiejus \=\?utf-8\?Q\?Jak\=C5\=A1tys\=22's\?\= message of "Tue, 14 Aug 2012 14:23:20 +0300") References: Message-ID: Motiejus Jak?tys writes: > On Tue, Aug 14, 2012 at 2:04 PM, Magnus Henoch > wrote: >> Zabrane Mickael writes: >> >> An advanced variant of that is to define a ?catch_for_dialyzer macro >> that expands to nothing for normal compilation, and to 'catch' when >> running Dialyzer. > > Interesting. How do you know when you are running in dialyzer (i.e. > how would you define that macro)? The only way I know is to have your build system pass different -D flags for normal compilation and dialyzer. Regards, Magnus From zabrane3@REDACTED Tue Aug 14 14:38:19 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 14 Aug 2012 14:38:19 +0200 Subject: [erlang-questions] http_uri_parse/1 inconsistent return value + dialyser headache In-Reply-To: References: Message-ID: <3D598640-B0C3-473A-9D2B-E19E7F4CD27E@gmail.com> Following Magnus & Motiejus great suggestions, here it is: %-------- put this marco somewhere (.hrl, top of your .erl) -ifdef(DIALYZER_CATCH). -define(CATCH_FOR_DIALYZER(X), catch X). -else. -define(CATCH_FOR_DIALYZER(X), X). -endif. %------- the new code -spec http_uri_parse([byte()]) -> {ok, tuple()} | {error, atom()}. http_uri_parse(URL) -> case ?CATCH_FOR_DIALYZER(http_uri:parse(URL)) of {ok,{_, _, _, _, _, _}} = Ret15 -> %% >=R15B Ret15; {_, _, _, _, _, _} = Ret14 -> %% =< R14B {ok, Ret14}; Error -> Error end. Add this flag "-DDIALYZER_CATCH=1" to your Dialyzer command line: $ dialyzer --src src --plt /path ... -DDIALYZER_CATCH=1 Everything's perfect now. Thanks for sharing these guys. You saved my day. Regards, Zabrane On Aug 14, 2012, at 2:06 PM, Magnus Henoch wrote: > Motiejus Jak?tys writes: > >> On Tue, Aug 14, 2012 at 2:04 PM, Magnus Henoch >> wrote: >>> Zabrane Mickael writes: >>> >>> An advanced variant of that is to define a ?catch_for_dialyzer macro >>> that expands to nothing for normal compilation, and to 'catch' when >>> running Dialyzer. >> >> Interesting. How do you know when you are running in dialyzer (i.e. >> how would you define that macro)? > > The only way I know is to have your build system pass different -D flags > for normal compilation and dialyzer. > > Regards, > Magnus From mononcqc@REDACTED Tue Aug 14 14:40:55 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 14 Aug 2012 08:40:55 -0400 Subject: [erlang-questions] Parametric modules and Dialyzer In-Reply-To: References: Message-ID: <502A4757.50101@ferd.ca> I really, really want to recommend option 5: not using parametrized modules. Dialyzer is the first of your concerns. Next up, you'll be having trouble with code tracing, which is also messed up due to the arity issues. Then what about function guards? There are a few valid use cases for parametrized modules, and Richard Carlsson can demonstrate them well, but I would generally advise against it (a lot of it is personal preferences, I got to admit). Shadowing, messed up arities, different semantics, etc. all annoy me a bit. Here's a quick list of the problems, based on the following module: -module(pmod,[Arg]). -compile(export_all). f() -> Arg. Using it in a shell started as: 1> c(pmod). {ok,pmod} 2> M = pmod:new(hello). {pmod,hello} 1. function building is broken: 3> fun M:f/0. ** exception error: bad argument in function erlang:make_fun/3 called as erlang:make_fun({pmod,hello},f,0) You can't get the above to work without either forgetting about parametrized modules (passing the argument to the function) or wrapping it in an anonymous function as fun() -> M:f() end, although this one has the problem of potentially being purged during code reload, which a proper pmod anonymous function wouldn't. 2. Tracing is broken due to changes in arity: 4> dbg:tp({M, f, 0}, []). ** exception error: no case clause matching {pmod,hello} in function dbg:do_tp/3 (dbg.erl, line 141) 5> dbg:tp({pmod, f, 0}, []). {ok,[]} 6> dbg:tp({pmod, f, 1}, []). {ok,[]} The first one plainly doesn't work, the second one will never match anything, and the third one is actually the valid one. 3. Tricky pattern matching. If I add the following function definition to the pmod module: id(Arg) -> Arg. And then try to use it, see what can happen: 7> c(pmod). {ok,pmod} 8> pmod:id(4). ** exception error: undefined function pmod:id/1 9> M:id(4). ** exception error: no function clause matching pmod:id(4,{pmod,hello}) (pmod.erl, line 6) 10> M:id(hello). hello The way you name the variable will impact pattern matching. Although it *is* likely the best behaviour available, it is weird to have this behaviour happening, you need to watch yourself. This kind of stuff won't happen if you use pmods with great care, but I'm a pessimist on these questions. Related to this, this isn't the most intuitive kind of error to debug in the middle of a larger file. If you add: y() -> fun(Arg) -> Arg end. You get compiler warnings telling you /pmod.erl:8: Warning: variable 'Arg' shadowed in 'fun'/, still a scoping issue for the distracted programmer. Imagine using the same variable in a case ... of match without thinking of it. Some programmers get it wrong with the normal scoping rules, nevermind with parametrized arguments on top of it. 4. They don't work well with tools like Dialyzer, as you found out. Those are the ones I got from the top of my head. I know people will defend Parametrized modules, that they have valid use cases (I can admit to a few!), that a lot of these issues are implementation details that can be fixed, but as of now, they aren't, and they mess stuff up. Sorry, I know this isn't the kind of answer you want to such a question. On 12-08-14 4:41 AM, VegaS wrote: > Hello all! > > We are using parametric modules in our project and were quite happy > with them unless we have started using Dialyzer. > Dialyzer fails on parametric modules because functions in parametric > modules are expanded with module params argument on compilation stage > - so function's specs are unmatched. > > I know that parametric modules not the official part of Erlang, but > I'd like to get both parametric modules and dialyzer working. I have > tried to do some work on this: > 1. Try to fix specs on parametric module's functions with > parse_transform - simply extend specs on this stage, but this won't > work, code just won't compile (because of specs for unknown functions) > 2. More complicated approach - try to implement parametric modules > with parse_transform (create "new" function and extend all module > functions and specs). This looks working unless some code like > SomeFun = local_fun/1, SomeFun(Arg) and some variations like this > local function is assigned to record field. > 3. Decided that there no way out besides patching OTP sources or not > using parametric modules at all. After looking at OTP sources I have > found, that parametric modules are expanded on on "expand_module" > compiler pass. And on that stage specs are not expanded. I Have > patched this stage with specs expand. All were correct, but dialyzer > fails with message like specs were not fixed, so no luck... This > happend because dialyzer using "abstract_code" from beam to extract > specs and pass "save_abstract_code" is just before "expand_module" pass. > Simple changing these passes order broke dialyzer at all. > 4. The last attempt were to patch dialyzer - on the stage, when it > extracts specs info from beam's abstract_code section. At this place I > insert function's specs expanding with last parametric parameter - and > it works! > > So the question is what the rigth way to make both parametric modules > and dialyzer working? > At my opinion there are no way except OTP sources patching - but what > the rigth place for making such patch? > > I can attach my patch here but it is little ugly (I were tired by > previous explorations) and I think I have chosen not rigth place for > changing. > > Thanks, > Lihanov Alexandr > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Tue Aug 14 14:41:32 2012 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Tue, 14 Aug 2012 14:41:32 +0200 Subject: [erlang-questions] Base64 decode binaries with ets bug? In-Reply-To: References: <747EA411-ECF2-48FF-A3FF-D908C7DE4EB2@gmail.com> Message-ID: <502A477C.401@erix.ericsson.se> Thanks for reporting, Valentino. It is indeed an ETS bug that can corrupt binaries in compressed tables. Below is a patch scheduled for R15B02. diff --git a/erts/emulator/beam/external.c b/erts/emulator/beam/external.c index 4348578..52f45b9 100644 --- a/erts/emulator/beam/external.c +++ b/erts/emulator/beam/external.c @@ -1889,7 +1889,9 @@ enc_term(ErtsAtomCacheMap *acmp, Eterm obj, byte* ep, Uint32 dflags, *ep++ = BINARY_INTERNAL_REF; } if (pb->flags) { + char* before_realloc = pb->val->orig_bytes; erts_emasculate_writable_binary(pb); + bytes += (pb->val->orig_bytes - before_realloc); } erts_refc_inc(&pb->val->refc, 2); /Sverker, Erlang/OTP From tyron.zerafa@REDACTED Tue Aug 14 16:56:19 2012 From: tyron.zerafa@REDACTED (Tyron Zerafa) Date: Tue, 14 Aug 2012 16:56:19 +0200 Subject: [erlang-questions] Copy function from one Module to another Message-ID: Hey all, Is there any mechanism in Erlang which I can use to copy functions from one module to another? So, if I have module *A* containing functions *a*, *b * and *c*, is there any way in which I can copy *a* to module B? Thanks Tyron -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Tue Aug 14 17:10:39 2012 From: gumm@REDACTED (Jesse Gumm) Date: Tue, 14 Aug 2012 10:10:39 -0500 Subject: [erlang-questions] Copy function from one Module to another In-Reply-To: References: Message-ID: That sounds like you're looking for -import(Module, Functions). http://www.erlang.org/doc/reference_manual/modules.html -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm On Aug 14, 2012 9:56 AM, "Tyron Zerafa" wrote: > Hey all, > Is there any mechanism in Erlang which I can use to copy functions > from one module to another? So, if I have module *A* containing functions > *a*, *b* and *c*, is there any way in which I can copy *a* to module B? > > Thanks > Tyron > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyron.zerafa@REDACTED Tue Aug 14 17:15:15 2012 From: tyron.zerafa@REDACTED (Tyron Zerafa) Date: Tue, 14 Aug 2012 17:15:15 +0200 Subject: [erlang-questions] Copy function from one Module to another In-Reply-To: References: Message-ID: Hey Jesse I need something stronger than just import. I need to put a bunch of functions from different modules into a single one and transfer this to a remote node. Then I want to be able to use these functions from the remote node. If I simply use import, I will not be able to use them on the remote node without explicitly transferring all the modules to which the functions belong to. On Tue, Aug 14, 2012 at 5:10 PM, Jesse Gumm wrote: > That sounds like you're looking for > > -import(Module, Functions). > > http://www.erlang.org/doc/reference_manual/modules.html > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866 || sigma-star.com || @jessegumm > On Aug 14, 2012 9:56 AM, "Tyron Zerafa" wrote: > >> Hey all, >> Is there any mechanism in Erlang which I can use to copy functions >> from one module to another? So, if I have module *A* containing >> functions *a*, *b* and *c*, is there any way in which I can copy *a* to >> module B? >> >> Thanks >> Tyron >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> -- Best Regards, Tyron Zerafa -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Tue Aug 14 17:25:30 2012 From: siraaj@REDACTED (Siraaj Khandkar) Date: Tue, 14 Aug 2012 11:25:30 -0400 Subject: [erlang-questions] Copy function from one Module to another In-Reply-To: References: Message-ID: How about sending a message with a fun? Or collecting desired funs in a record and sending that? On Tue, Aug 14, 2012 at 11:15 AM, Tyron Zerafa wrote: > > Hey Jesse > I need something stronger than just import. I need to put a bunch of > functions from different modules into a single one and transfer this to a > remote node. Then I want to be able to use these functions from the remote > node. > If I simply use import, I will not be able to use them on the remote node > without explicitly transferring all the modules to which the functions > belong to. > > > On Tue, Aug 14, 2012 at 5:10 PM, Jesse Gumm wrote: >> >> That sounds like you're looking for >> >> -import(Module, Functions). >> >> http://www.erlang.org/doc/reference_manual/modules.html >> >> -- >> Jesse Gumm >> Owner, Sigma Star Systems >> 414.940.4866 || sigma-star.com || @jessegumm >> >> On Aug 14, 2012 9:56 AM, "Tyron Zerafa" wrote: >>> >>> Hey all, >>> Is there any mechanism in Erlang which I can use to copy functions >>> from one module to another? So, if I have module A containing functions a, b >>> and c, is there any way in which I can copy a to module B? From tyron.zerafa@REDACTED Tue Aug 14 17:28:32 2012 From: tyron.zerafa@REDACTED (Tyron Zerafa) Date: Tue, 14 Aug 2012 17:28:32 +0200 Subject: [erlang-questions] Copy function from one Module to another In-Reply-To: References: Message-ID: I was thinking about the latter where I will collect all functions in a record and send that. The problem is that I will have to come up with a way to extract a function form a module, that is why I was checking out whether there is anything already implemented in Erlang. On Tue, Aug 14, 2012 at 5:25 PM, Siraaj Khandkar wrote: > How about sending a message with a fun? Or collecting desired funs in > a record and sending that? > > > On Tue, Aug 14, 2012 at 11:15 AM, Tyron Zerafa > wrote: > > > > Hey Jesse > > I need something stronger than just import. I need to put a bunch of > > functions from different modules into a single one and transfer this to a > > remote node. Then I want to be able to use these functions from the > remote > > node. > > If I simply use import, I will not be able to use them on the remote node > > without explicitly transferring all the modules to which the functions > > belong to. > > > > > > On Tue, Aug 14, 2012 at 5:10 PM, Jesse Gumm wrote: > >> > >> That sounds like you're looking for > >> > >> -import(Module, Functions). > >> > >> http://www.erlang.org/doc/reference_manual/modules.html > >> > >> -- > >> Jesse Gumm > >> Owner, Sigma Star Systems > >> 414.940.4866 || sigma-star.com || @jessegumm > >> > >> On Aug 14, 2012 9:56 AM, "Tyron Zerafa" wrote: > >>> > >>> Hey all, > >>> Is there any mechanism in Erlang which I can use to copy functions > >>> from one module to another? So, if I have module A containing > functions a, b > >>> and c, is there any way in which I can copy a to module B? > -- Best Regards, Tyron Zerafa -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Tue Aug 14 17:35:24 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 14 Aug 2012 17:35:24 +0200 Subject: [erlang-questions] Password generator in Erlang Message-ID: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> Hi guys, This one fires a lot of collisions (http://schemecookbook.org/Erlang/NumberRandomNumber): ------------------ test() -> crypto:start(), N = 10000, io:format("Generate ~p random passwords and check for collisions ...~n", [N]), test(N, dict:new(), 0). test(0, _, C) -> io:format("Number of collisions: ~p~n", [C]); test(N, Dict, C) -> Password = passwd(), case dict:find(Password, Dict) of {ok, _} -> %% collision detected test(N - 1, Dict, C + 1); error -> test(N - 1, dict:append(Password, 1, Dict), C) end. passwd() -> passwd(8). passwd(Length) -> {A1,A2,A3} = now(), random:seed(A1, A2, A3), lists:flatten(lists:foldl(fun(_,AccIn) -> [random:uniform(90) + 32 | AccIn] end, [], lists:seq(1,Length))). ---------------- > passwd:test(). Generate 10000 random password and check for collisions ... Number of collisions: 1182 Anything better guys? Something with less collisions if possible. Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Tue Aug 14 17:43:24 2012 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Tue, 14 Aug 2012 17:43:24 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> Message-ID: <502A721C.1020308@erix.ericsson.se> Try only call random:seed(A1, A2, A3) once at start. For cryptographic safe random generation look at crypto module. /Sverker Zabrane Mickael wrote: > Hi guys, > > This one fires a lot of collisions > (http://schemecookbook.org/Erlang/NumberRandomNumber): > > ------------------ > test() -> > crypto:start(), > N = 10000, > io:format("Generate ~p random passwords and check for collisions ...~n", [N]), > test(N, dict:new(), 0). > test(0, _, C) -> > io:format("Number of collisions: ~p~n", [C]); > test(N, Dict, C) -> > Password = passwd(), > case dict:find(Password, Dict) of > {ok, _} -> %% collision detected > test(N - 1, Dict, C + 1); > error -> > test(N - 1, dict:append(Password, 1, Dict), C) > end. > > passwd() -> > passwd(8). > passwd(Length) -> > {A1,A2,A3} = now(), > random:seed(A1, A2, A3), > lists:flatten(lists:foldl(fun(_,AccIn) -> > [random:uniform(90) + 32 | AccIn] end, > [], lists:seq(1,Length))). > > ---------------- > > > passwd:test(). > Generate 10000 random password and check for collisions ... > Number of collisions: 1182 > > Anything better guys? Something with less collisions if possible. > > Regards, > Zabrane > > From essen@REDACTED Tue Aug 14 17:40:45 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 14 Aug 2012 17:40:45 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> Message-ID: <502A717D.5060702@ninenines.eu> os:cmd("pwgen | cut -c 1-8"). On 08/14/2012 05:35 PM, Zabrane Mickael wrote: > Hi guys, > > This one fires a lot of collisions > (http://schemecookbook.org/Erlang/NumberRandomNumber): > > ------------------ > test() -> > crypto:start(), > N = 10000, > io:format("Generate ~p random passwords and check for collisions > ...~n", [N]), > test(N, dict:new(), 0). > test(0, _, C) -> > io:format("Number of collisions: ~p~n", [C]); > test(N, Dict, C) -> > Password = passwd(), > case dict:find(Password, Dict) of > {ok, _} -> %% collision detected > test(N - 1, Dict, C + 1); > error -> > test(N - 1, dict:append(Password, 1, Dict), C) > end. > > passwd() -> > passwd(8). > passwd(Length) -> > {A1,A2,A3} = now(), > random:seed(A1, A2, A3), > lists:flatten(lists:foldl(fun(_,AccIn) -> > [random:uniform(90) + 32 | AccIn] > end, > [], lists:seq(1,Length))). > > ---------------- > > > passwd:test(). > Generate 10000 random password and check for collisions ... > Number of collisions: 1182 > > Anything better guys? Something with less collisions if possible. > > Regards, > Zabrane > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From mononcqc@REDACTED Tue Aug 14 18:09:38 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 14 Aug 2012 12:09:38 -0400 Subject: [erlang-questions] Copy function from one Module to another In-Reply-To: References: Message-ID: <502A7842.9010408@ferd.ca> Why do you want to do this, instead of, for example, just sending the module to the other node? {Mod, Bin, File} = code:get_object_code(Mod), rpc:call(Node, code, load_binary, [Mod, File, Bin]). or alternatively use rpc:multicall(Nodes) for many nodes and doing the same. On 12-08-14 11:15 AM, Tyron Zerafa wrote: > > Hey Jesse > I need something stronger than just import. I need to put a bunch > of functions from different modules into a single one and transfer > this to a remote node. Then I want to be able to use these functions > from the remote node. > If I simply use import, I will not be able to use them on the remote > node without explicitly transferring all the modules to which the > functions belong to. > > > On Tue, Aug 14, 2012 at 5:10 PM, Jesse Gumm > wrote: > > That sounds like you're looking for > > -import(Module, Functions). > > http://www.erlang.org/doc/reference_manual/modules.html > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866 || sigma-star.com > || @jessegumm > > On Aug 14, 2012 9:56 AM, "Tyron Zerafa" > wrote: > > Hey all, > Is there any mechanism in Erlang which I can use to copy > functions from one module to another? So, if I have module *A* > containing functions *a*, *b* and *c*, is there any way in > which I can copy *a* to module B? > > Thanks > Tyron > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Best Regards, > Tyron Zerafa > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Tue Aug 14 18:09:31 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 14 Aug 2012 18:09:31 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <502A717D.5060702@ninenines.eu> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A717D.5060702@ninenines.eu> Message-ID: On Aug 14, 2012, at 5:40 PM, Lo?c Hoguin wrote: > os:cmd("pwgen | cut -c 1-8"). a portable one ;-) From zabrane3@REDACTED Tue Aug 14 18:12:14 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 14 Aug 2012 18:12:14 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <502A721C.1020308@erix.ericsson.se> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> Message-ID: <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> Thank Serker. Moving andom:seed(A1, A2, A3) at start fix it. 1> passwd:test(). Generate 10000 random password and check for collisions ... Number of collisions: 0 -------------------- test() -> {A1,A2,A3} = now(), random:seed(A1, A2, A3), N = 10000, io:format("Generate ~p random password and check for collisions ...~n", [N]), test(N, dict:new(), 0). test(0, _, C) -> io:format("Number of collisions: ~p~n", [C]); test(N, Dict, C) -> Password = passwd(), case dict:find(Password, Dict) of {ok, _} -> %% collision detected test(N - 1, Dict, C + 1); error -> test(N - 1, dict:append(Password, 1, Dict), C) end. passwd() -> passwd(8). passwd(Length) -> lists:flatten(lists:foldl(fun(_,AccIn) -> [random:uniform(90) + 32 | AccIn] end, [], lists:seq(1,Length))). -------------------- Regards, Zabrane On Aug 14, 2012, at 5:43 PM, Sverker Eriksson wrote: > Try only call random:seed(A1, A2, A3) once at start. > > For cryptographic safe random generation look at crypto module. > > /Sverker > > > Zabrane Mickael wrote: >> Hi guys, >> >> This one fires a lot of collisions (http://schemecookbook.org/Erlang/NumberRandomNumber): >> >> ------------------ >> test() -> >> crypto:start(), >> N = 10000, >> io:format("Generate ~p random passwords and check for collisions ...~n", [N]), >> test(N, dict:new(), 0). >> test(0, _, C) -> >> io:format("Number of collisions: ~p~n", [C]); >> test(N, Dict, C) -> >> Password = passwd(), >> case dict:find(Password, Dict) of >> {ok, _} -> %% collision detected >> test(N - 1, Dict, C + 1); error -> >> test(N - 1, dict:append(Password, 1, Dict), C) >> end. >> >> passwd() -> >> passwd(8). >> passwd(Length) -> >> {A1,A2,A3} = now(), >> random:seed(A1, A2, A3), >> lists:flatten(lists:foldl(fun(_,AccIn) -> >> [random:uniform(90) + 32 | AccIn] end, >> [], lists:seq(1,Length))). >> >> ---------------- >> >> > passwd:test(). >> Generate 10000 random password and check for collisions ... >> Number of collisions: 1182 >> >> Anything better guys? Something with less collisions if possible. >> >> Regards, >> Zabrane >> >> > From kenneth.lundin@REDACTED Tue Aug 14 18:17:45 2012 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 14 Aug 2012 18:17:45 +0200 Subject: [erlang-questions] Remote command execution via SSH ? In-Reply-To: References: Message-ID: You can use or look at the code in ct_ssh:exec/3 in the Commontest application. It does what you are looking for i.e. executes a command and returns the output as data. /Kenneth , Erlang/OTP Ericsson Den 13 aug 2012 23:54 skrev "Thomas Elsgaard" : > Hi > > Does anybody have a good simple example of executing commands on a > remote server via SSH? > > Nothing fancy, just something like: > > ------ > Connect to host > Execute command '/bin/ls' > Read output from command > Close connection > ------ > > Best regards > > Thomas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyron.zerafa@REDACTED Tue Aug 14 18:20:20 2012 From: tyron.zerafa@REDACTED (Tyron Zerafa) Date: Tue, 14 Aug 2012 18:20:20 +0200 Subject: [erlang-questions] Copy function from one Module to another In-Reply-To: <502A7842.9010408@ferd.ca> References: <502A7842.9010408@ferd.ca> Message-ID: I want to implement code migration where a user just will be able to call migrate(RemoteNode, Fun) and that local fun will end up on the RemoteNode. Thus this needs to be generic. I cannot assume that: 1) All functions within the module are going to be used - transferring code which will not be used will result in useless bandwidth costs. 2) All functions required are within a single module I was going to perform a static analysis of the function to be migrated and transfer only the functions it depends on. On Tue, Aug 14, 2012 at 6:09 PM, Fred Hebert wrote: > Why do you want to do this, instead of, for example, just sending the > module to the other node? > > {Mod, Bin, File} = code:get_object_code(Mod), > rpc:call(Node, code, load_binary, [Mod, File, Bin]). > > or alternatively use rpc:multicall(Nodes) for many nodes and doing the > same. > > > On 12-08-14 11:15 AM, Tyron Zerafa wrote: > > > Hey Jesse > I need something stronger than just import. I need to put a bunch of > functions from different modules into a single one and transfer this to a > remote node. Then I want to be able to use these functions from the remote > node. > If I simply use import, I will not be able to use them on the remote node > without explicitly transferring all the modules to which the functions > belong to. > > > On Tue, Aug 14, 2012 at 5:10 PM, Jesse Gumm wrote: > >> That sounds like you're looking for >> >> -import(Module, Functions). >> >> http://www.erlang.org/doc/reference_manual/modules.html >> >> -- >> Jesse Gumm >> Owner, Sigma Star Systems >> 414.940.4866 || sigma-star.com || @jessegumm >> On Aug 14, 2012 9:56 AM, "Tyron Zerafa" wrote: >> >>> Hey all, >>> Is there any mechanism in Erlang which I can use to copy functions >>> from one module to another? So, if I have module *A* containing >>> functions *a*, *b* and *c*, is there any way in which I can copy *a* to >>> module B? >>> >>> Thanks >>> Tyron >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> > > > -- > Best Regards, > Tyron Zerafa > > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > -- Best Regards, Tyron Zerafa -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Aug 14 18:44:23 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 14 Aug 2012 18:44:23 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A717D.5060702@ninenines.eu> Message-ID: <502A8067.7090607@ninenines.eu> On 08/14/2012 06:09 PM, Zabrane Mickael wrote: > > On Aug 14, 2012, at 5:40 PM, Lo?c Hoguin wrote: > >> os:cmd("pwgen | cut -c 1-8"). > > a portable one ;-) I believe pwgen and cut are available on all of win/linux/bsd/osx. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From nem@REDACTED Tue Aug 14 20:01:25 2012 From: nem@REDACTED (Geoff Cant) Date: Tue, 14 Aug 2012 11:01:25 -0700 Subject: [erlang-questions] Copy function from one Module to another In-Reply-To: References: <502A7842.9010408@ferd.ca> Message-ID: <7539A176-3E24-402E-920C-71834BC92569@erlang.geek.nz> You won't be able to do exactly what you have asked for as functions only exist inside modules. Even funs are just context plus a reference to code in a module. You cannot run a fun on a node that doesn't have the module it references. While I'm sure that Fred's idea is the best (you'd have to be sorely bandwidth constrained before other trade offs make sense) the middle ground would be to compile a new module with just the function + deps you want to migrate and send that to the remote node. You would probably need debug_info in the original module for this to work. In your position I would use Fred's scheme. The 3-liner beats the complicated recompilation solution 99 times out of 100. -Geoff On 14/08/2012, at 9:20, Tyron Zerafa wrote: > I want to implement code migration where a user just will be able to call migrate(RemoteNode, Fun) and that local fun will end up on the RemoteNode. > Thus this needs to be generic. I cannot assume that: > 1) All functions within the module are going to be used - transferring code which will not be used will result in useless bandwidth costs. > 2) All functions required are within a single module > > I was going to perform a static analysis of the function to be migrated and transfer only the functions it depends on. > > On Tue, Aug 14, 2012 at 6:09 PM, Fred Hebert wrote: > Why do you want to do this, instead of, for example, just sending the module to the other node? > > {Mod, Bin, File} = code:get_object_code(Mod), > rpc:call(Node, code, load_binary, [Mod, File, Bin]). > > or alternatively use rpc:multicall(Nodes) for many nodes and doing the same. > > > On 12-08-14 11:15 AM, Tyron Zerafa wrote: >> >> Hey Jesse >> I need something stronger than just import. I need to put a bunch of functions from different modules into a single one and transfer this to a remote node. Then I want to be able to use these functions from the remote node. >> If I simply use import, I will not be able to use them on the remote node without explicitly transferring all the modules to which the functions belong to. >> >> >> On Tue, Aug 14, 2012 at 5:10 PM, Jesse Gumm wrote: >> That sounds like you're looking for >> >> -import(Module, Functions). >> >> http://www.erlang.org/doc/reference_manual/modules.html >> >> -- >> Jesse Gumm >> Owner, Sigma Star Systems >> 414.940.4866 || sigma-star.com || @jessegumm >> >> On Aug 14, 2012 9:56 AM, "Tyron Zerafa" wrote: >> Hey all, >> Is there any mechanism in Erlang which I can use to copy functions from one module to another? So, if I have module A containing functions a, b and c, is there any way in which I can copy a to module B? >> >> Thanks >> Tyron >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> -- >> Best Regards, >> Tyron Zerafa >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Best Regards, > Tyron Zerafa > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ronny.meeus@REDACTED Tue Aug 14 22:10:57 2012 From: ronny.meeus@REDACTED (Ronny Meeus) Date: Tue, 14 Aug 2012 22:10:57 +0200 Subject: [erlang-questions] UDP client/server performance In-Reply-To: <20120814094019.GA21542@ioctl> References: <20120814094019.GA21542@ioctl> Message-ID: Hello thanks for the info. I'm currently playing with the procket and it is working well. I have 2 questions: - I need to run the erl as root (or use sudo to start it) or I get: 2> {ok, FD} = procket:open(53, [{protocol, udp},{type, dgram},{family, inet}]). ** exception error: no match of right hand side value {error,eperm} - How do I use recvfrom/4 in the server? If I call recvfrom/4 when there is no message present in the socket, it returns an error since the socket is put in async mode. Does this mean that the socket needs to be polled by the Erlang process or is it possible to get a notification (msg) when there is data present in the socket so that the recvfrom/4 can be called at that moment? Thanks. Ronny On Tue, Aug 14, 2012 at 11:40 AM, Michael Santos wrote: > On Tue, Aug 14, 2012 at 08:25:23AM +0200, Ronny Meeus wrote: >> Is there any code available that implements a NIF to have access to UDP sockets? >> I have found the procket implementation but it looks like it is using >> a separate process to communicate with the socket. This sounds like >> overkill to me for this test. > > You can get the socket directly using procket:socket/3. See > procket:sendto/4 and procket:recvfrom/4 as well for retrieving the > struct sockaddr for the peer. > > -module(udp_srv). > -export([s/0]). > > -define(UINT16(N), N:2/native-unsigned-integer-unit:8). > -define(PF_INET, 2). > -define(PORT, 6001). > > s() -> > {ok,FD} = procket:socket(inet, dgram, 0), > % sockaddr for Linux > Sockaddr = <>, > ok = procket:bind(FD, Sockaddr), > Ref = erlang:open_port({fd, FD, FD}, [stream, binary]), > loop(Ref). > > loop(Ref) -> > receive > {Ref,{data,Data}} -> > error_logger:info_report(Data), > loop(Ref) > end. From zabrane3@REDACTED Tue Aug 14 22:26:59 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 14 Aug 2012 22:26:59 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <502A8067.7090607@ninenines.eu> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A717D.5060702@ninenines.eu> <502A8067.7090607@ninenines.eu> Message-ID: <8E398555-6A1C-4889-9D2B-E6C10037FAC1@gmail.com> Thanks for the info Lo?c. > I believe pwgen and cut are available on all of win/linux/bsd/osx. Regards, Zabrane From comptekki@REDACTED Tue Aug 14 22:34:08 2012 From: comptekki@REDACTED (Wes James) Date: Tue, 14 Aug 2012 14:34:08 -0600 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <8E398555-6A1C-4889-9D2B-E6C10037FAC1@gmail.com> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A717D.5060702@ninenines.eu> <502A8067.7090607@ninenines.eu> <8E398555-6A1C-4889-9D2B-E6C10037FAC1@gmail.com> Message-ID: <48B08481-51E7-4D88-B0CE-46EE5BC11889@gmail.com> Pwgen/cut are not on stock win 7 pro. Maybe add with Cygwin. Wes On Aug 14, 2012, at 14:26, Zabrane Mickael wrote: > Thanks for the info Lo?c. > >> I believe pwgen and cut are available on all of win/linux/bsd/osx. > > Regards, > Zabrane > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Wed Aug 15 01:42:06 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 15 Aug 2012 11:42:06 +1200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <502A8067.7090607@ninenines.eu> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A717D.5060702@ninenines.eu> <502A8067.7090607@ninenines.eu> Message-ID: <21F42903-8185-4E78-B419-9DA9512EA294@cs.otago.ac.nz> On 15/08/2012, at 4:44 AM, Lo?c Hoguin wrote: > On 08/14/2012 06:09 PM, Zabrane Mickael wrote: >> >> On Aug 14, 2012, at 5:40 PM, Lo?c Hoguin wrote: >> >>> os:cmd("pwgen | cut -c 1-8"). >> >> a portable one ;-) > > I believe pwgen and cut are available on all of win/linux/bsd/osx. m% man pwgen No manual entry for pwgen Mac OS X 10.6.8. v% man pwgen No manual entry for pwgen Linux 2.6.31.5-127.fc12.x86_64 f% man pwgen No manual entry for pwgen. Solaris. Available? May be, I don't know. Installed? NO. From ok@REDACTED Wed Aug 15 01:48:05 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 15 Aug 2012 11:48:05 +1200 Subject: [erlang-questions] Copy function from one Module to another In-Reply-To: References: Message-ID: <0BB86535-7CE8-4A64-B6A9-D90D27861EE4@cs.otago.ac.nz> On 15/08/2012, at 3:15 AM, Tyron Zerafa wrote: > > Hey Jesse > I need something stronger than just import. I need to put a bunch of functions from different modules into a single one and transfer this to a remote node. Then I want to be able to use these functions from the remote node. Never mind a remote node, I just can't see taking a random bunch of functions from different modules and stuffing them into another module making any kind of sense. Consider: -module(a). .. f() -> g(). g() -> 12. -module(b). .. h() -> g(). g() -> 34. Now, "put" a:f/0 and b/h:0 into a new module c. What happens to g? The only way I can even begin to make sense of this is to construct the new module thus: -module(c). -export([f/0,h/0]). f() -> a:f(). h() -> b:h(). and then to provide *all* of the modules. From essen@REDACTED Wed Aug 15 01:49:07 2012 From: essen@REDACTED (essen@REDACTED) Date: Wed, 15 Aug 2012 01:49:07 +0200 Subject: [erlang-questions] =?utf-8?q?Re=C2=A0=3A_Re=3A__Password_generato?= =?utf-8?q?r_in_Erlang?= In-Reply-To: <33360A3E-545E-4AA8-AF6E-C38A21426ED9@jtendo.com> Message-ID: <0MITil-1T04DN0VN1-00490h@mrelayeu.kundenserver.de> An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Aug 15 01:58:17 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 15 Aug 2012 11:58:17 +1200 Subject: [erlang-questions] =?iso-8859-1?q?Re=A0=3A_Re=3A__Password_genera?= =?iso-8859-1?q?tor_in_Erlang?= In-Reply-To: <0MITil-1T04DN0VN1-00490h@mrelayeu.kundenserver.de> References: <0MITil-1T04DN0VN1-00490h@mrelayeu.kundenserver.de> Message-ID: <7BDB8F8F-203B-48A3-9056-CC90C2F645E8@cs.otago.ac.nz> The (English version of) pwgen has been translated to Javascript: http://8-p.info/pwgen/pwgen.js It isn't very long. In fact it's a pretty simple algorithm. [Waving hands vigorously] It just threads a random path through a finite state automaton. Translating it to Erlang should be easy for anyone who wants to. From zabrane3@REDACTED Wed Aug 15 02:41:54 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 15 Aug 2012 02:41:54 +0200 Subject: [erlang-questions] [ANN] Erlang Bindings for Xapian In-Reply-To: References: <1D24E487-3655-45BE-A5B0-0A6E3F9E248F@worrell.nl> Message-ID: Hi Michael, What's the status of the project now? Any major progress? Regards, Zabrane From michael.santos@REDACTED Wed Aug 15 02:53:55 2012 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 14 Aug 2012 20:53:55 -0400 Subject: [erlang-questions] UDP client/server performance In-Reply-To: References: <20120814094019.GA21542@ioctl> Message-ID: <20120815005355.GB28968@ioctl> On Tue, Aug 14, 2012 at 10:10:57PM +0200, Ronny Meeus wrote: > Hello > > thanks for the info. > I'm currently playing with the procket and it is working well. > > I have 2 questions: > > - I need to run the erl as root (or use sudo to start it) or I get: > > 2> {ok, FD} = procket:open(53, [{protocol, udp},{type, dgram},{family, inet}]). > ** exception error: no match of right hand side value {error,eperm} Make priv/procket setuid or allow your user to call priv/procket using sudo. > - How do I use recvfrom/4 in the server? If I call recvfrom/4 when > there is no message present in the socket, it returns an error since > the socket is put in async mode. Yes, the socket has to be non-blocking or it will block the scheduler. > Does this mean that the socket needs > to be polled by the Erlang process or is it possible to get a > notification (msg) when there is data present in the socket so that > the recvfrom/4 can be called at that moment? You could plug the fd back into gen_udp:open(Port, [{fd, FD}]) ;) For a UDP socket, I can't think of any options other than the ones you've mentioned: either spin on EAGAIN or use another NIF to notify you when the socket is ready. There are a few ports to libev and libuv around. > On Tue, Aug 14, 2012 at 11:40 AM, Michael Santos > wrote: > > On Tue, Aug 14, 2012 at 08:25:23AM +0200, Ronny Meeus wrote: > >> Is there any code available that implements a NIF to have access to UDP sockets? > >> I have found the procket implementation but it looks like it is using > >> a separate process to communicate with the socket. This sounds like > >> overkill to me for this test. > > > > You can get the socket directly using procket:socket/3. See > > procket:sendto/4 and procket:recvfrom/4 as well for retrieving the > > struct sockaddr for the peer. > > > > -module(udp_srv). > > -export([s/0]). > > > > -define(UINT16(N), N:2/native-unsigned-integer-unit:8). > > -define(PF_INET, 2). > > -define(PORT, 6001). > > > > s() -> > > {ok,FD} = procket:socket(inet, dgram, 0), > > % sockaddr for Linux > > Sockaddr = <>, > > ok = procket:bind(FD, Sockaddr), > > Ref = erlang:open_port({fd, FD, FD}, [stream, binary]), > > loop(Ref). > > > > loop(Ref) -> > > receive > > {Ref,{data,Data}} -> > > error_logger:info_report(Data), > > loop(Ref) > > end. From zabrane3@REDACTED Wed Aug 15 03:04:14 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 15 Aug 2012 03:04:14 +0200 Subject: [erlang-questions] Erlang record accessors (set/get) In-Reply-To: References: <-9083630128156938984@unknownmsgid> <171ACD12-0FFB-455D-9076-1A269FD7E86F@gmail.com> Message-ID: Hi guys, We want to provide a simple API to one of our customers not very familiar with Erlang. This API involves records manipulations (set/get operations) and you know what I mean. Before re-writing the wheel, I'd like to know is someone already used "recbird" before: http://blogtrader.net/blog/recbird_an_erlang_dynamic_record Any use case? Is it stable? Better alternatives? Feedbacks welcome! Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Aug 15 03:10:54 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 15 Aug 2012 03:10:54 +0200 Subject: [erlang-questions] Erlang record accessors (set/get) In-Reply-To: References: <-9083630128156938984@unknownmsgid> <171ACD12-0FFB-455D-9076-1A269FD7E86F@gmail.com> Message-ID: <502AF71E.8050508@ninenines.eu> Val = element(#myrec.fieldname, Rec). Rec2 = setelement(#myrec.fieldname, Rec, NewVal). Or something approximating that. On 08/15/2012 03:04 AM, Zabrane Mickael wrote: > Hi guys, > > We want to provide a simple API to one of our customers not very > familiar with Erlang. > This API involves records manipulations (set/get operations) and you > know what I mean. > > Before re-writing the wheel, I'd like to know is someone already used > "recbird" before: > http://blogtrader.net/blog/recbird_an_erlang_dynamic_record > > Any use case? Is it stable? Better alternatives? > Feedbacks welcome! > > Regards, > Zabrane > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From mjtruog@REDACTED Wed Aug 15 03:20:24 2012 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 14 Aug 2012 18:20:24 -0700 Subject: [erlang-questions] Erlang record accessors (set/get) In-Reply-To: <502AF71E.8050508@ninenines.eu> References: <-9083630128156938984@unknownmsgid> <171ACD12-0FFB-455D-9076-1A269FD7E86F@gmail.com> <502AF71E.8050508@ninenines.eu> Message-ID: <502AF958.5040000@gmail.com> It may seem a bit silly, but you could wrap element and setelement in macros that relate to the record name with short get/set names. That would probably make it simpler for those that are addicted to get/set. On 08/14/2012 06:10 PM, Lo?c Hoguin wrote: > Val = element(#myrec.fieldname, Rec). > Rec2 = setelement(#myrec.fieldname, Rec, NewVal). > > Or something approximating that. > > On 08/15/2012 03:04 AM, Zabrane Mickael wrote: >> Hi guys, >> >> We want to provide a simple API to one of our customers not very >> familiar with Erlang. >> This API involves records manipulations (set/get operations) and you >> know what I mean. >> >> Before re-writing the wheel, I'd like to know is someone already used >> "recbird" before: >> http://blogtrader.net/blog/recbird_an_erlang_dynamic_record >> >> Any use case? Is it stable? Better alternatives? >> Feedbacks welcome! >> >> Regards, >> Zabrane >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > From juanjo@REDACTED Wed Aug 15 04:08:03 2012 From: juanjo@REDACTED (Juan Jose Comellas) Date: Tue, 14 Aug 2012 23:08:03 -0300 Subject: [erlang-questions] Erlang record accessors (set/get) In-Reply-To: References: <-9083630128156938984@unknownmsgid> <171ACD12-0FFB-455D-9076-1A269FD7E86F@gmail.com> Message-ID: We developed a similar parse transform that you can find here: https://github.com/Erlymob/emob/blob/master/src/dynarec.erl It's very simple to use. The explanation is at the beginning of the file. There's also Ulf Wiger's exprecs module from his parse_trans application: https://github.com/uwiger/parse_trans/blob/master/src/exprecs.erl Hope it helps, Juanjo On Tue, Aug 14, 2012 at 10:04 PM, Zabrane Mickael wrote: > Hi guys, > > We want to provide a simple API to one of our customers not very familiar > with Erlang. > This API involves records manipulations (set/get operations) and you know > what I mean. > > Before re-writing the wheel, I'd like to know is someone already used > "recbird" before: > http://blogtrader.net/blog/recbird_an_erlang_dynamic_record > > Any use case? Is it stable? Better alternatives? > Feedbacks welcome! > > Regards, > Zabrane > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kyletu@REDACTED Wed Aug 15 06:23:20 2012 From: kyletu@REDACTED (kyletu) Date: Wed, 15 Aug 2012 12:23:20 +0800 Subject: [erlang-questions] help References: Message-ID: <201208151223175426672@ptmind.com> help kyletu From: erlang-questions-request Date: 2012-08-15 12:20 To: kyletu Subject: Welcome to the "erlang-questions" mailing list (Digest mode) Welcome to the erlang-questions@REDACTED mailing list! To post to this list, send your message to: erlang-questions@REDACTED General information about the mailing list is at: http://erlang.org/mailman/listinfo/erlang-questions If you ever want to unsubscribe or change your options (eg, switch to or from digest mode, change your password, etc.), visit your subscription page at: http://erlang.org/mailman/options/erlang-questions/kyletu%40ptmind.com You can also make such adjustments via email by sending a message to: erlang-questions-request@REDACTED with the word `help' in the subject or body (don't include the quotes), and you will get back a message with instructions. You must know your password to change your options (including changing the password, itself) or to unsubscribe without confirmation. It is: ZzSU5oFo Normally, Mailman will remind you of your erlang.org mailing list passwords once every month, although you can disable this if you prefer. This reminder will also include instructions on how to unsubscribe or change your account options. There is also a button on your options page that will email your current password to you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kyletu@REDACTED Wed Aug 15 06:36:15 2012 From: kyletu@REDACTED (kyletu) Date: Wed, 15 Aug 2012 12:36:15 +0800 Subject: [erlang-questions] how to call mysql in erlang(using erlang-mysql-driver) Message-ID: <201208151236121852035@ptmind.com> Hello,I want call mysql PROCEDURE in erlang. but I get a error message... Is there somebody know how to do it? PROCEDURE code: CREATE PROCEDURE simpleproc_04() BEGIN select * from taokeeper_stat limit 10; END erlang call code: mysql:fetch(poolId,<<"call simpleproc_04">>). then I get the error message: mysql_conn:433: fetch <<"call simpleproc_04">> (id <0.44.0>){error,{mysql_result,[],[],0,0, "PROCEDURE taokeeper.simpleproc_04 can't return a result set in the given context", 1312,"0A000"}} Thanks to read it... kyletu -------------- next part -------------- An HTML attachment was scrubbed... URL: From kyletu@REDACTED Wed Aug 15 06:38:23 2012 From: kyletu@REDACTED (kyletu) Date: Wed, 15 Aug 2012 12:38:23 +0800 Subject: [erlang-questions] how to call mysql PROCEDURE in erlang ?(using erlang-mysql-driver) Message-ID: <201208151238198821896@ptmind.com> Hello,I want call mysql PROCEDURE in erlang. but I get a error message... Is there somebody know how to do it? PROCEDURE code: CREATE PROCEDURE simpleproc_04() BEGIN select * from taokeeper_stat limit 10; END erlang call code: mysql:fetch(poolId,<<"call simpleproc_04">>). then I get the error message: mysql_conn:433: fetch <<"call simpleproc_04">> (id <0.44.0>){error,{mysql_result,[],[],0,0, "PROCEDURE taokeeper.simpleproc_04 can't return a result set in the given context", 1312,"0A000"}} Thanks to read it... kyletu -------------- next part -------------- An HTML attachment was scrubbed... URL: From freeakk@REDACTED Wed Aug 15 07:30:00 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Wed, 15 Aug 2012 09:30:00 +0400 Subject: [erlang-questions] [ANN] Erlang Bindings for Xapian In-Reply-To: References: <1D24E487-3655-45BE-A5B0-0A6E3F9E248F@worrell.nl> Message-ID: Hi, The main part is ready, I am planning the release at August, 21. Best regards, Uvarov Michael From ronny.meeus@REDACTED Wed Aug 15 09:59:22 2012 From: ronny.meeus@REDACTED (Ronny Meeus) Date: Wed, 15 Aug 2012 09:59:22 +0200 Subject: [erlang-questions] UDP client/server performance In-Reply-To: <20120815005355.GB28968@ioctl> References: <20120814094019.GA21542@ioctl> <20120815005355.GB28968@ioctl> Message-ID: On Wed, Aug 15, 2012 at 2:53 AM, Michael Santos wrote: > On Tue, Aug 14, 2012 at 10:10:57PM +0200, Ronny Meeus wrote: >> Hello >> >> thanks for the info. >> I'm currently playing with the procket and it is working well. >> >> I have 2 questions: >> >> - I need to run the erl as root (or use sudo to start it) or I get: >> >> 2> {ok, FD} = procket:open(53, [{protocol, udp},{type, dgram},{family, inet}]). >> ** exception error: no match of right hand side value {error,eperm} > > Make priv/procket setuid or allow your user to call priv/procket using > sudo. I tried all options mentioned in your readme already before but non of them seem to work on Ubuntu. I think there is an issue with the sudo stuff on this distribution since no matter what I do, it always ask for a password when the procket command is executed. Now I succeeded by explicitly specifying the name of the program to use: procket:open(53, [{progname,"/usr/local/bin/procket"},{protocol, udp},{type,dgram}]). > >> - How do I use recvfrom/4 in the server? If I call recvfrom/4 when >> there is no message present in the socket, it returns an error since >> the socket is put in async mode. > > Yes, the socket has to be non-blocking or it will block the scheduler. > >> Does this mean that the socket needs >> to be polled by the Erlang process or is it possible to get a >> notification (msg) when there is data present in the socket so that >> the recvfrom/4 can be called at that moment? > > You could plug the fd back into gen_udp:open(Port, [{fd, FD}]) ;) > > For a UDP socket, I can't think of any options other than the ones > you've mentioned: either spin on EAGAIN or use another NIF to notify you > when the socket is ready. There are a few ports to libev and libuv around. OK I will look around. Thanks. Yesterday I did a quick test and created an application that once it receives a message, it starts to send this back in a loop. It looks like it is able to send almost 200Kpkts/sec on a low end PC while the gen_udp solution only was able to send something like 10Kpks/sec on a much more powerful PC. So it looks promising ... > >> On Tue, Aug 14, 2012 at 11:40 AM, Michael Santos >> wrote: >> > On Tue, Aug 14, 2012 at 08:25:23AM +0200, Ronny Meeus wrote: >> >> Is there any code available that implements a NIF to have access to UDP sockets? >> >> I have found the procket implementation but it looks like it is using >> >> a separate process to communicate with the socket. This sounds like >> >> overkill to me for this test. >> > >> > You can get the socket directly using procket:socket/3. See >> > procket:sendto/4 and procket:recvfrom/4 as well for retrieving the >> > struct sockaddr for the peer. >> > >> > -module(udp_srv). >> > -export([s/0]). >> > >> > -define(UINT16(N), N:2/native-unsigned-integer-unit:8). >> > -define(PF_INET, 2). >> > -define(PORT, 6001). >> > >> > s() -> >> > {ok,FD} = procket:socket(inet, dgram, 0), >> > % sockaddr for Linux >> > Sockaddr = <>, >> > ok = procket:bind(FD, Sockaddr), >> > Ref = erlang:open_port({fd, FD, FD}, [stream, binary]), >> > loop(Ref). >> > >> > loop(Ref) -> >> > receive >> > {Ref,{data,Data}} -> >> > error_logger:info_report(Data), >> > loop(Ref) >> > end. From zabrane3@REDACTED Wed Aug 15 11:00:13 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 15 Aug 2012 11:00:13 +0200 Subject: [erlang-questions] [ANN] Erlang Bindings for Xapian In-Reply-To: References: <1D24E487-3655-45BE-A5B0-0A6E3F9E248F@worrell.nl> Message-ID: <4991F58D-A3FC-4F24-92BA-0B4D6BE8BDE4@gmail.com> Hi Michael, > The main part is ready, I am planning the release at August, 21. Good to hear that. Please keep us posted. N.B: does it work on Windows too? Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Wed Aug 15 11:00:59 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 15 Aug 2012 11:00:59 +0200 Subject: [erlang-questions] Erlang record accessors (set/get) In-Reply-To: References: <-9083630128156938984@unknownmsgid> <171ACD12-0FFB-455D-9076-1A269FD7E86F@gmail.com> Message-ID: <12D60DD4-263F-4327-8E82-1A11BC3F1782@gmail.com> Thanks for your feedbacks guys. Regards, Zabrane On Aug 15, 2012, at 4:08 AM, Juan Jose Comellas wrote: > We developed a similar parse transform that you can find here: > > https://github.com/Erlymob/emob/blob/master/src/dynarec.erl > > It's very simple to use. The explanation is at the beginning of the file. There's also Ulf Wiger's exprecs module from his parse_trans application: > > https://github.com/uwiger/parse_trans/blob/master/src/exprecs.erl > > Hope it helps, > > Juanjo > > > On Tue, Aug 14, 2012 at 10:04 PM, Zabrane Mickael wrote: > Hi guys, > > We want to provide a simple API to one of our customers not very familiar with Erlang. > This API involves records manipulations (set/get operations) and you know what I mean. > > Before re-writing the wheel, I'd like to know is someone already used "recbird" before: > http://blogtrader.net/blog/recbird_an_erlang_dynamic_record > > Any use case? Is it stable? Better alternatives? > Feedbacks welcome! > > Regards, > Zabrane > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From freeakk@REDACTED Wed Aug 15 11:34:16 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Wed, 15 Aug 2012 13:34:16 +0400 Subject: [erlang-questions] Erlang record accessors (set/get) In-Reply-To: <12D60DD4-263F-4327-8E82-1A11BC3F1782@gmail.com> References: <-9083630128156938984@unknownmsgid> <171ACD12-0FFB-455D-9076-1A269FD7E86F@gmail.com> <12D60DD4-263F-4327-8E82-1A11BC3F1782@gmail.com> Message-ID: Something like this (but for multiple fields) is used here: https://github.com/freeakk/rum#example-2 From freeakk@REDACTED Wed Aug 15 11:38:06 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Wed, 15 Aug 2012 13:38:06 +0400 Subject: [erlang-questions] [ANN] Erlang Bindings for Xapian In-Reply-To: <4991F58D-A3FC-4F24-92BA-0B4D6BE8BDE4@gmail.com> References: <1D24E487-3655-45BE-A5B0-0A6E3F9E248F@worrell.nl> <4991F58D-A3FC-4F24-92BA-0B4D6BE8BDE4@gmail.com> Message-ID: > N.B: does it work on Windows too? Maybe. I did not use any platform-specific code. But I didn't test it on Windows too. From zabrane3@REDACTED Wed Aug 15 11:54:57 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 15 Aug 2012 11:54:57 +0200 Subject: [erlang-questions] [ANN] Erlang Bindings for Xapian In-Reply-To: References: <1D24E487-3655-45BE-A5B0-0A6E3F9E248F@worrell.nl> <4991F58D-A3FC-4F24-92BA-0B4D6BE8BDE4@gmail.com> Message-ID: I can help on that. Regards, Zabrane On Aug 15, 2012, at 11:38 AM, Michael Uvarov wrote: >> N.B: does it work on Windows too? > Maybe. I did not use any platform-specific code. But I didn't test it > on Windows too. From kenneth.lundin@REDACTED Wed Aug 15 12:22:06 2012 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 15 Aug 2012 12:22:06 +0200 Subject: [erlang-questions] UDP client/server performance In-Reply-To: References: Message-ID: Hi Ronny, I think you are a little bit fast in drawing the conclusion that you must look at other solutions than just using the standard gen_udp module. If you have found some real bottle neck in performance we should of course try to find out why the difference between C and Erlang is so big, I don't think it should be that big. In case there is something unnecessarily inefficient going on when using gen_udp we should try to address that instead of looking at more complex and not so general alternatives A deeper analysis of what your test is doing would be good, is it comparable with the C case? Can the difference be explained etc. I am not convinced that you will get better speed the alternative ways you have been recommended. The socket code is written in C even when you use gen_udp so I don't really see the big difference. I also suppose you in reality want to do something with the data on the Erlang side and that would probably change the characteristics. Another thing is if there is multiple cores involved etc. Kenneth , Erlang/OTP Ericsson On 8/14/12, Ronny Meeus wrote: > Is there any code available that implements a NIF to have access to UDP > sockets? > I have found the procket implementation but it looks like it is using > a separate process to communicate with the socket. This sounds like > overkill to me for this test. > > Ronny > > On Mon, Aug 13, 2012 at 10:29 PM, Ronny Meeus > wrote: >> Hello >> >> I did a quick check. >> - The values between 2 machines are comparable for Erlang. >> - For the C code there is a difference: the async version drops from >> 700Kpks/sec to 400Kpks/sec while the sync version drops from >> 86Kpkts/sec to 10Kpkts/sec. >> >> >> On Mon, Aug 13, 2012 at 9:42 PM, JD Bothma wrote: >>> I'm curious: >>> >>> What are the numbers like (c versus erlang) when it's between two >>> physical machines? Or between two (vmware or virtualbox, not xen) >>> virtual machines? >>> >>> Or said otherwise: does the fact that it's the same machine, network >>> stack and loopback interface affect the comparison between c and >>> erlang? >>> >>> JD >>> >>> On 13 August 2012 21:11, Ronny Meeus wrote: >>>> Hello >>>> >>>> I have created a simple UDP based client/server application. >>>> The code is available on bitbucket >>>> (https://bitbucket.org/meeusr/erl-examples) in the networking >>>> directory. >>>> >>>> The server is just an echo server running in 1 process. >>>> The client can behave like a sync client (sending 1 message and >>>> waiting for the reply) or like an async client (sending X messages >>>> before waiting for all the replies). >>>> I see that my PC (Intel(R) Quad code i7 CPU 860 @ 2.80GHz) is able to >>>> process something like 7K - 10K messages per second. Both the sync and >>>> the async are in the same order of magnitude. Please note that I use >>>> the loopback interface. >>>> >>>> $ erl +sbt db >>>> Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] >>>> [async-threads:0] [hipe] [kernel-poll:false] >>>> >>>> Eshell V5.9.1 (abort with ^G) >>>> 1> c(gen_udp_test). >>>> gen_udp_test.erl:47: Warning: variable 'Data' is unused >>>> gen_udp_test.erl:47: Warning: variable 'Msg' is unused >>>> gen_udp_test.erl:62: Warning: variable 'Data' is unused >>>> gen_udp_test.erl:62: Warning: variable 'Msg' is unused >>>> {ok,gen_udp_test} >>>> 2> gen_udp_test:start_server(). >>>> true >>>> ... >>>> 12> gen_udp_test:client_async(5000). >>>> stop : 841482 >>>> ok >>>> 13> gen_udp_test:client_async(6000). >>>> stop : 1082679 >>>> ok >>>> >>>> The timings are the number of usec for the processing of the X number >>>> of messages (round-trip). >>>> >>>> If I implement the same application in C (client and server are 2 >>>> applications), I see that the application performs an order of >>>> magnitude better for the sync solution (86Kpkts/sec) and 100 times >>>> better (700Kpkts/sec) for the async version. >>>> The code for this is also available in the repo on bitbucket. >>>> >>>> I know there this test is only looking to a part of the application >>>> (transport of the data), but I find the difference rather big. >>>> Is this normal or is there something wrong in my Erlang code? >>>> >>>> Thanks. >>>> Regards, >>>> Ronny >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From max.lapshin@REDACTED Wed Aug 15 12:32:10 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 15 Aug 2012 14:32:10 +0400 Subject: [erlang-questions] UDP client/server performance In-Reply-To: References: Message-ID: I've met difference between erlang and C code only once. I had to receive MPEG-TS UDP packets, about 100 mbits of them, pack them into large packets and send via http. One MPEG-TS UDP packet is about 1300 bytes (7 blocks of 188 bytes), so there is about 70 000 of messages per second. I've rewritten UDP handling code in C (with using all erl_driver capabilities) and made packaging inside C driver. Now erlang code received about 700 messages per second and CPU got down from 90% to 10%. So, my ehnancement was in lowering amount of messages and preallocating big buffers inside C level. From watson.timothy@REDACTED Tue Aug 14 10:04:12 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 14 Aug 2012 09:04:12 +0100 Subject: [erlang-questions] common test, test_server and ct_hooks In-Reply-To: References: <4B447EE0-BDF3-43DB-836E-3400A26B2CAE@gmail.com> Message-ID: Hi Peter On 13 Aug 2012, at 13:38, Peter Andersson wrote: > Hi Tim, > > When a test case times out, the test server control process kills the > test case process, on which the Config data is known (it's last > returned by init_per_testcase/2 on the same hanging process). The > result is that the latest Config value is unknown when the call to the > post_end_per_testcase hook function takes place. This is not a bug as > such (and OTP-9594 is a different issue), but it's a limitation for > sure. OK this makes perfect sense. > There are 2 possible solutions to this problem: > > 1. You keep saving the last known Config data (or at least your > relevant subset of it) in the hook state. If the test case times out, > you can restore the data from there. No need to use an ets table. You > will be able to restore the Config data that init_per_testcase/2 was > called with, but not the final value that was passed to the test case > function. > > 2. Common Test gets updated so that the control process saves the last > known Config for a test case so that it can be restored and passed to > the post_end_per_testcase hook call. The result will be the same as #1 > above, but it will make it easier to implement the CTH callback > module. It might also be possible to have the test case process send > the Config data to the control process after init_per_testcase, so > that the very latest Config can be saved and restored. Implementation > of this can be somewhat tricky because of parallel test case groups - > but certainly not impossible! ;-) > > I'd be very happy (well, sort of) to implement #2 above. Will #1 be an > ok workaround for you until #2 is in place? > Yes #1 will work fine for the time being. In fact, as it's mainly a pid that I'm tracking (so that a gen_server can be instructed to tear-down and stop) I've opted for local name registration as a workaround for the time being. At some point I'll have to think about name clashes, at which point I'll need #2 but for now there aren't any as the server name/id is tied to the suite, group or test-case name anyway. > I know it's usually not appropriate to be talking about doing #1 and > #2, but in this context you know what I mean! :-) > Absolutely and that sounds like a good phased approach for my needs. Thanks again for getting back to me. Cheers, Tim -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 235 bytes Desc: Message signed with OpenPGP using GPGMail URL: From watson.timothy@REDACTED Tue Aug 14 10:10:31 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 14 Aug 2012 09:10:31 +0100 Subject: [erlang-questions] Remote command execution via SSH ? In-Reply-To: <50298642.9000902@gmail.com> References: <50298642.9000902@gmail.com> Message-ID: <8F434A56-67E1-4B51-8D74-B85706EF5A89@gmail.com> On 13 Aug 2012, at 23:57, Michael Truog wrote: > On 08/13/2012 02:54 PM, Thomas Elsgaard wrote: >> >> Hi >> >> Does anybody have a good simple example of executing commands on a >> remote server via SSH? >> >> Nothing fancy, just something like: >> >> ------ >> Connect to host >> Execute command '/bin/ls' >> Read output from command >> Close connection >> ------ >> > Just do: > slave:start_link('nodename', 'user', "-rsh ssh"). That works a treat if the node has Erlang installed under the same file system layout and one of two other constraints. If that isn't the case it fails. There is an ssh application with which you could wrap this up of course, and iirc a couple of people on github have started to put together projects to address this kind of thing, though I've no idea how complete or stable they are, nor indeed what the projects are called. Slave module works well if the constraints don't bother you though - I use it a lot. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 235 bytes Desc: Message signed with OpenPGP using GPGMail URL: From ronny.meeus@REDACTED Wed Aug 15 14:14:05 2012 From: ronny.meeus@REDACTED (Ronny Meeus) Date: Wed, 15 Aug 2012 14:14:05 +0200 Subject: [erlang-questions] UDP client/server performance In-Reply-To: References: Message-ID: Hello my final goal is to implement a statsd like application completely in Erlang (as a exercise to learn more about the various aspects of Erlang). The first step for me is to implement a UDP echo server to have a feeling about the performance. Since this was very low in my opinion (the server handled only 5000 messages per second), I wrote the same in C. While running the etop I also observed that the system was using a lot of memory while sending the messages. Basically both applications do the same thing: the client is sending a string "TEST" and the server echos it back. The bad performance of the Erlang solution triggered me to send a mail to the community, maybe I was doing something wrong. So it might be good to have a look at the code I have created and check whether it is using the gen_udp in a correct and most performant way. I agree that it is better to improve the gen_udp code so that it is beneficial for all applications, but I'm afraid that my skills are not good enough for that. Ronny On Wed, Aug 15, 2012 at 12:22 PM, Kenneth Lundin wrote: > Hi Ronny, > > I think you are a little bit fast in drawing the conclusion that you > must look at other solutions > than just using the standard gen_udp module. > > If you have found some real bottle neck in performance we should of > course try to find out why the difference between C and Erlang is so > big, I don't think it should be that big. > > In case there is something unnecessarily inefficient going on when > using gen_udp we should > try to address that instead of looking at more complex and not so > general alternatives > > A deeper analysis of what your test is doing would be good, is it > comparable with the C case? > Can the difference be explained etc. > > I am not convinced that you will get better speed the alternative ways > you have been recommended. The socket code is written in C even when > you use gen_udp so I don't > really see the big difference. I also suppose you in reality want to > do something with the > data on the Erlang side and that would probably change the > characteristics. Another thing is if there is multiple cores involved > etc. > > Kenneth , Erlang/OTP Ericsson > > On 8/14/12, Ronny Meeus wrote: >> Is there any code available that implements a NIF to have access to UDP >> sockets? >> I have found the procket implementation but it looks like it is using >> a separate process to communicate with the socket. This sounds like >> overkill to me for this test. >> >> Ronny >> >> On Mon, Aug 13, 2012 at 10:29 PM, Ronny Meeus >> wrote: >>> Hello >>> >>> I did a quick check. >>> - The values between 2 machines are comparable for Erlang. >>> - For the C code there is a difference: the async version drops from >>> 700Kpks/sec to 400Kpks/sec while the sync version drops from >>> 86Kpkts/sec to 10Kpkts/sec. >>> >>> >>> On Mon, Aug 13, 2012 at 9:42 PM, JD Bothma wrote: >>>> I'm curious: >>>> >>>> What are the numbers like (c versus erlang) when it's between two >>>> physical machines? Or between two (vmware or virtualbox, not xen) >>>> virtual machines? >>>> >>>> Or said otherwise: does the fact that it's the same machine, network >>>> stack and loopback interface affect the comparison between c and >>>> erlang? >>>> >>>> JD >>>> >>>> On 13 August 2012 21:11, Ronny Meeus wrote: >>>>> Hello >>>>> >>>>> I have created a simple UDP based client/server application. >>>>> The code is available on bitbucket >>>>> (https://bitbucket.org/meeusr/erl-examples) in the networking >>>>> directory. >>>>> >>>>> The server is just an echo server running in 1 process. >>>>> The client can behave like a sync client (sending 1 message and >>>>> waiting for the reply) or like an async client (sending X messages >>>>> before waiting for all the replies). >>>>> I see that my PC (Intel(R) Quad code i7 CPU 860 @ 2.80GHz) is able to >>>>> process something like 7K - 10K messages per second. Both the sync and >>>>> the async are in the same order of magnitude. Please note that I use >>>>> the loopback interface. >>>>> >>>>> $ erl +sbt db >>>>> Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] >>>>> [async-threads:0] [hipe] [kernel-poll:false] >>>>> >>>>> Eshell V5.9.1 (abort with ^G) >>>>> 1> c(gen_udp_test). >>>>> gen_udp_test.erl:47: Warning: variable 'Data' is unused >>>>> gen_udp_test.erl:47: Warning: variable 'Msg' is unused >>>>> gen_udp_test.erl:62: Warning: variable 'Data' is unused >>>>> gen_udp_test.erl:62: Warning: variable 'Msg' is unused >>>>> {ok,gen_udp_test} >>>>> 2> gen_udp_test:start_server(). >>>>> true >>>>> ... >>>>> 12> gen_udp_test:client_async(5000). >>>>> stop : 841482 >>>>> ok >>>>> 13> gen_udp_test:client_async(6000). >>>>> stop : 1082679 >>>>> ok >>>>> >>>>> The timings are the number of usec for the processing of the X number >>>>> of messages (round-trip). >>>>> >>>>> If I implement the same application in C (client and server are 2 >>>>> applications), I see that the application performs an order of >>>>> magnitude better for the sync solution (86Kpkts/sec) and 100 times >>>>> better (700Kpkts/sec) for the async version. >>>>> The code for this is also available in the repo on bitbucket. >>>>> >>>>> I know there this test is only looking to a part of the application >>>>> (transport of the data), but I find the difference rather big. >>>>> Is this normal or is there something wrong in my Erlang code? >>>>> >>>>> Thanks. >>>>> Regards, >>>>> Ronny >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From zabrane3@REDACTED Wed Aug 15 15:49:09 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 15 Aug 2012 15:49:09 +0200 Subject: [erlang-questions] Erlang record accessors (set/get) In-Reply-To: References: <-9083630128156938984@unknownmsgid> <171ACD12-0FFB-455D-9076-1A269FD7E86F@gmail.com> Message-ID: <9089F4C8-253B-49DB-AD9C-58808ED16F80@gmail.com> Hi Juan, What's the current emob's license (GPL, BSD...)? Regards, Zabrane On Aug 15, 2012, at 4:08 AM, Juan Jose Comellas wrote: > We developed a similar parse transform that you can find here: > > https://github.com/Erlymob/emob/blob/master/src/dynarec.erl > > It's very simple to use. The explanation is at the beginning of the file. There's also Ulf Wiger's exprecs module from his parse_trans application: > > https://github.com/uwiger/parse_trans/blob/master/src/exprecs.erl > > Hope it helps, > > Juanjo > > > On Tue, Aug 14, 2012 at 10:04 PM, Zabrane Mickael wrote: > Hi guys, > > We want to provide a simple API to one of our customers not very familiar with Erlang. > This API involves records manipulations (set/get operations) and you know what I mean. > > Before re-writing the wheel, I'd like to know is someone already used "recbird" before: > http://blogtrader.net/blog/recbird_an_erlang_dynamic_record > > Any use case? Is it stable? Better alternatives? > Feedbacks welcome! > > Regards, > Zabrane > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Wed Aug 15 15:51:04 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 15 Aug 2012 15:51:04 +0200 Subject: [erlang-questions] Erlang record accessors (set/get) In-Reply-To: <9089F4C8-253B-49DB-AD9C-58808ED16F80@gmail.com> References: <-9083630128156938984@unknownmsgid> <171ACD12-0FFB-455D-9076-1A269FD7E86F@gmail.com> <9089F4C8-253B-49DB-AD9C-58808ED16F80@gmail.com> Message-ID: Sorry, just found it inside the code: New BSD, right? Regards, Zabrane On Aug 15, 2012, at 3:49 PM, Zabrane Mickael wrote: > Hi Juan, > > What's the current emob's license (GPL, BSD...)? > > Regards, > Zabrane > > On Aug 15, 2012, at 4:08 AM, Juan Jose Comellas wrote: > >> We developed a similar parse transform that you can find here: >> >> https://github.com/Erlymob/emob/blob/master/src/dynarec.erl >> >> It's very simple to use. The explanation is at the beginning of the file. There's also Ulf Wiger's exprecs module from his parse_trans application: >> >> https://github.com/uwiger/parse_trans/blob/master/src/exprecs.erl >> >> Hope it helps, >> >> Juanjo >> >> >> On Tue, Aug 14, 2012 at 10:04 PM, Zabrane Mickael wrote: >> Hi guys, >> >> We want to provide a simple API to one of our customers not very familiar with Erlang. >> This API involves records manipulations (set/get operations) and you know what I mean. >> >> Before re-writing the wheel, I'd like to know is someone already used "recbird" before: >> http://blogtrader.net/blog/recbird_an_erlang_dynamic_record >> >> Any use case? Is it stable? Better alternatives? >> Feedbacks welcome! >> >> Regards, >> Zabrane >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.santos@REDACTED Wed Aug 15 16:03:07 2012 From: michael.santos@REDACTED (Michael Santos) Date: Wed, 15 Aug 2012 10:03:07 -0400 Subject: [erlang-questions] UDP client/server performance In-Reply-To: References: <20120814094019.GA21542@ioctl> <20120815005355.GB28968@ioctl> Message-ID: <20120815140307.GD28968@ioctl> On Wed, Aug 15, 2012 at 09:59:22AM +0200, Ronny Meeus wrote: > On Wed, Aug 15, 2012 at 2:53 AM, Michael Santos > wrote: > > On Tue, Aug 14, 2012 at 10:10:57PM +0200, Ronny Meeus wrote: > >> Hello > >> > >> thanks for the info. > >> I'm currently playing with the procket and it is working well. > >> > >> I have 2 questions: > >> > >> - I need to run the erl as root (or use sudo to start it) or I get: > >> > >> 2> {ok, FD} = procket:open(53, [{protocol, udp},{type, dgram},{family, inet}]). > >> ** exception error: no match of right hand side value {error,eperm} > > > > Make priv/procket setuid or allow your user to call priv/procket using > > sudo. > > I tried all options mentioned in your readme already before but non of > them seem to work on Ubuntu. I think there is an issue with the sudo > stuff on this distribution since no matter what I do, it always ask > for a password when the procket command is executed. > > Now I succeeded by explicitly specifying the name of the program to use: > procket:open(53, [{progname,"/usr/local/bin/procket"},{protocol, > udp},{type,dgram}]). procket uses priv/procket as the default path. Sorry, README should be clearer. > >> - How do I use recvfrom/4 in the server? If I call recvfrom/4 when > >> there is no message present in the socket, it returns an error since > >> the socket is put in async mode. > > > > Yes, the socket has to be non-blocking or it will block the scheduler. > > > >> Does this mean that the socket needs > >> to be polled by the Erlang process or is it possible to get a > >> notification (msg) when there is data present in the socket so that > >> the recvfrom/4 can be called at that moment? > > > > You could plug the fd back into gen_udp:open(Port, [{fd, FD}]) ;) > > > > For a UDP socket, I can't think of any options other than the ones > > you've mentioned: either spin on EAGAIN or use another NIF to notify you > > when the socket is ready. There are a few ports to libev and libuv around. > > OK I will look around. > Thanks. > > Yesterday I did a quick test and created an application that once it > receives a message, it starts to send this back in a loop. > It looks like it is able to send almost 200Kpkts/sec on a low end PC > while the gen_udp solution only was able to send something like > 10Kpks/sec on a much more powerful PC. So it looks promising ... That's cool! But remember, the C and the Erlang examples you posted are doing different things. The C code is single tasking. It doesn't do any error handling either. The Erlang code is multitasking, switching between processes, monitoring processes for failure, .... If you run C NIF code in a tight loop, you'll get something closer to the single tasking C code at the expense of concurrency. Maybe that is fine for your needs though. Anyway, good luck in hitting your numbers! From mahesh@REDACTED Wed Aug 15 16:08:57 2012 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Wed, 15 Aug 2012 10:08:57 -0400 Subject: [erlang-questions] Erlang record accessors (set/get) In-Reply-To: References: <-9083630128156938984@unknownmsgid> <171ACD12-0FFB-455D-9076-1A269FD7E86F@gmail.com> <9089F4C8-253B-49DB-AD9C-58808ED16F80@gmail.com> Message-ID: <54251F63-F83F-4FDD-B974-068F9F8D6DE4@dieswaytoofast.com> yup... Cheers Mahesh Paolini-Subramanya That Tall Bald Indian Guy... Blog | Twitter | Google+ On Aug 15, 2012, at 9:51 AM, Zabrane Mickael wrote: > Sorry, just found it inside the code: New BSD, right? > > Regards, > Zabrane > > > On Aug 15, 2012, at 3:49 PM, Zabrane Mickael wrote: > >> Hi Juan, >> >> What's the current emob's license (GPL, BSD...)? >> >> Regards, >> Zabrane >> >> On Aug 15, 2012, at 4:08 AM, Juan Jose Comellas wrote: >> >>> We developed a similar parse transform that you can find here: >>> >>> https://github.com/Erlymob/emob/blob/master/src/dynarec.erl >>> >>> It's very simple to use. The explanation is at the beginning of the file. There's also Ulf Wiger's exprecs module from his parse_trans application: >>> >>> https://github.com/uwiger/parse_trans/blob/master/src/exprecs.erl >>> >>> Hope it helps, >>> >>> Juanjo >>> >>> >>> On Tue, Aug 14, 2012 at 10:04 PM, Zabrane Mickael wrote: >>> Hi guys, >>> >>> We want to provide a simple API to one of our customers not very familiar with Erlang. >>> This API involves records manipulations (set/get operations) and you know what I mean. >>> >>> Before re-writing the wheel, I'd like to know is someone already used "recbird" before: >>> http://blogtrader.net/blog/recbird_an_erlang_dynamic_record >>> >>> Any use case? Is it stable? Better alternatives? >>> Feedbacks welcome! >>> >>> Regards, >>> Zabrane >>> >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bile@REDACTED Wed Aug 15 16:29:47 2012 From: bile@REDACTED (Antonio SJ Musumeci) Date: Wed, 15 Aug 2012 10:29:47 -0400 Subject: [erlang-questions] log_roller log parsing / binary match speeds Message-ID: I've been using log_roller for a project and want to be able to dump the binary files as erlang terms in plain text. Looking through the log_roller code I found the below which works but seems rather slow. My question is: is this expected performance? Is there anything I can do to speed this up? loop(<<16#FF:8, 16#FF:8, 16#FF:8, 16#FF:8, LogSize:16/integer, Log:LogSize/binary, 16#EE:8, 16#EE:8, 16#EE:8, 16#EE:8, Tail/binary>>) -> % process log loop(Tail); loop(<<>>) -> ok. On a 2.3GHz Xeon this is taking 25 seconds for a 100MB file. The binary is sourced by calling file:read_file/1. It's being run as an escript. thanks From carlsson.richard@REDACTED Wed Aug 15 21:03:57 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 15 Aug 2012 22:03:57 +0300 Subject: [erlang-questions] Why io::format() type is not an iolist? In-Reply-To: References: Message-ID: <502BF29D.7050504@gmail.com> On 08/09/2012 10:45 AM, Aleksandr Vinokurov wrote: > > > Hello all > > I've found the subject and did not understand the root of such > restriction: why io:format/2 can't understand iolist for a Format arg? In order to interpret the Format argument, it would need to flatten the iolist first anyway. For example, io:format(["~",[<<"w">>]], [Term]). It's simply easier from the library's point of view to just say that the Format argument needs to be a flat string (guaranteeing that there's no needless overhead for flattening in the majority of cases), and if you have an iolist, then define your own format function like this: format(IOList, Args) -> io:format(binary_to_list(erlang:iolist_to_binary(IOList)), Args). /Richard From dmkolesnikov@REDACTED Wed Aug 15 21:50:42 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Wed, 15 Aug 2012 22:50:42 +0300 Subject: [erlang-questions] SSL behavior!? Message-ID: <5FB27B14-022E-42DF-9E96-EB361D36A764@gmail.com> Hello, I am experience very wired behavior with SSL in R15... So, I do have both client & server implemented on erlang, using ssl in active mode. The implementation is straight forward it Client-side: {ok, Tcp} = gen_tcp:connect(Host, Port, ?SOCK_OPTS, T), {ok, Ssl} = ssl:connect(Tcp, []), Server-side: {ok, LSock} = ssl:listen(Port, [ {ip, IP}, {certfile, Cert}, {keyfile, Key}, {reuseaddr, true} | ?SOCK_OPTS ]), ... {ok, Sock} = ssl:transport_accept(LSock), ok = ssl:ssl_accept(Sock), -define(SOCK_OPTS, [ {active, once}, {mode, binary}, {nodelay, true}, %% BTW, if I drop nodelay or put it to false, same issue {recbuf, 16 * 1024}, {sndbuf, 16 * 1024} ]). My issue is that client send data, the data got fragmented into multiple "Application Data" messages. I've been validating it by sniffing the traffic. I cannot get idea why this happens... Client log: 22:34:54.948 [info] ssl connected {{127,0,0,1},8443}, local addr {{127,0,0,1},53804}, suite {tlsv1,{dhe_rsa,aes_256_cbc,sha}} 22:34:54.948 [debug] ssl send {{127,0,0,1},8443} <<"0123456789abcdef">> Server log: 22:34:54.948 [info] ssl accepted {{127,0,0,1},53804}, local addr {{127,0,0,1},8443}, suite {tlsv1,{dhe_rsa,aes_256_cbc,sha}} 22:34:54.949 [debug] ssl recv {{127,0,0,1},53804} <<"0">> 22:34:54.949 [debug] ssl recv {{127,0,0,1},53804} <<"123456789abcdef">> thanks in advanced, Best Regards, Dmitry From essen@REDACTED Wed Aug 15 22:38:51 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 15 Aug 2012 22:38:51 +0200 Subject: [erlang-questions] SSL behavior!? In-Reply-To: <5FB27B14-022E-42DF-9E96-EB361D36A764@gmail.com> References: <5FB27B14-022E-42DF-9E96-EB361D36A764@gmail.com> Message-ID: <502C08DB.7030705@ninenines.eu> Hey, The same can happen for gen_tcp. You can't assume what you are sending isn't going to be fragmented when you receive it, packets have a size limit (MTU). You need to either know how much data you are waiting for, or try to parse it to validate it got received fully before processing it. On 08/15/2012 09:50 PM, Dmitry Kolesnikov wrote: > Hello, > > I am experience very wired behavior with SSL in R15... So, I do have both client & server implemented on erlang, using ssl in active mode. > > The implementation is straight forward it > > Client-side: > {ok, Tcp} = gen_tcp:connect(Host, Port, ?SOCK_OPTS, T), > {ok, Ssl} = ssl:connect(Tcp, []), > > Server-side: > {ok, LSock} = ssl:listen(Port, [ > {ip, IP}, > {certfile, Cert}, > {keyfile, Key}, > {reuseaddr, true} | ?SOCK_OPTS > ]), > ... > {ok, Sock} = ssl:transport_accept(LSock), > ok = ssl:ssl_accept(Sock), > > > -define(SOCK_OPTS, [ > {active, once}, > {mode, binary}, > {nodelay, true}, %% BTW, if I drop nodelay or put it to false, same issue > {recbuf, 16 * 1024}, > {sndbuf, 16 * 1024} > ]). > > > My issue is that client send data, the data got fragmented into multiple "Application Data" messages. I've been validating it by sniffing the traffic. I cannot get idea why this happens... > > Client log: > 22:34:54.948 [info] ssl connected {{127,0,0,1},8443}, local addr {{127,0,0,1},53804}, suite {tlsv1,{dhe_rsa,aes_256_cbc,sha}} > 22:34:54.948 [debug] ssl send {{127,0,0,1},8443} <<"0123456789abcdef">> > > Server log: > 22:34:54.948 [info] ssl accepted {{127,0,0,1},53804}, local addr {{127,0,0,1},8443}, suite {tlsv1,{dhe_rsa,aes_256_cbc,sha}} > 22:34:54.949 [debug] ssl recv {{127,0,0,1},53804} <<"0">> > 22:34:54.949 [debug] ssl recv {{127,0,0,1},53804} <<"123456789abcdef">> > > thanks in advanced, > Best Regards, Dmitry > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From zabrane3@REDACTED Thu Aug 16 00:31:32 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 16 Aug 2012 00:31:32 +0200 Subject: [erlang-questions] Report: sheriff + dialyzer = hard time for "parse_transform" In-Reply-To: <502C08DB.7030705@ninenines.eu> References: <5FB27B14-022E-42DF-9E96-EB361D36A764@gmail.com> <502C08DB.7030705@ninenines.eu> Message-ID: Hi Lo?c, I'm evaluating Sheriff for our team. Under R15B01 (OSX/Linux): $ git clone https://github.com/extend/sheriff.git $ cd sheriff $ make ==> edown (compile) ==> parse_trans (compile) ==> sheriff (compile) Now: $ make build-plt (takes few minutes) $ make dialyze Checking whether the PLT .sheriff_dialyzer.plt is up-to-date... yes Proceeding with analysis... dialyzer: Analysis failed with error: Could not scan the following file(s): [{"/home/zab/sheriff/src/sheriff.erl", ["/home/zab/sheriff/src/sheriff.erl:none: error in parse transform 'parse_trans_codegen': {undef,\n [{parse_trans_codegen,\n parse_transform,\n [[{attribute,1,file,\n {\"/home/zab/sheriff/src/sheriff.erl\",\n 1}},\n {attribute,21,module,\n sheriff},\n {attribute,24,export,\n [{check,2},\n {parse_transform,2},\n {build_type,3}]},\n {attribute,39,spec,\n {{check,2},\n [{type,39,'fun',\n [{type,39,product,\n [{type,39,any,[]},\n [...] Any fix? Regards, Zabrane From essen@REDACTED Thu Aug 16 00:54:04 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 16 Aug 2012 00:54:04 +0200 Subject: [erlang-questions] Report: sheriff + dialyzer = hard time for "parse_transform" In-Reply-To: References: <5FB27B14-022E-42DF-9E96-EB361D36A764@gmail.com> <502C08DB.7030705@ninenines.eu> Message-ID: <502C288C.3000603@ninenines.eu> Hey, On 08/16/2012 12:31 AM, Zabrane Mickael wrote: > Hi Lo?c, > > I'm evaluating Sheriff for our team. > Under R15B01 (OSX/Linux): > > $ git clone https://github.com/extend/sheriff.git > $ cd sheriff > $ make > ==> edown (compile) > ==> parse_trans (compile) > ==> sheriff (compile) > > Now: > $ make build-plt (takes few minutes) > $ make dialyze > Checking whether the PLT .sheriff_dialyzer.plt is up-to-date... yes > Proceeding with analysis... > dialyzer: Analysis failed with error: > Could not scan the following file(s): [{"/home/zab/sheriff/src/sheriff.erl", > ["/home/zab/sheriff/src/sheriff.erl:none: error in parse transform 'parse_trans_codegen': {undef,\n [{parse_trans_codegen,\n parse_transform,\n [[{attribute,1,file,\n {\"/home/zab/sheriff/src/sheriff.erl\",\n 1}},\n {attribute,21,module,\n sheriff},\n {attribute,24,export,\n [{check,2},\n {parse_transform,2},\n {build_type,3}]},\n {attribute,39,spec,\n {{check,2},\n [{type,39,'fun',\n [{type,39,product,\n [{type,39,any,[]},\n > [...] > > Any fix? Don't think I ever tried to dialyze it, tests were good enough for that purpose. This sounds like a parse_trans error, but why would it only occur on dialyzing? Regardless dialyzer would work fine if operating on compiled files with debug enabled. This has no effect on the actual usefulness of sheriff anyway. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From zabrane3@REDACTED Thu Aug 16 01:15:13 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 16 Aug 2012 01:15:13 +0200 Subject: [erlang-questions] Report: sheriff + dialyzer = hard time for "parse_transform" In-Reply-To: <502C288C.3000603@ninenines.eu> References: <5FB27B14-022E-42DF-9E96-EB361D36A764@gmail.com> <502C08DB.7030705@ninenines.eu> <502C288C.3000603@ninenines.eu> Message-ID: <4FBADF84-EDA7-46E9-AC4A-1D5216E3F670@gmail.com> Hi Lo?c, > Don't think I ever tried to dialyze it, tests were good enough for that purpose. This sounds like a parse_trans error, but why would it only occur on dialyzing? Yep for sure. > Regardless dialyzer would work fine if operating on compiled files with debug enabled. How can I check that (i.e test only from beams with debug enabled)? Regards, Zabrane From dmkolesnikov@REDACTED Thu Aug 16 06:47:27 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Thu, 16 Aug 2012 07:47:27 +0300 Subject: [erlang-questions] SSL behavior!? In-Reply-To: <20497694.9144.1345091786711.JavaMail.mobile-sync@vcbjg4> References: <5FB27B14-022E-42DF-9E96-EB361D36A764@gmail.com> <20497694.9144.1345091786711.JavaMail.mobile-sync@vcbjg4> Message-ID: <-4973558989212900051@unknownmsgid> Hello, Yes for sure, you have to deal with IP fragmentation. But, like I said in my post fragmentation happens on SSL level, two distinguished "App Data" frames where generated and those frames were put into single packet. I would not raise an issue if I would observe a fragmentation on large dataset but for 16 byte packet looks suspicious for me, especially of first fragment is one byte... I've not seen that issue on R14, it pops up when I jumped to R15 and made slight adjustments my code. Best Regards, Dmitry >-|-|-*> On 15.8.2012, at 23.38, "Lo?c Hoguin" wrote: > Hey, > > The same can happen for gen_tcp. You can't assume what you are sending > isn't going to be fragmented when you receive it, packets have a size > limit (MTU). You need to either know how much data you are waiting for, > or try to parse it to validate it got received fully before processing it. > > On 08/15/2012 09:50 PM, Dmitry Kolesnikov wrote: >> Hello, >> >> I am experience very wired behavior with SSL in R15... So, I do have both client & server implemented on erlang, using ssl in active mode. >> >> The implementation is straight forward it >> >> Client-side: >> {ok, Tcp} = gen_tcp:connect(Host, Port, ?SOCK_OPTS, T), >> {ok, Ssl} = ssl:connect(Tcp, []), >> >> Server-side: >> {ok, LSock} = ssl:listen(Port, [ >> {ip, IP}, >> {certfile, Cert}, >> {keyfile, Key}, >> {reuseaddr, true} | ?SOCK_OPTS >> ]), >> ... >> {ok, Sock} = ssl:transport_accept(LSock), >> ok = ssl:ssl_accept(Sock), >> >> >> -define(SOCK_OPTS, [ >> {active, once}, >> {mode, binary}, >> {nodelay, true}, %% BTW, if I drop nodelay or put it to false, same issue >> {recbuf, 16 * 1024}, >> {sndbuf, 16 * 1024} >> ]). >> >> >> My issue is that client send data, the data got fragmented into multiple "Application Data" messages. I've been validating it by sniffing the traffic. I cannot get idea why this happens... >> >> Client log: >> 22:34:54.948 [info] ssl connected {{127,0,0,1},8443}, local addr {{127,0,0,1},53804}, suite {tlsv1,{dhe_rsa,aes_256_cbc,sha}} >> 22:34:54.948 [debug] ssl send {{127,0,0,1},8443} <<"0123456789abcdef">> >> >> Server log: >> 22:34:54.948 [info] ssl accepted {{127,0,0,1},53804}, local addr {{127,0,0,1},8443}, suite {tlsv1,{dhe_rsa,aes_256_cbc,sha}} >> 22:34:54.949 [debug] ssl recv {{127,0,0,1},53804} <<"0">> >> 22:34:54.949 [debug] ssl recv {{127,0,0,1},53804} <<"123456789abcdef">> >> >> thanks in advanced, >> Best Regards, Dmitry >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu From ingela.andin@REDACTED Thu Aug 16 09:04:32 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 16 Aug 2012 09:04:32 +0200 Subject: [erlang-questions] SSL behavior!? In-Reply-To: <-4973558989212900051@unknownmsgid> References: <5FB27B14-022E-42DF-9E96-EB361D36A764@gmail.com> <20497694.9144.1345091786711.JavaMail.mobile-sync@vcbjg4> <-4973558989212900051@unknownmsgid> Message-ID: Hi! This behaviour is a security counter measure, 1/n-1 splitting countermeasure Rizzo/Duong-Beast, RC4 chiphers are not vulnerable to this attack so if you use an RC4-cipher suite it will not happen but for the other cipher suites you will have to live with it until TLS-1.1 is supported. Regards Ingela Erlang/OTP team - Ericsson AB 2012/8/16, dmitry kolesnikov : > Hello, > > Yes for sure, you have to deal with IP fragmentation. But, like I said > in my post fragmentation happens on SSL level, two distinguished "App > Data" frames where generated and those frames were put into single > packet. > > I would not raise an issue if I would observe a fragmentation on large > dataset but for 16 byte packet looks suspicious for me, especially of > first fragment is one byte... I've not seen that issue on R14, it pops > up when I jumped to R15 and made slight adjustments my code. > > > Best Regards, > Dmitry >-|-|-*> > > > On 15.8.2012, at 23.38, "Lo?c Hoguin" wrote: > >> Hey, >> >> The same can happen for gen_tcp. You can't assume what you are sending >> isn't going to be fragmented when you receive it, packets have a size >> limit (MTU). You need to either know how much data you are waiting for, >> or try to parse it to validate it got received fully before processing >> it. >> >> On 08/15/2012 09:50 PM, Dmitry Kolesnikov wrote: >>> Hello, >>> >>> I am experience very wired behavior with SSL in R15... So, I do have both >>> client & server implemented on erlang, using ssl in active mode. >>> >>> The implementation is straight forward it >>> >>> Client-side: >>> {ok, Tcp} = gen_tcp:connect(Host, Port, ?SOCK_OPTS, T), >>> {ok, Ssl} = ssl:connect(Tcp, []), >>> >>> Server-side: >>> {ok, LSock} = ssl:listen(Port, [ >>> {ip, IP}, >>> {certfile, Cert}, >>> {keyfile, Key}, >>> {reuseaddr, true} | ?SOCK_OPTS >>> ]), >>> ... >>> {ok, Sock} = ssl:transport_accept(LSock), >>> ok = ssl:ssl_accept(Sock), >>> >>> >>> -define(SOCK_OPTS, [ >>> {active, once}, >>> {mode, binary}, >>> {nodelay, true}, %% BTW, if I drop nodelay or put it to false, same >>> issue >>> {recbuf, 16 * 1024}, >>> {sndbuf, 16 * 1024} >>> ]). >>> >>> >>> My issue is that client send data, the data got fragmented into multiple >>> "Application Data" messages. I've been validating it by sniffing the >>> traffic. I cannot get idea why this happens... >>> >>> Client log: >>> 22:34:54.948 [info] ssl connected {{127,0,0,1},8443}, local addr >>> {{127,0,0,1},53804}, suite {tlsv1,{dhe_rsa,aes_256_cbc,sha}} >>> 22:34:54.948 [debug] ssl send {{127,0,0,1},8443} <<"0123456789abcdef">> >>> >>> Server log: >>> 22:34:54.948 [info] ssl accepted {{127,0,0,1},53804}, local addr >>> {{127,0,0,1},8443}, suite {tlsv1,{dhe_rsa,aes_256_cbc,sha}} >>> 22:34:54.949 [debug] ssl recv {{127,0,0,1},53804} <<"0">> >>> 22:34:54.949 [debug] ssl recv {{127,0,0,1},53804} <<"123456789abcdef">> >>> >>> thanks in advanced, >>> Best Regards, Dmitry >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From zerthurd@REDACTED Thu Aug 16 09:53:20 2012 From: zerthurd@REDACTED (Maxim Treskin) Date: Thu, 16 Aug 2012 14:53:20 +0700 Subject: [erlang-questions] Report: sheriff + dialyzer = hard time for "parse_transform" In-Reply-To: <4FBADF84-EDA7-46E9-AC4A-1D5216E3F670@gmail.com> References: <5FB27B14-022E-42DF-9E96-EB361D36A764@gmail.com> <502C08DB.7030705@ninenines.eu> <502C288C.3000603@ninenines.eu> <4FBADF84-EDA7-46E9-AC4A-1D5216E3F670@gmail.com> Message-ID: Hi Just add path to parse_trans library in ERL_LIBS before run Dialyzer On 16 August 2012 06:15, Zabrane Mickael wrote: > Hi Lo?c, > > > Don't think I ever tried to dialyze it, tests were good enough for that > purpose. This sounds like a parse_trans error, but why would it only occur > on dialyzing? > > Yep for sure. > > > Regardless dialyzer would work fine if operating on compiled files with > debug enabled. > > How can I check that (i.e test only from beams with debug enabled)? > > Regards, > Zabrane > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Max Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Thu Aug 16 10:48:52 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 16 Aug 2012 10:48:52 +0200 Subject: [erlang-questions] Report: sheriff + dialyzer = hard time for "parse_transform" In-Reply-To: References: <5FB27B14-022E-42DF-9E96-EB361D36A764@gmail.com> <502C08DB.7030705@ninenines.eu> <502C288C.3000603@ninenines.eu> <4FBADF84-EDA7-46E9-AC4A-1D5216E3F670@gmail.com> Message-ID: <040B7D3A-CB1E-4DF5-9BF5-684C43C3629D@gmail.com> Hi Maxim, You are right. Changing the following Makefile line fixed the problem: dialyze: ERL_LIBS=$(PWD)/deps $(DIALYZER) --src src --plt .sheriff_dialyzer.plt --no_native \ [...] Lo?c, could you please push this into Sheriff (github)? Regards, Zabrane On Aug 16, 2012, at 9:53 AM, Maxim Treskin wrote: > Hi > > Just add path to parse_trans library in ERL_LIBS before run Dialyzer > > On 16 August 2012 06:15, Zabrane Mickael wrote: > Hi Lo?c, > > > Don't think I ever tried to dialyze it, tests were good enough for that purpose. This sounds like a parse_trans error, but why would it only occur on dialyzing? > > Yep for sure. > > > Regardless dialyzer would work fine if operating on compiled files with debug enabled. > > How can I check that (i.e test only from beams with debug enabled)? > > Regards, > Zabrane > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Max Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Aug 16 13:02:46 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 16 Aug 2012 13:02:46 +0200 Subject: [erlang-questions] Report: sheriff + dialyzer = hard time for "parse_transform" In-Reply-To: <040B7D3A-CB1E-4DF5-9BF5-684C43C3629D@gmail.com> References: <5FB27B14-022E-42DF-9E96-EB361D36A764@gmail.com> <502C08DB.7030705@ninenines.eu> <502C288C.3000603@ninenines.eu> <4FBADF84-EDA7-46E9-AC4A-1D5216E3F670@gmail.com> <040B7D3A-CB1E-4DF5-9BF5-684C43C3629D@gmail.com> Message-ID: <502CD356.6070906@ninenines.eu> Doh! Done. On 08/16/2012 10:48 AM, Zabrane Mickael wrote: > Hi Maxim, > > You are right. Changing the following Makefile line fixed the problem: > > dialyze: > ERL_LIBS=$(PWD)/deps $(DIALYZER) --src src --plt .sheriff_dialyzer.plt > --no_native \ > [...] > > > Lo?c, could you please push this into Sheriff (github)? > > Regards, > Zabrane > > > On Aug 16, 2012, at 9:53 AM, Maxim Treskin wrote: > >> Hi >> >> Just add path to parse_trans library in ERL_LIBS before run Dialyzer >> >> On 16 August 2012 06:15, Zabrane Mickael > > wrote: >> >> Hi Lo?c, >> >> > Don't think I ever tried to dialyze it, tests were good enough >> for that purpose. This sounds like a parse_trans error, but why >> would it only occur on dialyzing? >> >> Yep for sure. >> >> > Regardless dialyzer would work fine if operating on compiled >> files with debug enabled. >> >> How can I check that (i.e test only from beams with debug enabled)? >> >> Regards, >> Zabrane >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> -- >> Max Treskin > > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From raimo+erlang-questions@REDACTED Thu Aug 16 16:22:05 2012 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 16 Aug 2012 16:22:05 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <21F42903-8185-4E78-B419-9DA9512EA294@cs.otago.ac.nz> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A717D.5060702@ninenines.eu> <502A8067.7090607@ninenines.eu> <21F42903-8185-4E78-B419-9DA9512EA294@cs.otago.ac.nz> Message-ID: <20120816142202.GA22866@erix.ericsson.se> On Wed, Aug 15, 2012 at 11:42:06AM +1200, Richard O'Keefe wrote: > > On 15/08/2012, at 4:44 AM, Lo?c Hoguin wrote: > > > On 08/14/2012 06:09 PM, Zabrane Mickael wrote: > >> > >> On Aug 14, 2012, at 5:40 PM, Lo?c Hoguin wrote: > >> > >>> os:cmd("pwgen | cut -c 1-8"). > >> > >> a portable one ;-) > > > > I believe pwgen and cut are available on all of win/linux/bsd/osx. > > m% man pwgen > No manual entry for pwgen > > Mac OS X 10.6.8. > > v% man pwgen > No manual entry for pwgen > > Linux 2.6.31.5-127.fc12.x86_64 > > f% man pwgen > No manual entry for pwgen. > > Solaris. > > Available? May be, I don't know. > Installed? NO. And the same applies to Ubuntu, FreeBSD and OpenBSD... This might be more portable: openssl rand -base64 6 | tr +/ ., > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From dmkolesnikov@REDACTED Thu Aug 16 16:24:34 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Thu, 16 Aug 2012 17:24:34 +0300 Subject: [erlang-questions] SSL behavior!? In-Reply-To: References: <5FB27B14-022E-42DF-9E96-EB361D36A764@gmail.com> <20497694.9144.1345091786711.JavaMail.mobile-sync@vcbjg4> <-4973558989212900051@unknownmsgid> Message-ID: Hello, Thanks a lot for explanation! Regards, Dmitry On Aug 16, 2012, at 10:04 AM, Ingela Andin wrote: > Hi! > > This behaviour is a security counter measure, 1/n-1 splitting > countermeasure Rizzo/Duong-Beast, RC4 chiphers are not vulnerable to > this attack so if you use an RC4-cipher suite it will not happen but > for the other cipher suites you will have to live with it until > TLS-1.1 is supported. > > Regards Ingela Erlang/OTP team - Ericsson AB > > 2012/8/16, dmitry kolesnikov : >> Hello, >> >> Yes for sure, you have to deal with IP fragmentation. But, like I said >> in my post fragmentation happens on SSL level, two distinguished "App >> Data" frames where generated and those frames were put into single >> packet. >> >> I would not raise an issue if I would observe a fragmentation on large >> dataset but for 16 byte packet looks suspicious for me, especially of >> first fragment is one byte... I've not seen that issue on R14, it pops >> up when I jumped to R15 and made slight adjustments my code. >> >> >> Best Regards, >> Dmitry >-|-|-*> >> >> >> On 15.8.2012, at 23.38, "Lo?c Hoguin" wrote: >> >>> Hey, >>> >>> The same can happen for gen_tcp. You can't assume what you are sending >>> isn't going to be fragmented when you receive it, packets have a size >>> limit (MTU). You need to either know how much data you are waiting for, >>> or try to parse it to validate it got received fully before processing >>> it. >>> >>> On 08/15/2012 09:50 PM, Dmitry Kolesnikov wrote: >>>> Hello, >>>> >>>> I am experience very wired behavior with SSL in R15... So, I do have both >>>> client & server implemented on erlang, using ssl in active mode. >>>> >>>> The implementation is straight forward it >>>> >>>> Client-side: >>>> {ok, Tcp} = gen_tcp:connect(Host, Port, ?SOCK_OPTS, T), >>>> {ok, Ssl} = ssl:connect(Tcp, []), >>>> >>>> Server-side: >>>> {ok, LSock} = ssl:listen(Port, [ >>>> {ip, IP}, >>>> {certfile, Cert}, >>>> {keyfile, Key}, >>>> {reuseaddr, true} | ?SOCK_OPTS >>>> ]), >>>> ... >>>> {ok, Sock} = ssl:transport_accept(LSock), >>>> ok = ssl:ssl_accept(Sock), >>>> >>>> >>>> -define(SOCK_OPTS, [ >>>> {active, once}, >>>> {mode, binary}, >>>> {nodelay, true}, %% BTW, if I drop nodelay or put it to false, same >>>> issue >>>> {recbuf, 16 * 1024}, >>>> {sndbuf, 16 * 1024} >>>> ]). >>>> >>>> >>>> My issue is that client send data, the data got fragmented into multiple >>>> "Application Data" messages. I've been validating it by sniffing the >>>> traffic. I cannot get idea why this happens... >>>> >>>> Client log: >>>> 22:34:54.948 [info] ssl connected {{127,0,0,1},8443}, local addr >>>> {{127,0,0,1},53804}, suite {tlsv1,{dhe_rsa,aes_256_cbc,sha}} >>>> 22:34:54.948 [debug] ssl send {{127,0,0,1},8443} <<"0123456789abcdef">> >>>> >>>> Server log: >>>> 22:34:54.948 [info] ssl accepted {{127,0,0,1},53804}, local addr >>>> {{127,0,0,1},8443}, suite {tlsv1,{dhe_rsa,aes_256_cbc,sha}} >>>> 22:34:54.949 [debug] ssl recv {{127,0,0,1},53804} <<"0">> >>>> 22:34:54.949 [debug] ssl recv {{127,0,0,1},53804} <<"123456789abcdef">> >>>> >>>> thanks in advanced, >>>> Best Regards, Dmitry >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> >>> -- >>> Lo?c Hoguin >>> Erlang Cowboy >>> Nine Nines >>> http://ninenines.eu >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From ates@REDACTED Thu Aug 16 16:53:01 2012 From: ates@REDACTED (Artem Teslenko) Date: Thu, 16 Aug 2012 16:53:01 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <20120816142202.GA22866@erix.ericsson.se> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A717D.5060702@ninenines.eu> <502A8067.7090607@ninenines.eu> <21F42903-8185-4E78-B419-9DA9512EA294@cs.otago.ac.nz> <20120816142202.GA22866@erix.ericsson.se> Message-ID: <20120816145301.GB21049@ipv6.dp.ua> One more way to generate the passwords: random:seed(now()). base64:encode_to_string(crypto:rand_bytes(16)). "EbvUy9ZHTqoAOvMSZ53X5A==" After that just remove the unnecessary characters. On Thu, 16 Aug 2012, Raimo Niskanen wrote: > On Wed, Aug 15, 2012 at 11:42:06AM +1200, Richard O'Keefe wrote: > > > > On 15/08/2012, at 4:44 AM, Lo?c Hoguin wrote: > > > > > On 08/14/2012 06:09 PM, Zabrane Mickael wrote: > > >> > > >> On Aug 14, 2012, at 5:40 PM, Lo?c Hoguin wrote: > > >> > > >>> os:cmd("pwgen | cut -c 1-8"). > > >> > > >> a portable one ;-) > > > > > > I believe pwgen and cut are available on all of win/linux/bsd/osx. > > > > m% man pwgen > > No manual entry for pwgen > > > > Mac OS X 10.6.8. > > > > v% man pwgen > > No manual entry for pwgen > > > > Linux 2.6.31.5-127.fc12.x86_64 > > > > f% man pwgen > > No manual entry for pwgen. > > > > Solaris. > > > > Available? May be, I don't know. > > Installed? NO. > > And the same applies to Ubuntu, FreeBSD and OpenBSD... > > This might be more portable: > openssl rand -base64 6 | tr +/ ., > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Thu Aug 16 17:10:53 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 16 Aug 2012 17:10:53 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <20120816145301.GB21049@ipv6.dp.ua> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A717D.5060702@ninenines.eu> <502A8067.7090607@ninenines.eu> <21F42903-8185-4E78-B419-9DA9512EA294@cs.otago.ac.nz> <20120816142202.GA22866@erix.ericsson.se> <20120816145301.GB21049@ipv6.dp.ua> Message-ID: <2C720501-E00C-45B8-9483-87424B82DE1E@gmail.com> Thanks for sharing this Artem. Very neat!! Regards, Zabrane On Aug 16, 2012, at 4:53 PM, Artem Teslenko wrote: > One more way to generate the passwords: > > random:seed(now()). > base64:encode_to_string(crypto:rand_bytes(16)). > > "EbvUy9ZHTqoAOvMSZ53X5A==" From jbarrett@REDACTED Thu Aug 16 17:25:27 2012 From: jbarrett@REDACTED (Jeremey Barrett) Date: Thu, 16 Aug 2012 10:25:27 -0500 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <20120816145301.GB21049@ipv6.dp.ua> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A717D.5060702@ninenines.eu> <502A8067.7090607@ninenines.eu> <21F42903-8185-4E78-B419-9DA9512EA294@cs.otago.ac.nz> <20120816142202.GA22866@erix.ericsson.se> <20120816145301.GB21049@ipv6.dp.ua> Message-ID: On Aug 16, 2012, at 9:53 AM, Artem Teslenko wrote: > One more way to generate the passwords: > > random:seed(now()). > base64:encode_to_string(crypto:rand_bytes(16)). Note: random:seed() is not related to crypto:rand_bytes(). Just the second line will do. Regards, Jeremey. From ates@REDACTED Thu Aug 16 17:54:58 2012 From: ates@REDACTED (Artem Teslenko) Date: Thu, 16 Aug 2012 17:54:58 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A717D.5060702@ninenines.eu> <502A8067.7090607@ninenines.eu> <21F42903-8185-4E78-B419-9DA9512EA294@cs.otago.ac.nz> <20120816142202.GA22866@erix.ericsson.se> <20120816145301.GB21049@ipv6.dp.ua> Message-ID: <20120816155457.GA22005@ipv6.dp.ua> Yes, you are right. The first line is unnecessary. On Thu, 16 Aug 2012, Jeremey Barrett wrote: > On Aug 16, 2012, at 9:53 AM, Artem Teslenko wrote: > > > One more way to generate the passwords: > > > > random:seed(now()). > > base64:encode_to_string(crypto:rand_bytes(16)). > > Note: random:seed() is not related to crypto:rand_bytes(). Just the second line will do. > > Regards, > Jeremey. > From raimo+erlang-questions@REDACTED Thu Aug 16 18:11:56 2012 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 16 Aug 2012 18:11:56 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <20120816145301.GB21049@ipv6.dp.ua> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A717D.5060702@ninenines.eu> <502A8067.7090607@ninenines.eu> <21F42903-8185-4E78-B419-9DA9512EA294@cs.otago.ac.nz> <20120816142202.GA22866@erix.ericsson.se> <20120816145301.GB21049@ipv6.dp.ua> Message-ID: <20120816161156.GA25194@erix.ericsson.se> On Thu, Aug 16, 2012 at 04:53:01PM +0200, Artem Teslenko wrote: > One more way to generate the passwords: > > random:seed(now()). > base64:encode_to_string(crypto:rand_bytes(16)). > > "EbvUy9ZHTqoAOvMSZ53X5A==" > > After that just remove the unnecessary characters. That is almost the same as my command line suggestion but in all erlang. Very nice! If you choose the number of bytes as a multiple of 6 you will not get any trailing `=' and translate any generated Base64 characters `+' and `/' (at least `/' is not popular by some password checkers) into something else, see my example of using `.' and `,' below, then it is the same suggestion. [case X of $+ -> $.; $/ -> $,; X -> X end || X <- base64:encode_to_string(crypto:rand_bytes(12))] -> "dSdCpgl.wuGaMzsr" / Raimo > > On Thu, 16 Aug 2012, Raimo Niskanen wrote: > > > On Wed, Aug 15, 2012 at 11:42:06AM +1200, Richard O'Keefe wrote: > > > > > > On 15/08/2012, at 4:44 AM, Lo?c Hoguin wrote: > > > > > > > On 08/14/2012 06:09 PM, Zabrane Mickael wrote: > > > >> > > > >> On Aug 14, 2012, at 5:40 PM, Lo?c Hoguin wrote: > > > >> > > > >>> os:cmd("pwgen | cut -c 1-8"). > > > >> > > > >> a portable one ;-) > > > > > > > > I believe pwgen and cut are available on all of win/linux/bsd/osx. > > > > > > m% man pwgen > > > No manual entry for pwgen > > > > > > Mac OS X 10.6.8. > > > > > > v% man pwgen > > > No manual entry for pwgen > > > > > > Linux 2.6.31.5-127.fc12.x86_64 > > > > > > f% man pwgen > > > No manual entry for pwgen. > > > > > > Solaris. > > > > > > Available? May be, I don't know. > > > Installed? NO. > > > > And the same applies to Ubuntu, FreeBSD and OpenBSD... > > > > This might be more portable: > > openssl rand -base64 6 | tr +/ ., > > > > > > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From rexxe98@REDACTED Fri Aug 17 00:35:49 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Thu, 16 Aug 2012 15:35:49 -0700 Subject: [erlang-questions] Sock.js and Cowboy - Example In-Reply-To: <50295194.4020700@gmail.com> References: <50295194.4020700@gmail.com> Message-ID: There's no reason to use ZeroMQ and Erlang together. Erlang provides the same functionality out of the box but has a different way of doing it: See these articles: 1. http://jlouisramblings.blogspot.com/2011/10/one-major-difference-zeromq-and-erlang.html 2. http://www.rabbitmq.com/blog/2011/06/30/zeromq-erlang/ I'd recommend looking at gproc. --Andrew On Mon, Aug 13, 2012 at 12:12 PM, Michael Truog wrote: > ** > On 08/13/2012 10:59 AM, Jai Gupta wrote: > > > Here's the answer to your question on SockJS mailing list: > > https://groups.google.com/d/msg/sockjs/m5UtBVcmJ_g/0M92khnAGmgJ > > > Thank you Marek for your detailed answer. > > Both Sock.js and Cowboy are great projects and it was good experience of > having opportunity of using them. > > I am trying to write some reasons so that you and can give your > suggestions. > > - We aim to build a benchmarking client for performance analysis. This > tool should be able to perform all sorts of actives that any user might do. > Both, with SockJS & Cowboy do not have a client which I could use. SocketIO > had a client which could be used in Node.js. However, SocketIO port of > Erlang is looking as if it is not in active development. > - Coming from C++/PHP background Erlang language is looking difficult. > Working on Node.js was natural but performance of Node.js is not good. In > our benchmark Node.js was performing 40k msg/sec using SockJS module and > around 60k msg/sec when using ZeroMQ. Even PHP gives 400K msg/sec with > ZeroMQ (although it has leaking memory > ). > > We are trying some basic prototype on other languages. > > > It is much better if you use https://github.com/zeromq/erlzmq2 for > Erlang<->ZeroMQ integration. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri Aug 17 01:58:51 2012 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 16 Aug 2012 16:58:51 -0700 Subject: [erlang-questions] Sock.js and Cowboy - Example In-Reply-To: References: <50295194.4020700@gmail.com> Message-ID: <502D893B.1090605@gmail.com> There are reasons to use ZeroMQ and Erlang together. A few, just thought of briefly are: 1) A quick way to integrate with external languages (non-Erlang languages) 2) A way of avoiding the limitations of distributed Erlang and its net tick time I won't bother to list the problems with using gproc, since that is discussed elsewhere (where it remains on-topic). On 08/16/2012 03:35 PM, Andrew Berman wrote: > There's no reason to use ZeroMQ and Erlang together. Erlang provides the same functionality out of the box but has a different way of doing it: > > See these articles: > > 1. http://jlouisramblings.blogspot.com/2011/10/one-major-difference-zeromq-and-erlang.html > 2. http://www.rabbitmq.com/blog/2011/06/30/zeromq-erlang/ > > I'd recommend looking at gproc. > > --Andrew > > On Mon, Aug 13, 2012 at 12:12 PM, Michael Truog > wrote: > > On 08/13/2012 10:59 AM, Jai Gupta wrote: >> >> Here's the answer to your question on SockJS mailing list: >> >> https://groups.google.com/d/msg/sockjs/m5UtBVcmJ_g/0M92khnAGmgJ >> >> >> Thank you Marek for your detailed answer. >> >> Both Sock.js and Cowboy are great projects and it was good experience of having opportunity of using them. >> >> I am trying to write some reasons so that you and can give your suggestions. >> >> * We aim to build a benchmarking client for performance analysis. This tool should be able to perform all sorts of actives that any user might do. Both, with SockJS & Cowboy do not have a client which I could use. SocketIO had a client which could be used in Node.js. However, SocketIO port of Erlang is looking as if it is not in active development. >> * Coming from C++/PHP background Erlang language is looking difficult. Working on Node.js was natural but performance of Node.js is not good. In our benchmark Node.js was performing 40k msg/sec using SockJS module and around 60k msg/sec when using ZeroMQ. Even PHP gives 400K msg/sec with ZeroMQ (although it has leaking memory ). >> >> We are trying some basic prototype on other languages. >> >> > > It is much better if you use https://github.com/zeromq/erlzmq2 for Erlang<->ZeroMQ integration. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthony.hw.kong@REDACTED Fri Aug 17 08:55:59 2012 From: anthony.hw.kong@REDACTED (Anthony Kong) Date: Fri, 17 Aug 2012 16:55:59 +1000 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <20120816145301.GB21049@ipv6.dp.ua> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A717D.5060702@ninenines.eu> <502A8067.7090607@ninenines.eu> <21F42903-8185-4E78-B419-9DA9512EA294@cs.otago.ac.nz> <20120816142202.GA22866@erix.ericsson.se> <20120816145301.GB21049@ipv6.dp.ua> Message-ID: Hi, Artem, When I use random::seed, I saw a 'undefined' output. Is it expected? $ erl Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.5 (abort with ^G) 1> random:seed(now()). undefined On Fri, Aug 17, 2012 at 12:53 AM, Artem Teslenko wrote: > One more way to generate the passwords: > > random:seed(now()). > base64:encode_to_string(crypto:rand_bytes(16)). > > "EbvUy9ZHTqoAOvMSZ53X5A==" > > After that just remove the unnecessary characters. > > On Thu, 16 Aug 2012, Raimo Niskanen wrote: > > > On Wed, Aug 15, 2012 at 11:42:06AM +1200, Richard O'Keefe wrote: > > > > > > On 15/08/2012, at 4:44 AM, Lo?c Hoguin wrote: > > > > > > > On 08/14/2012 06:09 PM, Zabrane Mickael wrote: > > > >> > > > >> On Aug 14, 2012, at 5:40 PM, Lo?c Hoguin wrote: > > > >> > > > >>> os:cmd("pwgen | cut -c 1-8"). > > > >> > > > >> a portable one ;-) > > > > > > > > I believe pwgen and cut are available on all of win/linux/bsd/osx. > > > > > > m% man pwgen > > > No manual entry for pwgen > > > > > > Mac OS X 10.6.8. > > > > > > v% man pwgen > > > No manual entry for pwgen > > > > > > Linux 2.6.31.5-127.fc12.x86_64 > > > > > > f% man pwgen > > > No manual entry for pwgen. > > > > > > Solaris. > > > > > > Available? May be, I don't know. > > > Installed? NO. > > > > And the same applies to Ubuntu, FreeBSD and OpenBSD... > > > > This might be more portable: > > openssl rand -base64 6 | tr +/ ., > > > > > > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ates@REDACTED Fri Aug 17 09:03:39 2012 From: ates@REDACTED (Artem Teslenko) Date: Fri, 17 Aug 2012 09:03:39 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A717D.5060702@ninenines.eu> <502A8067.7090607@ninenines.eu> <21F42903-8185-4E78-B419-9DA9512EA294@cs.otago.ac.nz> <20120816142202.GA22866@erix.ericsson.se> <20120816145301.GB21049@ipv6.dp.ua> Message-ID: <20120817070339.GA29838@ipv6.dp.ua> Hello Anthony, Yes, it's ok, from the man page: seed(X1 :: {A1, A2, A3}) -> undefined | ran() undefined value is returned when you run seed/1 first time. Anyway it's unnecessary for crypto:rand_bytes(16). On Fri, 17 Aug 2012, Anthony Kong wrote: > Hi, Artem, > > When I use random::seed, I saw a 'undefined' output. Is it expected? > > > $ erl > Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.5 (abort with ^G) > 1> random:seed(now()). > undefined > > > > > On Fri, Aug 17, 2012 at 12:53 AM, Artem Teslenko wrote: > > > One more way to generate the passwords: > > > > random:seed(now()). > > base64:encode_to_string(crypto:rand_bytes(16)). > > > > "EbvUy9ZHTqoAOvMSZ53X5A==" > > > > After that just remove the unnecessary characters. > > > > On Thu, 16 Aug 2012, Raimo Niskanen wrote: > > > > > On Wed, Aug 15, 2012 at 11:42:06AM +1200, Richard O'Keefe wrote: > > > > > > > > On 15/08/2012, at 4:44 AM, Lo?c Hoguin wrote: > > > > > > > > > On 08/14/2012 06:09 PM, Zabrane Mickael wrote: > > > > >> > > > > >> On Aug 14, 2012, at 5:40 PM, Lo?c Hoguin wrote: > > > > >> > > > > >>> os:cmd("pwgen | cut -c 1-8"). > > > > >> > > > > >> a portable one ;-) > > > > > > > > > > I believe pwgen and cut are available on all of win/linux/bsd/osx. > > > > > > > > m% man pwgen > > > > No manual entry for pwgen > > > > > > > > Mac OS X 10.6.8. > > > > > > > > v% man pwgen > > > > No manual entry for pwgen > > > > > > > > Linux 2.6.31.5-127.fc12.x86_64 > > > > > > > > f% man pwgen > > > > No manual entry for pwgen. > > > > > > > > Solaris. > > > > > > > > Available? May be, I don't know. > > > > Installed? NO. > > > > > > And the same applies to Ubuntu, FreeBSD and OpenBSD... > > > > > > This might be more portable: > > > openssl rand -base64 6 | tr +/ ., > > > > > > > > > > > > > > > _______________________________________________ > > > > erlang-questions mailing list > > > > erlang-questions@REDACTED > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > From samuelrivas@REDACTED Fri Aug 17 09:04:34 2012 From: samuelrivas@REDACTED (Samuel) Date: Fri, 17 Aug 2012 09:04:34 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> Message-ID: > Thank Serker. > > Moving andom:seed(A1, A2, A3) at start fix it. > > 1> passwd:test(). > Generate 10000 random password and check for collisions ... > Number of collisions: 0 There are a number of problems with both implementations. First, random:uniform is not cryptographically secure, which means is somewhat predictable. As already mentioned use any other generator meant to be secure as the one in crypto or the ssl library. Even using a secure pseudrandom generator: Your first implementation destroys the security, as you are creating a seed for each random number an attacker just needs to guess the seed sequence, not the pseudorandom sequence. In your case you had a side effect of generating collisions, but that was not the worst problem. The second implementation is more secure in that sense, but still the original seed is guessable. An attacker can generate possible password sequences by bruteforce just tying possible now tuples around the time he thinks the real seed was created. So, if you want to create passwords difficult to guess, you need at least a cryptographically secure PRG, which will give you an unpredictable sequence of bytes, and an unguessable seed, which will give prevent any attacker from creating the same sequence of bytes again an completely break all your passwords. Best -- Samuel From zhang.heng16@REDACTED Fri Aug 17 09:15:38 2012 From: zhang.heng16@REDACTED (zhang.heng16@REDACTED) Date: Fri, 17 Aug 2012 15:15:38 +0800 Subject: [erlang-questions] installation error Message-ID: Hi, this mail is from Jeff in China.I need your professional help. The below is my problem?? In the stage of installation, i received some mistake notice at my first time using erlang just as followings: ********************************************************************* ********************** APPLICATIONS DISABLED ********************** ********************************************************************* odbc : ODBC library - link check failed ********************************************************************* ********************************************************************* ********************** APPLICATIONS INFORMATION ******************* ********************************************************************* wx : wxWidgets not found, wx will NOT be usable ********************************************************************* ********************************************************************* ********************** DOCUMENTATION INFORMATION ****************** ********************************************************************* documentation : fop is missing. Using fakefop to generate placeholder PDF files. ********************************************************************* It seemed that my system lacked odbc library. However, i have installed unixODBC on the system yet. I tried to search some solutions from the internet. But there lie no effective solutions. I doubt whether a solution someone proposed is the best way, that I need to install the following 5 packages:libc6-dev,libssl0.9.8,libssl-dev,libncurses5 and libncurses5-dev. In order to get the best solution, i write this help mail.I wish you could analyze the real problem and what i should do next. PS.My system is Redhat Enterprise Linux As 4. And the version of unixODBC is 2.3.1. Wish to get a quich response! Best Regards Jeff -------------------------------------------------------- ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s). If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited. If you have received this mail in error, please delete it and notify us immediately. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Fri Aug 17 09:34:01 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 17 Aug 2012 09:34:01 +0200 Subject: [erlang-questions] installation error In-Reply-To: References: Message-ID: Hi Zhang, Two ways: 1. You can skip ODBC with this: touch lib/odbc/SKIP # ./configure [...] && make & make install 2. Install UnixODBC before: # yum -y install unixODBC unixODBC-devel <-- Feodra/Redhat/CentOS # apt-get install unixodbc unixodbc-dev <- Debian/Ubuntu ... # ./configure [...] && make & make install Regards, Zabrane On Aug 17, 2012, at 9:15 AM, zhang.heng16@REDACTED wrote: > > Hi, > > this mail is from Jeff in China.I need your professional help. The below is my problem?? > > In the stage of installation, i received some mistake notice at my first time using erlang just as followings: > > ********************************************************************* > ********************** APPLICATIONS DISABLED ********************** > ********************************************************************* > > odbc : ODBC library - link check failed > > ********************************************************************* > ********************************************************************* > ********************** APPLICATIONS INFORMATION ******************* > ********************************************************************* > > wx : wxWidgets not found, wx will NOT be usable > > ********************************************************************* > ********************************************************************* > ********************** DOCUMENTATION INFORMATION ****************** > ********************************************************************* > > documentation : > fop is missing. > Using fakefop to generate placeholder PDF files. > > ********************************************************************* > > It seemed that my system lacked odbc library. However, i have installed unixODBC on the system yet. I tried to search some solutions from the internet. But there lie no effective solutions. I doubt whether a solution someone proposed is the best way, that I need to install the following 5 packages:libc6-dev,libssl0.9.8,libssl-dev,libncurses5 and libncurses5-dev. > > In order to get the best solution, i write this help mail.I wish you could analyze the real problem and what i should do next. > PS.My system is Redhat Enterprise Linux As 4. And the version of unixODBC is 2.3.1. > > Wish to get a quich response! > > Best Regards > > Jeff > > > -------------------------------------------------------- > ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s). If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited. If you have received this mail in error, please delete it and notify us immediately. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgsmcmlxxv@REDACTED Fri Aug 17 09:34:48 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Fri, 17 Aug 2012 09:34:48 +0200 Subject: [erlang-questions] installation error In-Reply-To: References: Message-ID: You need unixODBC headers (development package). The simple unixODBC library won't suffice. Hope it will help. CGS On Fri, Aug 17, 2012 at 9:15 AM, wrote: > > Hi, > > this mail is from Jeff in China.I need your professional help. The below > is my problem?? > > In the stage of installation, i received some mistake notice at my first > time using erlang just as followings: > > ********************************************************************* > ********************** APPLICATIONS DISABLED ********************** > ********************************************************************* > > odbc : ODBC library - link check failed > > ********************************************************************* > ********************************************************************* > ********************** APPLICATIONS INFORMATION ******************* > ********************************************************************* > > wx : wxWidgets not found, wx will NOT be usable > > ********************************************************************* > ********************************************************************* > ********************** DOCUMENTATION INFORMATION ****************** > ********************************************************************* > > documentation : > fop is missing. > Using fakefop to generate placeholder PDF files. > > ********************************************************************* > > It seemed that my system lacked odbc library. However, i have installed > unixODBC on the system yet. I tried to search some solutions from the > internet. But there lie no effective solutions. I doubt whether a solution > someone proposed is the best way, that I need to install the following 5 > packages:libc6-dev,libssl0.9.8,libssl-dev,libncurses5 and libncurses5-dev. > > In order to get the best solution, i write this help mail.I wish you could > analyze the real problem and what i should do next. > PS.My system is Redhat Enterprise Linux As 4. And the version of unixODBC > is 2.3.1. > > Wish to get a quich response! > > Best Regards > > Jeff > > > -------------------------------------------------------- > ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s). If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited. If you have received this mail in error, please delete it and notify us immediately. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Fri Aug 17 09:44:05 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 17 Aug 2012 09:44:05 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> Message-ID: Hi Samuel, Thanks for these useful insights. We're not a security software company and our usage of these passwords isn't critical. Anyway, we want to learn if possible. > First, random:uniform is not cryptographically secure, which means is > somewhat predictable. As already mentioned use any other generator > meant to be secure as the one in crypto or the ssl library. > Even using a secure pseudrandom generator: > > Your first implementation destroys the security, as you are creating a > seed for each random number an attacker just needs to guess the seed > sequence, not the pseudorandom sequence. In your case you had a side > effect of generating collisions, but that was not the worst problem. Yup. Sverker response was very clear to this regard! > The second implementation is more secure in that sense, but still the > original seed is guessable. An attacker can generate possible password > sequences by bruteforce just tying possible now tuples around the time > he thinks the real seed was created. So, how one can generate a secure un-predicatable seeds? > So, if you want to create passwords difficult to guess, you need at > least a cryptographically secure PRG, which will give you an > unpredictable sequence of bytes, and an unguessable seed, which will > give prevent any attacker from creating the same sequence of bytes > again an completely break all your passwords. We also moved to "Tiny Mersenne Twister" (https://github.com/jj1bdx/tinymt-erlang) instead of using the standard random:uniform since the last Yaws security alert (http://erlang.org/pipermail/erlang-questions/2012-June/067626.html). Is this sufficient or should we also find a way to generate a unpredicatble seed for it? Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgsmcmlxxv@REDACTED Fri Aug 17 10:12:52 2012 From: cgsmcmlxxv@REDACTED (CGS) Date: Fri, 17 Aug 2012 10:12:52 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> Message-ID: Hi Zabrane, If you want just a well distributed RNG with long period, you may take a look at WELL RNG (http://www.iro.umontreal.ca/~panneton/WELLRNG.html) as well. I also needed that and I had an attempt to translate it in Erlang, so, you may use it if you think fit ( https://github.com/cgsmcmlxxv/WELL44497ac). It is recommended for Monte Carlo simulations and less for cryptography, though. CGS On Fri, Aug 17, 2012 at 9:44 AM, Zabrane Mickael wrote: > Hi Samuel, > > Thanks for these useful insights. > > We're not a security software company and our usage of these passwords > isn't critical. > Anyway, we want to learn if possible. > > First, random:uniform is not cryptographically secure, which means is > somewhat predictable. As already mentioned use any other generator > meant to be secure as the one in crypto or the ssl library. > > Even using a secure pseudrandom generator: > > Your first implementation destroys the security, as you are creating a > seed for each random number an attacker just needs to guess the seed > sequence, not the pseudorandom sequence. In your case you had a side > effect of generating collisions, but that was not the worst problem. > > > Yup. Sverker response was very clear to this regard! > > The second implementation is more secure in that sense, but still the > original seed is guessable. An attacker can generate possible password > sequences by bruteforce just tying possible now tuples around the time > he thinks the real seed was created. > > > So, how one can generate a secure un-predicatable seeds? > > So, if you want to create passwords difficult to guess, you need at > least a cryptographically secure PRG, which will give you an > unpredictable sequence of bytes, and an unguessable seed, which will > give prevent any attacker from creating the same sequence of bytes > again an completely break all your passwords. > > > We also moved to "Tiny Mersenne Twister" ( > https://github.com/jj1bdx/tinymt-erlang) instead of using > the standard random:uniform since the last Yaws security alert ( > http://erlang.org/pipermail/erlang-questions/2012-June/067626.html). > Is this sufficient or should we also find a way to generate a > unpredicatble seed for it? > > Regards, > Zabrane > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Fri Aug 17 10:39:29 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 17 Aug 2012 10:39:29 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> Message-ID: Hey CGS, Let me give it a try ;-) Regards, Zabrane On Aug 17, 2012, at 10:12 AM, CGS wrote: > Hi Zabrane, > > If you want just a well distributed RNG with long period, you may take a look at WELL RNG (http://www.iro.umontreal.ca/~panneton/WELLRNG.html) as well. I also needed that and I had an attempt to translate it in Erlang, so, you may use it if you think fit (https://github.com/cgsmcmlxxv/WELL44497ac). It is recommended for Monte Carlo simulations and less for cryptography, though. > > CGS -------------- next part -------------- An HTML attachment was scrubbed... URL: From samuelrivas@REDACTED Fri Aug 17 12:53:53 2012 From: samuelrivas@REDACTED (Samuel) Date: Fri, 17 Aug 2012 12:53:53 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> Message-ID: > The second implementation is more secure in that sense, but still the > original seed is guessable. An attacker can generate possible password > sequences by bruteforce just tying possible now tuples around the time > he thinks the real seed was created. > > > So, how one can generate a secure un-predicatable seeds? That's the tricky part :) At least you have to avoid generating clearly predictable seeds as the seed is your private key in this case. With the seed anyone can reproduce the sequence. crypto:strong_rand_bytes strives for better security properties, and I understand it abstracts how to generate a good key for you, trying to suck entropy from your system (so you may need to sit there banging the keys and moving the mouse around for that ;) ) I am not a security expert by far, I just know some things that do not work :). For things that work, the common approach is relying in popular libraries not known to be broken. and trying not to use them in a fancy way as the history is full of famous broken cryptographic uses (you can read about flaws CSS, WEP, etc). Of course, whether that approach is advisable or not is more a philosophical question, not knowing they are not broken doesn't mean that no one knows how to break them and has the key access information is thought to be safely encrypted :) > We also moved to "Tiny Mersenne Twister" > (https://github.com/jj1bdx/tinymt-erlang) instead of using > the standard random:uniform since the last Yaws security alert > (http://erlang.org/pipermail/erlang-questions/2012-June/067626.html). > Is this sufficient or should we also find a way to generate a unpredicatble > seed for it? As said, I am not a security expert, but as far as I can read, the goals of that algorithm are to keep a small state with good statistical properties, it says nothing about security (which doesn't necessarily mean it is insecure, of course). A PRG can have good statistical properties and still be insecure, being a secure PRG is a stronger assumption. That is why erlang:random is fine for non cryptographic uses, but for security you need something more complex. Anyway, you always need a seed no one can guess. Same seed, same sequence, so if someone guesses your seed it basically gets all your passwords in return. Regards -- Samuel From zabrane3@REDACTED Fri Aug 17 16:38:06 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 17 Aug 2012 16:38:06 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> Message-ID: Hi Samuel, Great feedbacks. I'll try to find out how to generate good seeds. May be some braves Erlangers know the answer? Regards, Zabrane On Aug 17, 2012, at 12:53 PM, Samuel wrote: >> The second implementation is more secure in that sense, but still the >> original seed is guessable. An attacker can generate possible password >> sequences by bruteforce just tying possible now tuples around the time >> he thinks the real seed was created. >> >> >> So, how one can generate a secure un-predicatable seeds? > > That's the tricky part :) At least you have to avoid generating > clearly predictable seeds as the seed is your private key in this > case. With the seed anyone can reproduce the sequence. > > crypto:strong_rand_bytes strives for better security properties, and I > understand it abstracts how to generate a good key for you, trying to > suck entropy from your system (so you may need to sit there banging > the keys and moving the mouse around for that ;) ) > > I am not a security expert by far, I just know some things that do not > work :). For things that work, the common approach is relying in > popular libraries not known to be broken. and trying not to use them > in a fancy way as the history is full of famous broken cryptographic > uses (you can read about flaws CSS, WEP, etc). > > Of course, whether that approach is advisable or not is more a > philosophical question, not knowing they are not broken doesn't mean > that no one knows how to break them and has the key access information > is thought to be safely encrypted :) > >> We also moved to "Tiny Mersenne Twister" >> (https://github.com/jj1bdx/tinymt-erlang) instead of using >> the standard random:uniform since the last Yaws security alert >> (http://erlang.org/pipermail/erlang-questions/2012-June/067626.html). >> Is this sufficient or should we also find a way to generate a unpredicatble >> seed for it? > > As said, I am not a security expert, but as far as I can read, the > goals of that algorithm are to keep a small state with good > statistical properties, it says nothing about security (which doesn't > necessarily mean it is insecure, of course). A PRG can have good > statistical properties and still be insecure, being a secure PRG is a > stronger assumption. That is why erlang:random is fine for non > cryptographic uses, but for security you need something more complex. > > Anyway, you always need a seed no one can guess. Same seed, same > sequence, so if someone guesses your seed it basically gets all your > passwords in return. > > Regards > -- > Samuel From raimo+erlang-questions@REDACTED Fri Aug 17 16:56:13 2012 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 17 Aug 2012 16:56:13 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> Message-ID: <20120817145613.GA15794@erix.ericsson.se> On Fri, Aug 17, 2012 at 04:38:06PM +0200, Zabrane Mickael wrote: > Hi Samuel, > > Great feedbacks. > > I'll try to find out how to generate good seeds. > May be some braves Erlangers know the answer? The simple answer is as mentioned before in this thread to use crypto:strong_rand_bytes/1, as it will with OS and library support take care of the seeding itself. This will be what is considered secure enough by the writers of OpenSSL. strong_rand_bytes(N) -> binary() Types: N = integer() Generates N bytes randomly uniform 0..255, and returns the result in a binary. Uses a cryptographically secure prng seeded and periodically mixed with operating system provided entropy. By default this is the RAND_bytes method from OpenSSL. May throw exception low_entropy in case the random generator failed due to lack of secure "randomness". / Raimo > > Regards, > Zabrane > > On Aug 17, 2012, at 12:53 PM, Samuel wrote: > > >> The second implementation is more secure in that sense, but still the > >> original seed is guessable. An attacker can generate possible password > >> sequences by bruteforce just tying possible now tuples around the time > >> he thinks the real seed was created. > >> > >> > >> So, how one can generate a secure un-predicatable seeds? > > > > That's the tricky part :) At least you have to avoid generating > > clearly predictable seeds as the seed is your private key in this > > case. With the seed anyone can reproduce the sequence. > > > > crypto:strong_rand_bytes strives for better security properties, and I > > understand it abstracts how to generate a good key for you, trying to > > suck entropy from your system (so you may need to sit there banging > > the keys and moving the mouse around for that ;) ) > > > > I am not a security expert by far, I just know some things that do not > > work :). For things that work, the common approach is relying in > > popular libraries not known to be broken. and trying not to use them > > in a fancy way as the history is full of famous broken cryptographic > > uses (you can read about flaws CSS, WEP, etc). > > > > Of course, whether that approach is advisable or not is more a > > philosophical question, not knowing they are not broken doesn't mean > > that no one knows how to break them and has the key access information > > is thought to be safely encrypted :) > > > >> We also moved to "Tiny Mersenne Twister" > >> (https://github.com/jj1bdx/tinymt-erlang) instead of using > >> the standard random:uniform since the last Yaws security alert > >> (http://erlang.org/pipermail/erlang-questions/2012-June/067626.html). > >> Is this sufficient or should we also find a way to generate a unpredicatble > >> seed for it? > > > > As said, I am not a security expert, but as far as I can read, the > > goals of that algorithm are to keep a small state with good > > statistical properties, it says nothing about security (which doesn't > > necessarily mean it is insecure, of course). A PRG can have good > > statistical properties and still be insecure, being a secure PRG is a > > stronger assumption. That is why erlang:random is fine for non > > cryptographic uses, but for security you need something more complex. > > > > Anyway, you always need a seed no one can guess. Same seed, same > > sequence, so if someone guesses your seed it basically gets all your > > passwords in return. > > > > Regards > > -- > > Samuel > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From jbarrett@REDACTED Fri Aug 17 17:00:03 2012 From: jbarrett@REDACTED (Jeremey Barrett) Date: Fri, 17 Aug 2012 10:00:03 -0500 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> Message-ID: <528E5E49-4B89-4BF8-AF5B-35FCCE782392@alertlogic.com> Hi all... if you want pseudo-random bytes, use crypto:rand_bytes() or crypto:strong_rand_bytes(), depending on your security requirements. rand_bytes() calls OpenSSL's RAND_pseudo_bytes(), and strong_rand_bytes() calls OpenSSL's RAND_bytes(). Read up on them for more info. strong_rand_bytes() will return an error if insufficient entropy is present. rand_bytes() will "just do it", which may not be what you want. OpenSSL is widely deployed, trusted, etc. The built-in wrappers in Erlang are very convenient, just use them. The crypto community has been over and over this for two decades. I cannot stress enough the value of just using a trusted, open implementation vs. fretting over details that may or may not be relevant. There are so many factors you cannot account for otherwise. Regards, Jeremey. On Aug 17, 2012, at 5:53 AM, Samuel wrote: >> The second implementation is more secure in that sense, but still the >> original seed is guessable. An attacker can generate possible password >> sequences by bruteforce just tying possible now tuples around the time >> he thinks the real seed was created. >> >> >> So, how one can generate a secure un-predicatable seeds? > > That's the tricky part :) At least you have to avoid generating > clearly predictable seeds as the seed is your private key in this > case. With the seed anyone can reproduce the sequence. > > crypto:strong_rand_bytes strives for better security properties, and I > understand it abstracts how to generate a good key for you, trying to > suck entropy from your system (so you may need to sit there banging > the keys and moving the mouse around for that ;) ) > > I am not a security expert by far, I just know some things that do not > work :). For things that work, the common approach is relying in > popular libraries not known to be broken. and trying not to use them > in a fancy way as the history is full of famous broken cryptographic > uses (you can read about flaws CSS, WEP, etc). > > Of course, whether that approach is advisable or not is more a > philosophical question, not knowing they are not broken doesn't mean > that no one knows how to break them and has the key access information > is thought to be safely encrypted :) > >> We also moved to "Tiny Mersenne Twister" >> (https://github.com/jj1bdx/tinymt-erlang) instead of using >> the standard random:uniform since the last Yaws security alert >> (http://erlang.org/pipermail/erlang-questions/2012-June/067626.html). >> Is this sufficient or should we also find a way to generate a unpredicatble >> seed for it? > > As said, I am not a security expert, but as far as I can read, the > goals of that algorithm are to keep a small state with good > statistical properties, it says nothing about security (which doesn't > necessarily mean it is insecure, of course). A PRG can have good > statistical properties and still be insecure, being a secure PRG is a > stronger assumption. That is why erlang:random is fine for non > cryptographic uses, but for security you need something more complex. > > Anyway, you always need a seed no one can guess. Same seed, same > sequence, so if someone guesses your seed it basically gets all your > passwords in return. > > Regards > -- > Samuel > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Fri Aug 17 17:04:38 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 17 Aug 2012 17:04:38 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <528E5E49-4B89-4BF8-AF5B-35FCCE782392@alertlogic.com> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> <528E5E49-4B89-4BF8-AF5B-35FCCE782392@alertlogic.com> Message-ID: <0438C996-9997-43EE-AEBA-23E10820BC02@gmail.com> Thanks guys. I'll give "crypto:strong_rand_bytes" a try ;-) Regards, Zabrane On Aug 17, 2012, at 5:00 PM, Jeremey Barrett wrote: > Hi all... if you want pseudo-random bytes, use crypto:rand_bytes() or crypto:strong_rand_bytes(), depending on your security requirements. rand_bytes() calls OpenSSL's RAND_pseudo_bytes(), and strong_rand_bytes() calls OpenSSL's RAND_bytes(). Read up on them for more info. > > strong_rand_bytes() will return an error if insufficient entropy is present. rand_bytes() will "just do it", which may not be what you want. > > OpenSSL is widely deployed, trusted, etc. The built-in wrappers in Erlang are very convenient, just use them. The crypto community has been over and over this for two decades. I cannot stress enough the value of just using a trusted, open implementation vs. fretting over details that may or may not be relevant. There are so many factors you cannot account for otherwise. > > Regards, > Jeremey. > > > On Aug 17, 2012, at 5:53 AM, Samuel wrote: > >>> The second implementation is more secure in that sense, but still the >>> original seed is guessable. An attacker can generate possible password >>> sequences by bruteforce just tying possible now tuples around the time >>> he thinks the real seed was created. >>> >>> >>> So, how one can generate a secure un-predicatable seeds? >> >> That's the tricky part :) At least you have to avoid generating >> clearly predictable seeds as the seed is your private key in this >> case. With the seed anyone can reproduce the sequence. >> >> crypto:strong_rand_bytes strives for better security properties, and I >> understand it abstracts how to generate a good key for you, trying to >> suck entropy from your system (so you may need to sit there banging >> the keys and moving the mouse around for that ;) ) >> >> I am not a security expert by far, I just know some things that do not >> work :). For things that work, the common approach is relying in >> popular libraries not known to be broken. and trying not to use them >> in a fancy way as the history is full of famous broken cryptographic >> uses (you can read about flaws CSS, WEP, etc). >> >> Of course, whether that approach is advisable or not is more a >> philosophical question, not knowing they are not broken doesn't mean >> that no one knows how to break them and has the key access information >> is thought to be safely encrypted :) >> >>> We also moved to "Tiny Mersenne Twister" >>> (https://github.com/jj1bdx/tinymt-erlang) instead of using >>> the standard random:uniform since the last Yaws security alert >>> (http://erlang.org/pipermail/erlang-questions/2012-June/067626.html). >>> Is this sufficient or should we also find a way to generate a unpredicatble >>> seed for it? >> >> As said, I am not a security expert, but as far as I can read, the >> goals of that algorithm are to keep a small state with good >> statistical properties, it says nothing about security (which doesn't >> necessarily mean it is insecure, of course). A PRG can have good >> statistical properties and still be insecure, being a secure PRG is a >> stronger assumption. That is why erlang:random is fine for non >> cryptographic uses, but for security you need something more complex. >> >> Anyway, you always need a seed no one can guess. Same seed, same >> sequence, so if someone guesses your seed it basically gets all your >> passwords in return. >> >> Regards >> -- >> Samuel >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From torben.lehoff@REDACTED Fri Aug 17 21:33:08 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 17 Aug 2012 21:33:08 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <0438C996-9997-43EE-AEBA-23E10820BC02@gmail.com> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> <528E5E49-4B89-4BF8-AF5B-35FCCE782392@alertlogic.com> <0438C996-9997-43EE-AEBA-23E10820BC02@gmail.com> Message-ID: <502E9C74.70709@gmail.com> One thing to keep in mind with crypto is that it depends on OpenSSL which means that if you want to create a binary version of your application you need to make sure that the execution platform has OpenSSL. If you are developing a server for your own use - no problem. If you want to break the dependency you can - on Un*x-like platforms - resort to getting a seed from /dev/urandom - there is an example of this in https://github.com/lehoff/cryptographic, it is ages since I worked on that and I cannot remember the state of that, but the general principle should be sound. There is a way to do this in Windows too, but we never got around to implementing that back then. Cheers, ___ /orben On 2012-08-17 17:04, Zabrane Mickael wrote: > Thanks guys. I'll give "crypto:strong_rand_bytes" a try ;-) > > Regards, > Zabrane > > > On Aug 17, 2012, at 5:00 PM, Jeremey Barrett wrote: > >> Hi all... if you want pseudo-random bytes, use crypto:rand_bytes() or crypto:strong_rand_bytes(), depending on your security requirements. rand_bytes() calls OpenSSL's RAND_pseudo_bytes(), and strong_rand_bytes() calls OpenSSL's RAND_bytes(). Read up on them for more info. >> >> strong_rand_bytes() will return an error if insufficient entropy is present. rand_bytes() will "just do it", which may not be what you want. >> >> OpenSSL is widely deployed, trusted, etc. The built-in wrappers in Erlang are very convenient, just use them. The crypto community has been over and over this for two decades. I cannot stress enough the value of just using a trusted, open implementation vs. fretting over details that may or may not be relevant. There are so many factors you cannot account for otherwise. >> >> Regards, >> Jeremey. >> >> >> On Aug 17, 2012, at 5:53 AM, Samuel wrote: >> >>>> The second implementation is more secure in that sense, but still the >>>> original seed is guessable. An attacker can generate possible password >>>> sequences by bruteforce just tying possible now tuples around the time >>>> he thinks the real seed was created. >>>> >>>> >>>> So, how one can generate a secure un-predicatable seeds? >>> That's the tricky part :) At least you have to avoid generating >>> clearly predictable seeds as the seed is your private key in this >>> case. With the seed anyone can reproduce the sequence. >>> >>> crypto:strong_rand_bytes strives for better security properties, and I >>> understand it abstracts how to generate a good key for you, trying to >>> suck entropy from your system (so you may need to sit there banging >>> the keys and moving the mouse around for that ;) ) >>> >>> I am not a security expert by far, I just know some things that do not >>> work :). For things that work, the common approach is relying in >>> popular libraries not known to be broken. and trying not to use them >>> in a fancy way as the history is full of famous broken cryptographic >>> uses (you can read about flaws CSS, WEP, etc). >>> >>> Of course, whether that approach is advisable or not is more a >>> philosophical question, not knowing they are not broken doesn't mean >>> that no one knows how to break them and has the key access information >>> is thought to be safely encrypted :) >>> >>>> We also moved to "Tiny Mersenne Twister" >>>> (https://github.com/jj1bdx/tinymt-erlang) instead of using >>>> the standard random:uniform since the last Yaws security alert >>>> (http://erlang.org/pipermail/erlang-questions/2012-June/067626.html). >>>> Is this sufficient or should we also find a way to generate a unpredicatble >>>> seed for it? >>> As said, I am not a security expert, but as far as I can read, the >>> goals of that algorithm are to keep a small state with good >>> statistical properties, it says nothing about security (which doesn't >>> necessarily mean it is insecure, of course). A PRG can have good >>> statistical properties and still be insecure, being a secure PRG is a >>> stronger assumption. That is why erlang:random is fine for non >>> cryptographic uses, but for security you need something more complex. >>> >>> Anyway, you always need a seed no one can guess. Same seed, same >>> sequence, so if someone guesses your seed it basically gets all your >>> passwords in return. >>> >>> Regards >>> -- >>> Samuel >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- http://www.linkedin.com/in/torbenhoffmann From zabrane3@REDACTED Fri Aug 17 22:44:56 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 17 Aug 2012 22:44:56 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <502E9C74.70709@gmail.com> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> <528E5E49-4B89-4BF8-AF5B-35FCCE782392@alertlogic.com> <0438C996-9997-43EE-AEBA-23E10820BC02@gmail.com> <502E9C74.70709@gmail.com> Message-ID: <7C499491-125D-4019-BDA4-766434795356@gmail.com> Hi Torben, On Aug 17, 2012, at 9:33 PM, Torben Hoffmann wrote: > One thing to keep in mind with crypto is that it depends on OpenSSL which means that if you want to create a binary version of your application you need to make sure that the execution platform has OpenSSL. > > If you are developing a server for your own use - no problem. That's the case for us. > > If you want to break the dependency you can - on Un*x-like platforms - resort to getting a seed from /dev/urandom - there is an example of this in https://github.com/lehoff/cryptographic, it is ages since I worked on that and I cannot remember the state of that, but the general principle should be sound. > > There is a way to do this in Windows too, but we never got around to implementing that back then. From the source code: /* entropy.c - C code for generating random numbers using the internal windows entropy library. */ https://github.com/lehoff/cryptographic/blob/master/c_src/entropy.c Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Fri Aug 17 23:59:53 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 17 Aug 2012 23:59:53 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: <7C499491-125D-4019-BDA4-766434795356@gmail.com> References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> <528E5E49-4B89-4BF8-AF5B-35FCCE782392@alertlogic.com> <0438C996-9997-43EE-AEBA-23E10820BC02@gmail.com> <502E9C74.70709@gmail.com> <7C499491-125D-4019-BDA4-766434795356@gmail.com> Message-ID: Ups, forgot about the addition of the Windows code. As far as my bad memory serves me I think the Windows code does not work. And it is /dev/random, not /dev/urandom. Must remember to read up on old code before talking about it :-$ ___ /orben Sent from my iPhone On 17/08/2012, at 22.44, Zabrane Mickael wrote: > Hi Torben, > > On Aug 17, 2012, at 9:33 PM, Torben Hoffmann wrote: > >> One thing to keep in mind with crypto is that it depends on OpenSSL which means that if you want to create a binary version of your application you need to make sure that the execution platform has OpenSSL. >> >> If you are developing a server for your own use - no problem. > > That's the case for us. > >> >> If you want to break the dependency you can - on Un*x-like platforms - resort to getting a seed from /dev/urandom - there is an example of this in https://github.com/lehoff/cryptographic, it is ages since I worked on that and I cannot remember the state of that, but the general principle should be sound. >> >> There is a way to do this in Windows too, but we never got around to implementing that back then. > > From the source code: > /* entropy.c - C code for generating random numbers using the internal windows entropy library. */ > https://github.com/lehoff/cryptographic/blob/master/c_src/entropy.c > > Regards, > Zabrane > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Sat Aug 18 00:26:15 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Sat, 18 Aug 2012 00:26:15 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> <528E5E49-4B89-4BF8-AF5B-35FCCE782392@alertlogic.com> <0438C996-9997-43EE-AEBA-23E10820BC02@gmail.com> <502E9C74.70709@gmail.com> <7C499491-125D-4019-BDA4-766434795356@gmail.com> Message-ID: On Aug 17, 2012, at 11:59 PM, Torben Hoffmann wrote: > Ups, forgot about the addition of the Windows code. As far as my bad memory serves me you're not alone ;-) > I think the Windows code does not work. > And it is /dev/random, not /dev/urandom. > Must remember to read up on old code before talking about it :-$ Thanks, Zabrane From dmkolesnikov@REDACTED Sat Aug 18 15:52:02 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sat, 18 Aug 2012 16:52:02 +0300 Subject: [erlang-questions] UDP client/server performance In-Reply-To: <20120815140307.GD28968@ioctl> References: <20120814094019.GA21542@ioctl> <20120815005355.GB28968@ioctl> <20120815140307.GD28968@ioctl> Message-ID: Hello, I would slightly disagree with your evaluation approach on UDP performance. It is obvious that Erlang code is slower to compare with C code. Therefore, any single threaded benchmark would always show a highest performance of C code. You have to push it in multi client env... I've re run you test cases and got a bit different results with vanilla gen_udp. So, the lowest latency of inter packet arrival time on Erlang is about 16-17 usec. This is the time used by underlying SW stack to process packet and deliver it to application process. I did not managed to get any lower latency on my hw (Mac Lion Server). Async case: pure C solution handled about 341K packet/sec, erlang server runs 256K packet/sec. One of the biggest problem for async is high packet drop rate in both C & Erlang cases. Sync case: pure C solution handled about 160K packet/sec, erlang server runs 45-50K packet/sec. Regards, Dmitry On Aug 15, 2012, at 5:03 PM, Michael Santos wrote: > On Wed, Aug 15, 2012 at 09:59:22AM +0200, Ronny Meeus wrote: >> On Wed, Aug 15, 2012 at 2:53 AM, Michael Santos >> wrote: >>> On Tue, Aug 14, 2012 at 10:10:57PM +0200, Ronny Meeus wrote: >>>> Hello >>>> >>>> thanks for the info. >>>> I'm currently playing with the procket and it is working well. >>>> >>>> I have 2 questions: >>>> >>>> - I need to run the erl as root (or use sudo to start it) or I get: >>>> >>>> 2> {ok, FD} = procket:open(53, [{protocol, udp},{type, dgram},{family, inet}]). >>>> ** exception error: no match of right hand side value {error,eperm} >>> >>> Make priv/procket setuid or allow your user to call priv/procket using >>> sudo. >> >> I tried all options mentioned in your readme already before but non of >> them seem to work on Ubuntu. I think there is an issue with the sudo >> stuff on this distribution since no matter what I do, it always ask >> for a password when the procket command is executed. >> >> Now I succeeded by explicitly specifying the name of the program to use: >> procket:open(53, [{progname,"/usr/local/bin/procket"},{protocol, >> udp},{type,dgram}]). > > procket uses priv/procket as the default path. Sorry, README should be > clearer. > >>>> - How do I use recvfrom/4 in the server? If I call recvfrom/4 when >>>> there is no message present in the socket, it returns an error since >>>> the socket is put in async mode. >>> >>> Yes, the socket has to be non-blocking or it will block the scheduler. >>> >>>> Does this mean that the socket needs >>>> to be polled by the Erlang process or is it possible to get a >>>> notification (msg) when there is data present in the socket so that >>>> the recvfrom/4 can be called at that moment? >>> >>> You could plug the fd back into gen_udp:open(Port, [{fd, FD}]) ;) >>> >>> For a UDP socket, I can't think of any options other than the ones >>> you've mentioned: either spin on EAGAIN or use another NIF to notify you >>> when the socket is ready. There are a few ports to libev and libuv around. >> >> OK I will look around. >> Thanks. >> >> Yesterday I did a quick test and created an application that once it >> receives a message, it starts to send this back in a loop. >> It looks like it is able to send almost 200Kpkts/sec on a low end PC >> while the gen_udp solution only was able to send something like >> 10Kpks/sec on a much more powerful PC. So it looks promising ... > > That's cool! But remember, the C and the Erlang examples you posted are > doing different things. The C code is single tasking. It doesn't do any > error handling either. The Erlang code is multitasking, switching > between processes, monitoring processes for failure, .... > > If you run C NIF code in a tight loop, you'll get something closer to > the single tasking C code at the expense of concurrency. Maybe that is > fine for your needs though. Anyway, good luck in hitting your numbers! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ronny.meeus@REDACTED Sat Aug 18 21:45:03 2012 From: ronny.meeus@REDACTED (Ronny Meeus) Date: Sat, 18 Aug 2012 21:45:03 +0200 Subject: [erlang-questions] UDP client/server performance In-Reply-To: References: <20120814094019.GA21542@ioctl> <20120815005355.GB28968@ioctl> <20120815140307.GD28968@ioctl> Message-ID: Hello Thanks for taking the time to run the examples. I have run the tests also on a different machine and there is see also better results so it looks like there is an issue with the beam running on the first machine. Maybe it is compiled with different options ... Anyhow, I will continue the development of my application based on Erlang since I like the mechanisms it offers (threading, messaging, error handling etc) a lot. I initially wrote the mail because of the huge difference in performance I saw. Ronny On Sat, Aug 18, 2012 at 3:52 PM, Dmitry Kolesnikov wrote: > Hello, > > I would slightly disagree with your evaluation approach on UDP performance. It is obvious that Erlang code is slower to compare with C code. Therefore, any single threaded benchmark would always show a highest performance of C code. You have to push it in multi client env... I've re run you test cases and got a bit different results with vanilla gen_udp. > > So, the lowest latency of inter packet arrival time on Erlang is about 16-17 usec. This is the time used by underlying SW stack to process packet and deliver it to application process. I did not managed to get any lower latency on my hw (Mac Lion Server). > > Async case: pure C solution handled about 341K packet/sec, erlang server runs 256K packet/sec. One of the biggest problem for async is high packet drop rate in both C & Erlang cases. > Sync case: pure C solution handled about 160K packet/sec, erlang server runs 45-50K packet/sec. > > Regards, Dmitry > > On Aug 15, 2012, at 5:03 PM, Michael Santos wrote: > >> On Wed, Aug 15, 2012 at 09:59:22AM +0200, Ronny Meeus wrote: >>> On Wed, Aug 15, 2012 at 2:53 AM, Michael Santos >>> wrote: >>>> On Tue, Aug 14, 2012 at 10:10:57PM +0200, Ronny Meeus wrote: >>>>> Hello >>>>> >>>>> thanks for the info. >>>>> I'm currently playing with the procket and it is working well. >>>>> >>>>> I have 2 questions: >>>>> >>>>> - I need to run the erl as root (or use sudo to start it) or I get: >>>>> >>>>> 2> {ok, FD} = procket:open(53, [{protocol, udp},{type, dgram},{family, inet}]). >>>>> ** exception error: no match of right hand side value {error,eperm} >>>> >>>> Make priv/procket setuid or allow your user to call priv/procket using >>>> sudo. >>> >>> I tried all options mentioned in your readme already before but non of >>> them seem to work on Ubuntu. I think there is an issue with the sudo >>> stuff on this distribution since no matter what I do, it always ask >>> for a password when the procket command is executed. >>> >>> Now I succeeded by explicitly specifying the name of the program to use: >>> procket:open(53, [{progname,"/usr/local/bin/procket"},{protocol, >>> udp},{type,dgram}]). >> >> procket uses priv/procket as the default path. Sorry, README should be >> clearer. >> >>>>> - How do I use recvfrom/4 in the server? If I call recvfrom/4 when >>>>> there is no message present in the socket, it returns an error since >>>>> the socket is put in async mode. >>>> >>>> Yes, the socket has to be non-blocking or it will block the scheduler. >>>> >>>>> Does this mean that the socket needs >>>>> to be polled by the Erlang process or is it possible to get a >>>>> notification (msg) when there is data present in the socket so that >>>>> the recvfrom/4 can be called at that moment? >>>> >>>> You could plug the fd back into gen_udp:open(Port, [{fd, FD}]) ;) >>>> >>>> For a UDP socket, I can't think of any options other than the ones >>>> you've mentioned: either spin on EAGAIN or use another NIF to notify you >>>> when the socket is ready. There are a few ports to libev and libuv around. >>> >>> OK I will look around. >>> Thanks. >>> >>> Yesterday I did a quick test and created an application that once it >>> receives a message, it starts to send this back in a loop. >>> It looks like it is able to send almost 200Kpkts/sec on a low end PC >>> while the gen_udp solution only was able to send something like >>> 10Kpks/sec on a much more powerful PC. So it looks promising ... >> >> That's cool! But remember, the C and the Erlang examples you posted are >> doing different things. The C code is single tasking. It doesn't do any >> error handling either. The Erlang code is multitasking, switching >> between processes, monitoring processes for failure, .... >> >> If you run C NIF code in a tight loop, you'll get something closer to >> the single tasking C code at the expense of concurrency. Maybe that is >> fine for your needs though. Anyway, good luck in hitting your numbers! >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From kunthar@REDACTED Sat Aug 18 22:57:35 2012 From: kunthar@REDACTED (Kunthar) Date: Sat, 18 Aug 2012 23:57:35 +0300 Subject: [erlang-questions] rebar weird error Message-ID: Hola rebar tam ERROR: Failed to generate target from spec: "copy file /usr/local/lib/erlang_R14B03/lib/erlang/erts-5.8.4/bin/beam.smp -> /home/zumbar/release/kakaranet_rel/rels/app/node/erts-5.8.4/bin/beam.smp: text file or pseudo-device busy\n" inet??? wassup with rebar? ./rebar -V rebar 2.0.0 R14B04 20120813_155847 git 2.0.0-164-ge34b702 -- BR, \|/ Kunthar From tyron.zerafa@REDACTED Sun Aug 19 01:52:54 2012 From: tyron.zerafa@REDACTED (Tyron Zerafa) Date: Sun, 19 Aug 2012 01:52:54 +0200 Subject: [erlang-questions] Function Dependency Message-ID: Hey, Is there a way in Erlang which can give me a function's dependencies? Such as dependency_of_fun(A) will result in all functions' names (including modules) being used in A? Also, apart from this, can I somehow extract a function clause by knowing its signature only? Thanks in advance for all your help -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Sun Aug 19 16:00:36 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 19 Aug 2012 16:00:36 +0200 Subject: [erlang-questions] rebar weird error In-Reply-To: References: Message-ID: On Sat, Aug 18, 2012 at 10:57 PM, Kunthar wrote: > Hola rebar tam > > ERROR: Failed to generate target from spec: "copy file > /usr/local/lib/erlang_R14B03/lib/erlang/erts-5.8.4/bin/beam.smp -> > /home/zumbar/release/kakaranet_rel/rels/app/node/erts-5.8.4/bin/beam.smp: > text file or pseudo-device busy\n" > > inet??? > wassup with rebar? > > ./rebar -V > rebar 2.0.0 R14B04 20120813_155847 git 2.0.0-164-ge34b702 /bin/cp failed to overwrite .../kakaranet_rel/.../bin/beam.smp. Make sure to stop the node. From ilya.khlopotov@REDACTED Sun Aug 19 20:33:02 2012 From: ilya.khlopotov@REDACTED (Ilya Khlopotov) Date: Sun, 19 Aug 2012 11:33:02 -0700 Subject: [erlang-questions] rebar weird error (Kunthar) In-Reply-To: References: Message-ID: <1345401182.2226.4.camel@master> Hello Kunthar, > text file or pseudo-device busy This is posix error which could happen if you try to modify a file which is binary of currently running program. Check that you don't have erlang VM currently running from /home/zumbar/release/kakaranet_rel/rels/app/node/ directory. If you do then stop that VM and try to run rebar again. Best regards, ILYA > ------------------------------ > > Message: 3 > Date: Sat, 18 Aug 2012 23:57:35 +0300 > From: Kunthar > To: Erlang > Subject: [erlang-questions] rebar weird error > Message-ID: > > Content-Type: text/plain; charset=ISO-8859-1 > > Hola rebar tam > > ERROR: Failed to generate target from spec: "copy file > /usr/local/lib/erlang_R14B03/lib/erlang/erts-5.8.4/bin/beam.smp -> > /home/zumbar/release/kakaranet_rel/rels/app/node/erts-5.8.4/bin/beam.smp: > text file or pseudo-device busy\n" > > inet??? > wassup with rebar? > > ./rebar -V > rebar 2.0.0 R14B04 20120813_155847 git 2.0.0-164-ge34b702 > > From ok@REDACTED Sun Aug 19 23:49:50 2012 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 20 Aug 2012 09:49:50 +1200 Subject: [erlang-questions] Function Dependency In-Reply-To: References: Message-ID: <3783E7DF-FEBB-4E2C-9570-4F6F6DC13BA9@cs.otago.ac.nz> On 19/08/2012, at 11:52 AM, Tyron Zerafa wrote: > Is there a way in Erlang which can give me a function's dependencies? It's an open source system. You have full source code for the compiler and everything. You can readily get the parsed form of any source file. You can even (gasp, the technology!) do a web search for "Erlang cross reference" and discover http://www.erlang.org/doc/apps/tools/xref_chapter.html which says "Xref is a cross reference tool that can be used for finding dependencies between functions, modules, applications and releases. It does so by analyzing the defined functions and the function calls." > Such as dependency_of_fun(A) will result in all functions' names (including modules) > being used in A? See the Xref documentation. > Also, apart from this, can I somehow extract a function clause by knowing its signature only? Signature? The Erlang equivalent of a signature is a -spec. Function clauses don't have names, signatures, specifications, or anything else. It really makes no sense to extract just one clause from a function. I mean, here is a two-clause function: app([], L) -> L; app([H|T], L) -> [H|app(T, L)]. What good does it do you just to have ``app([], L) -> L''? What are you going to do with half of a function (or less)? From ulf.leopold@REDACTED Mon Aug 20 15:20:53 2012 From: ulf.leopold@REDACTED (Ulf Leopold) Date: Mon, 20 Aug 2012 15:20:53 +0200 Subject: [erlang-questions] "Empty" appups on the form {"", [], []} in OTP Message-ID: Hi All, I see that many apps in OTP (R14 & R15) have appups on the form: {"",[],[]} systools:make_relup does not seem to like them. How should they be interpreted? Does it mean that this particular app has no sanctioned upgrade/downgrade instruction? Thanks, Ulf -------------- next part -------------- An HTML attachment was scrubbed... URL: From calleja.justin@REDACTED Mon Aug 20 23:22:31 2012 From: calleja.justin@REDACTED (Justin Calleja) Date: Mon, 20 Aug 2012 17:22:31 -0400 Subject: [erlang-questions] how to call mysql in erlang(using erlang-mysql-driver) In-Reply-To: <201208151236121852035@ptmind.com> References: <201208151236121852035@ptmind.com> Message-ID: Hi, I haven't really taken a look at it, but this project might be relevant: http://sourceforge.net/projects/erlmysql/ cheers, Justin On 15 August 2012 00:36, kyletu wrote: > ** > Hello,I want call mysql PROCEDURE in erlang. but I get a error > message... > Is there somebody know how to do it? > > > PROCEDURE code: > > > *CREATE PROCEDURE simpleproc_04()******* > > *BEGIN* > > select * from taokeeper_stat limit 10; > > *END* > > erlang call code: > > mysql:fetch(poolId,<<"call simpleproc_04">>). > > > > then I get the error message: > > mysql_conn:433: fetch <<"call simpleproc_04">> (id <0.44.0>) > {error,{mysql_result,[],[],0,0, > "PROCEDURE taokeeper.simpleproc_04 can't return a result set in the given context", > 1312,"0A000"}} > > > > > > Thanks to read it... > > > > ** > > ****** > > > ------------------------------ > kyletu > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From calleja.justin@REDACTED Mon Aug 20 23:48:01 2012 From: calleja.justin@REDACTED (Justin Calleja) Date: Mon, 20 Aug 2012 17:48:01 -0400 Subject: [erlang-questions] Function Dependency In-Reply-To: <3783E7DF-FEBB-4E2C-9570-4F6F6DC13BA9@cs.otago.ac.nz> References: <3783E7DF-FEBB-4E2C-9570-4F6F6DC13BA9@cs.otago.ac.nz> Message-ID: Hi Tyron, I have never used xref myself but if it doesn't fit your needs you could roll up your own custom transformation on the parsed version of the code you're trying to compile. Of course, I am assuming you have access to the source code here. Not sure what you meant by "extract a function clause by knowing its signature only" but it seems to suggest that you're looking for something along these lines. Basically, when compiling your source code you can pass in an option to the compiler, {parse_transform, ModuleName} where ModuleName is expected to contain parse_transform/2 which will be passed an encoding of the source code you're compiling before it is actually compiled (i.e. an abstract syntax tree encoded in Erlang terms). You can then do your analyzing (checking what functions are depended on) and extracting inside ModuleName:parse_transform/2, making sure to return something to compile. Hi Richard, Here's one instance that comes to mind with regards to wanting part of a function definition (i.e. just some of it's clauses): When implementing a gen_server you handle synchronous requests using handle_call/3. You might just want the implementation of handle_call/3 which deals with certain messages. That is just something off the top of my head. There could be other reasons. cheers, Justin On 19 August 2012 17:49, Richard O'Keefe wrote: > > On 19/08/2012, at 11:52 AM, Tyron Zerafa wrote: > > Is there a way in Erlang which can give me a function's dependencies? > > It's an open source system. > You have full source code for the compiler and everything. > You can readily get the parsed form of any source file. > > You can even (gasp, the technology!) do a web search for > "Erlang cross reference" and discover > http://www.erlang.org/doc/apps/tools/xref_chapter.html > which says > "Xref is a cross reference tool that can be used for > finding dependencies between functions, modules, applications > and releases. It does so by analyzing the defined > functions and the function calls." > > > Such as dependency_of_fun(A) will result in all functions' names > (including modules) > > being used in A? > > See the Xref documentation. > > > Also, apart from this, can I somehow extract a function clause by > knowing its signature only? > > Signature? The Erlang equivalent of a signature is a -spec. > Function clauses don't have names, signatures, specifications, or anything > else. > It really makes no sense to extract just one clause from a function. > I mean, here is a two-clause function: > > app([], L) -> L; > app([H|T], L) -> [H|app(T, L)]. > > What good does it do you just to have ``app([], L) -> L''? > What are you going to do with half of a function (or less)? > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Aug 21 00:58:06 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 21 Aug 2012 10:58:06 +1200 Subject: [erlang-questions] Function Dependency In-Reply-To: References: <3783E7DF-FEBB-4E2C-9570-4F6F6DC13BA9@cs.otago.ac.nz> Message-ID: On 21/08/2012, at 9:48 AM, Justin Calleja wrote: > > Here's one instance that comes to mind with regards to wanting part of a function definition (i.e. just some of it's clauses): > When implementing a gen_server you handle synchronous requests using handle_call/3. You might just want the implementation of handle_call/3 which deals with certain messages. Yes, but that part - might be more than one clause - which need not be contiguous in the function - and some or all of them might be shared with the handling of other messages. It is possible to deliberately construct a function as a collection of clauses with small non-overlapping heads that simply redirect to other functions, so that it would make sense to extract part of _that_ function, and you would do so *not* by extracting the clause but by simply picking up the entire function that it delegates to. That doesn't mean it makes sense to extract part of a function that wasn't written to be treated that way. From maruthavanan_s@REDACTED Tue Aug 21 07:32:19 2012 From: maruthavanan_s@REDACTED (Maruthavanan Subbarayan) Date: Tue, 21 Aug 2012 01:32:19 -0400 Subject: [erlang-questions] Additional information while creating a crash dump Message-ID: Hi All, Is there anyway I can ask erlang VM to execute something while creating a erl_crash.dump file? Thanks,Marutha -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangdlf@REDACTED Tue Aug 21 07:46:51 2012 From: erlangdlf@REDACTED (deng lifen) Date: Tue, 21 Aug 2012 13:46:51 +0800 Subject: [erlang-questions] Additional information while creating a crash dump In-Reply-To: References: Message-ID: <2C2B96CD-E4F7-4FFA-9C07-01C644B5A149@gmail.com> Ctrl+C & A or execute: kill -s SIGUSR1 erlpid On 2012-8-21, at ??1:32, Maruthavanan Subbarayan wrote: > Hi All, > > Is there anyway I can ask erlang VM to execute something while creating a erl_crash.dump file? > > Thanks, > Marutha > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions deng lifen erlangdlf@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyron.zerafa@REDACTED Tue Aug 21 08:23:17 2012 From: tyron.zerafa@REDACTED (Tyron Zerafa) Date: Tue, 21 Aug 2012 08:23:17 +0200 Subject: [erlang-questions] Function Dependency In-Reply-To: References: <3783E7DF-FEBB-4E2C-9570-4F6F6DC13BA9@cs.otago.ac.nz> Message-ID: Hey, Thanks for your answers, I tried using xref and it is working perfectly, I am able to get both the local and remote functions which I am calling from a specific function. I need to extract the function body for the following reasons. I need to transfer only the code that a function makes use of between nodes, nothing more. Xref is giving me the signature of all those functions that I need. Now, I want to use these signatures to extract the remaining body of the functions. Another short question, is there any way in which I can differentiate whether a function comes from an Erlang release or not? Thus, if for example I ask is_native(io_lib,format,2) - > true, otherwise is_native(MyModule, MyFunction,0) -> false. I do not need to transfer functions which are native to Erlang and which can be found on any Erlang node. Thanks again for your help On Tue, Aug 21, 2012 at 12:58 AM, Richard O'Keefe wrote: > > On 21/08/2012, at 9:48 AM, Justin Calleja wrote: > > > > Here's one instance that comes to mind with regards to wanting part of a > function definition (i.e. just some of it's clauses): > > When implementing a gen_server you handle synchronous requests using > handle_call/3. You might just want the implementation of handle_call/3 > which deals with certain messages. > > Yes, but that part > - might be more than one clause > - which need not be contiguous in the function > - and some or all of them might be shared with the handling > of other messages. > > It is possible to deliberately construct a function as a > collection of clauses with small non-overlapping heads that > simply redirect to other functions, so that it would make > sense to extract part of _that_ function, and you would > do so *not* by extracting the clause but by simply picking > up the entire function that it delegates to. > > That doesn't mean it makes sense to extract part of a > function that wasn't written to be treated that way. > > > -- Best Regards, Tyron Zerafa -------------- next part -------------- An HTML attachment was scrubbed... URL: From calleja.justin@REDACTED Tue Aug 21 12:46:39 2012 From: calleja.justin@REDACTED (Justin Calleja) Date: Tue, 21 Aug 2012 12:46:39 +0200 Subject: [erlang-questions] Function Dependency In-Reply-To: References: <3783E7DF-FEBB-4E2C-9570-4F6F6DC13BA9@cs.otago.ac.nz> Message-ID: Hi Tyron, I don't know of any such function. You might get by with: > listing all modules in your path with code:get_path(). > Filtering out the ones which you don't want (i.e. the ones which come with the Erlang distribution). Probably you'd use the $ERL_TOP environment variable if it's set or something. > Listing the beams in these directories and taking it from there basically.. i.e. you could then check using ModuleName:module_info(exports) or something... it's a DIY solution off the top of my head but it should work. Hi Richard, Of course, encapsulating the logic to handle gen_server messages in different functions and then delegating to them is a good idea... I just wanted to point out that there **might** be a reason to get access to one or more of a function's clauses (and that parse_transform/2 is probably the way to go about it). cheers, Justin On 21 August 2012 08:23, Tyron Zerafa wrote: > Hey, > Thanks for your answers, I tried using xref and it is working > perfectly, I am able to get both the local and remote functions which I am > calling from a specific function. > > I need to extract the function body for the following reasons. I need to > transfer only the code that a function makes use of between nodes, nothing > more. Xref is giving me the signature of all those functions that I need. > Now, I want to use these signatures to extract the remaining body of the > functions. > > Another short question, is there any way in which I can differentiate > whether a function comes from an Erlang release or not? Thus, if for > example I ask is_native(io_lib,format,2) - > true, > otherwise is_native(MyModule, MyFunction,0) -> false. I do not need to > transfer functions which are native to Erlang and which can be found on any > Erlang node. > > Thanks again for your help > > On Tue, Aug 21, 2012 at 12:58 AM, Richard O'Keefe wrote: > >> >> On 21/08/2012, at 9:48 AM, Justin Calleja wrote: >> > >> > Here's one instance that comes to mind with regards to wanting part of >> a function definition (i.e. just some of it's clauses): >> > When implementing a gen_server you handle synchronous requests using >> handle_call/3. You might just want the implementation of handle_call/3 >> which deals with certain messages. >> >> Yes, but that part >> - might be more than one clause >> - which need not be contiguous in the function >> - and some or all of them might be shared with the handling >> of other messages. >> >> It is possible to deliberately construct a function as a >> collection of clauses with small non-overlapping heads that >> simply redirect to other functions, so that it would make >> sense to extract part of _that_ function, and you would >> do so *not* by extracting the clause but by simply picking >> up the entire function that it delegates to. >> >> That doesn't mean it makes sense to extract part of a >> function that wasn't written to be treated that way. >> >> >> > > > -- > Best Regards, > Tyron Zerafa > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenji@REDACTED Tue Aug 21 13:25:33 2012 From: kenji@REDACTED (Kenji Rikitake) Date: Tue, 21 Aug 2012 20:25:33 +0900 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> Message-ID: Samuel is right. SFMT is NOT cryptographically safe. Kenji Rikitake On Fri, Aug 17, 2012 at 7:53 PM, Samuel wrote: >> We also moved to "Tiny Mersenne Twister" >> (https://github.com/jj1bdx/tinymt-erlang) instead of using >> the standard random:uniform since the last Yaws security alert >> (http://erlang.org/pipermail/erlang-questions/2012-June/067626.html). >> Is this sufficient or should we also find a way to generate a unpredicatble >> seed for it? > > As said, I am not a security expert, but as far as I can read, the > goals of that algorithm are to keep a small state with good > statistical properties, it says nothing about security (which doesn't > necessarily mean it is insecure, of course). A PRG can have good > statistical properties and still be insecure, being a secure PRG is a > stronger assumption. That is why erlang:random is fine for non > cryptographic uses, but for security you need something more complex. > > Anyway, you always need a seed no one can guess. Same seed, same > sequence, so if someone guesses your seed it basically gets all your > passwords in return. > > Regards > -- > Samuel From kenji.rikitake@REDACTED Tue Aug 21 13:28:37 2012 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Tue, 21 Aug 2012 20:28:37 +0900 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> <528E5E49-4B89-4BF8-AF5B-35FCCE782392@alertlogic.com> <0438C996-9997-43EE-AEBA-23E10820BC02@gmail.com> <502E9C74.70709@gmail.com> <7C499491-125D-4019-BDA4-766434795356@gmail.com> Message-ID: Just FYI to you all: An interesting paper how /dev/(u)random could fail in an environment where external entropy is very much restricted or biased. https://factorable.net/paper.html (Two PDF files available; I suggest you to read the extended version of the paper.) If you really want to make a secure password generator, always think about how to get enough entropy, preferably using a trustable hardware true RNG. (And I suggest you to check your SSH host keys and SSL/TLS certs as well, at: https://factorable.net/keycheck.html Kenji Rikitake (Excuse me that this is not necessarily an Erlang topic, but I think this is enough related to the thread) From zabrane3@REDACTED Tue Aug 21 13:46:58 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 21 Aug 2012 13:46:58 +0200 Subject: [erlang-questions] Password generator in Erlang In-Reply-To: References: <62A7B196-371E-450A-8D73-7C3F9CD70489@gmail.com> <502A721C.1020308@erix.ericsson.se> <6936E038-730D-43FE-857A-D20A4E4673BA@gmail.com> <528E5E49-4B89-4BF8-AF5B-35FCCE782392@alertlogic.com> <0438C996-9997-43EE-AEBA-23E10820BC02@gmail.com> <502E9C74.70709@gmail.com> <7C499491-125D-4019-BDA4-766434795356@gmail.com> Message-ID: Thanks for sharing these valuable information Kenji. Regards, Zabrane On Aug 21, 2012, at 1:28 PM, Kenji Rikitake wrote: > Just FYI to you all: > > An interesting paper how /dev/(u)random could fail > in an environment where external entropy is > very much restricted or biased. > > https://factorable.net/paper.html > (Two PDF files available; I suggest you to read > the extended version of the paper.) > > If you really want to make a secure password generator, > always think about how to get enough entropy, > preferably using a trustable hardware true RNG. > > (And I suggest you to check your SSH host keys > and SSL/TLS certs as well, at: > https://factorable.net/keycheck.html > > Kenji Rikitake > (Excuse me that this is not necessarily an Erlang topic, > but I think this is enough related to the thread) From g@REDACTED Tue Aug 21 17:54:07 2012 From: g@REDACTED (Garrett Smith) Date: Tue, 21 Aug 2012 10:54:07 -0500 Subject: [erlang-questions] [ANN] Chicago area meetup this Wed Message-ID: For folks in the Chicago area who would otherwise be watching CSI tomorrow night, this might be of interest: http://www.meetup.com/ErlangChicago/events/77541602/ In particular, Eric Merrit will be introducing Joxa -- a full featured Lisp on the Erlang VM. If you're able to attend, please either RSVP via meetup.com, or reply to me off-list with your name as it appears on your photo ID. Garrett From zabrane3@REDACTED Tue Aug 21 18:15:22 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 21 Aug 2012 18:15:22 +0200 Subject: [erlang-questions] [ANN] Chicago area meetup this Wed In-Reply-To: References: Message-ID: <5B076827-E14E-4407-B5E8-602EC742D90A@gmail.com> Hi Garret, Are you planning to record the meeting (i'm in Europe)? Is "Joxa" in github? Regards, Zabrane On Aug 21, 2012, at 5:54 PM, Garrett Smith wrote: > For folks in the Chicago area who would otherwise be watching CSI > tomorrow night, this might be of interest: > > http://www.meetup.com/ErlangChicago/events/77541602/ > > In particular, Eric Merrit will be introducing Joxa -- a full featured > Lisp on the Erlang VM. > > If you're able to attend, please either RSVP via meetup.com, or reply > to me off-list with your name as it appears on your photo ID. > > Garrett > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ericbmerritt@REDACTED Tue Aug 21 18:23:50 2012 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 21 Aug 2012 11:23:50 -0500 Subject: [erlang-questions] [ANN] Chicago area meetup this Wed In-Reply-To: <5B076827-E14E-4407-B5E8-602EC742D90A@gmail.com> References: <5B076827-E14E-4407-B5E8-602EC742D90A@gmail.com> Message-ID: Zabrane, We will record the meeting if we can find some one willing to do it. We cant promise anything there though it should be possible. Joxa is on github here -> https://github.com/erlware/joxa Let me know if you run into any issues with it of course. Eric On Tue, Aug 21, 2012 at 11:15 AM, Zabrane Mickael wrote: > Hi Garret, > > Are you planning to record the meeting (i'm in Europe)? > Is "Joxa" in github? > > Regards, > Zabrane > > On Aug 21, 2012, at 5:54 PM, Garrett Smith wrote: > >> For folks in the Chicago area who would otherwise be watching CSI >> tomorrow night, this might be of interest: >> >> http://www.meetup.com/ErlangChicago/events/77541602/ >> >> In particular, Eric Merrit will be introducing Joxa -- a full featured >> Lisp on the Erlang VM. >> >> If you're able to attend, please either RSVP via meetup.com, or reply >> to me off-list with your name as it appears on your photo ID. >> >> Garrett >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Tue Aug 21 22:25:01 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 21 Aug 2012 22:25:01 +0200 Subject: [erlang-questions] Generate good Erlang doc Message-ID: <72C149EB-5C93-4BB9-9FB7-55B5E4906403@gmail.com> Hi guys, Do you've any experience on generating Erlang doc using Sphinx (http://sphinx.pocoo.org/)? I found this Sphinx mode for Erlang: http://packages.python.org/sphinxcontrib-erlangdomain/ It seems a bit old (2010) but still sable Do you know something similar to Sphinx or better (not edoc please ;-))? Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Tue Aug 21 23:34:47 2012 From: g@REDACTED (Garrett Smith) Date: Tue, 21 Aug 2012 16:34:47 -0500 Subject: [erlang-questions] Generate good Erlang doc In-Reply-To: <72C149EB-5C93-4BB9-9FB7-55B5E4906403@gmail.com> References: <72C149EB-5C93-4BB9-9FB7-55B5E4906403@gmail.com> Message-ID: I've been using sphinx for general documentation for years and am still under the impression it's the best out there, even for languages other than Python. If there's a half way decent way to generate API docs from Erlang source, I'd suggest starting there and contributing improvements as needed. I'd be very happy to help with that! I'm currently just linking to edoc generated docs for Erlang, which is okay, but it'd be so much better to be integrated with sphinx! On Tue, Aug 21, 2012 at 3:25 PM, Zabrane Mickael wrote: > Hi guys, > > Do you've any experience on generating Erlang doc using Sphinx > (http://sphinx.pocoo.org/)? > > I found this Sphinx mode for Erlang: > http://packages.python.org/sphinxcontrib-erlangdomain/ > It seems a bit old (2010) but still sable > > Do you know something similar to Sphinx or better (not edoc please ;-))? > > Regards, > Zabrane > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From martin@REDACTED Tue Aug 21 23:56:59 2012 From: martin@REDACTED (Martin Janiczek) Date: Tue, 21 Aug 2012 23:56:59 +0200 Subject: [erlang-questions] Erlang.NET mbox doesn't receive message Message-ID: <5034042B.8020606@janiczek.cz> Hi guys, I'm using Erlang.NET ( https://github.com/takayuki/Erlang.NET ) and am struggling with receiving message that clearly got sent. There is Erlang server and C# client. The server listens for {Client, msg_from_client} and then sends Client ! msg_from_server. The debug logging from client shows that the message got sent from server, but somehow the row OtpErlangObject response = mbox.receive(); doesn't catch it and waits until it gets ^C'd. I don't really know what does the sync parameter in OptNode.createMbox(bool sync) do. When I tried to change it from true to false, instead of waiting indefinitely it throws this exception: Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object Why doesn't the code see the message even though it gets sent? And what should I use in the createMbox(bool sync)? Thanks! Relevant part of server code: ------------------------------------------------------------ loop() -> io:format("~nwaiting for msgs~n"), receive {Client, msg_from_client} -> io:format("got msg from client ~p~n", [Client]), Client ! msg_from_server, io:format("wrote to client~n") end, loop(). Client code: ------------------------------------------------------------ using System; using Erlang.NET; namespace Client { public class V01 { public static void Main(string[] args) { OtpNode me = new OtpNode("client", "abc"); //if (!me.ping("server@REDACTED", 2000)) //{ // Console.WriteLine("can't ping!"); // Environment.Exit(1); //} // do not ping before the connect! OtpSelf self = new OtpSelf("client", "abc"); OtpPeer other = new OtpPeer("server@REDACTED"); OtpConnection conn = self.connect(other); conn.sendRPC("global","sync",new OtpErlangList()); OtpMbox mbox = me.createMbox(true); OtpErlangObject[] arr = {mbox.Self, new OtpErlangAtom("msg_from_client")}; OtpErlangTuple msg = new OtpErlangTuple(arr); OtpErlangObject[] msg_args = {new OtpErlangAtom("server"), msg}; Console.WriteLine("About to send"); conn.sendRPC("global","send",msg_args); Console.WriteLine("Sent, gonna receive"); OtpErlangObject response = mbox.receive(); Console.WriteLine("Got it!"); Console.WriteLine(response); Console.WriteLine("Hash:"); Console.WriteLine(response.GetHashCode()); me.close(); Environment.Exit(0); } } } Server output: ------------------------------------------------------------ waiting for msgs got msg from client <6547.1.0> wrote to client waiting for msgs Client output (with verbose logging): ------------------------------------------------------------ (... handshakes and whatnot prior to the sending and receiving ...) About to send 23:37:55 - -> REG_SEND {6,#Pid,'',rex} 23:37:55 - {#Pid,{call,global,send,[server,{#Pid,msg_from_client}],user}} Sent, gonna receive 23:37:55 - <- SEND {2,'',#Pid} 23:37:55 - {rex,ok} 23:37:55 - <- SEND {2,'',#Pid} 23:37:55 - {rex,#Pid} 23:37:55 - <- SEND {2,'',#Pid} 23:37:55 - msg_from_server From ok@REDACTED Wed Aug 22 00:19:10 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 22 Aug 2012 10:19:10 +1200 Subject: [erlang-questions] Function Dependency In-Reply-To: References: <3783E7DF-FEBB-4E2C-9570-4F6F6DC13BA9@cs.otago.ac.nz> Message-ID: <25BB25D0-B346-47D1-88BF-2241AB49A2B9@cs.otago.ac.nz> On 21/08/2012, at 6:23 PM, Tyron Zerafa wrote: > > I need to extract the function body for the following reasons. I need to transfer only the code that a function makes use of between nodes, nothing more. Xref is giving me the signature of all those functions that I need. Now, I want to use these signatures to extract the remaining body of the functions. Xref *never* provides any kind of signatures. All it provides is function *names* (module:atom/arity triples). Those are *NOT* what computer science calls signatures. > > Another short question, is there any way in which I can differentiate whether a function comes from an Erlang release or not? Thus, if for example I ask is_native(io_lib,format,2) - > true, otherwise is_native(MyModule, MyFunction,0) -> false. I do not need to transfer functions which are native to Erlang and which can be found on any Erlang node. Suppose you have the name of a function in MFA {Module,Function,Arity} form. If any function in a module comes from an Erlang/OTP release, all of them do. So you need to find out where a module comes from. code:is_loaded(Module_Name) -> {file,"/where/it/came/from"} if the module is loaded; -> preloaded if the module is preloaded -> cover_compiled if the module was processed by cover(3) for coverage analysis All of the files from an Erlang release will be somewhere inside a common directory; on my Solaris 10 box it's /users/local/lib/erlang/lib. Just what are you going to do if a function you want to send over depends on a driver written in C? The receiving system may not have that driver, and even if you could extract it, that might do no good because one of the systems might be an ARM7 and the other might be an x86_64. Then too, suppose you want to extract the function whose name is {nurke,fred,1} and you discover that it depends on {seagoon,neddy,5}, but you think that's ok, because the receiving system tells you it *has* {seagoon,neddy,5}. How do you know it means the same thing? Yesterday I was reading a short article in New Scientist about someone transplanting bits of zebra finch embryo brains into quail embryo brains. I can see the scientific reasons for this but some things just fail the gut test, they really do. It's not at all surprising that the quanch embryos don't hatch. That kind of thing only works between closely related species. Get my drift? You cannot expect to "transplant" an arbitrary function from one Erlang system to another unless you have ensured that they are very very similar. And if you can exert that much control over both configurations, why not ensure that they are identical? I don't even want to think of the security implications of a system deliberately allowing another one to inject arbitrary code into itself. But you should! From zabrane3@REDACTED Wed Aug 22 01:38:26 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 22 Aug 2012 01:38:26 +0200 Subject: [erlang-questions] Generate good Erlang doc In-Reply-To: References: <72C149EB-5C93-4BB9-9FB7-55B5E4906403@gmail.com> Message-ID: Thanks Garrett .. I'm giving Sphinx a try. Regards, Zabrane On Aug 21, 2012, at 11:34 PM, Garrett Smith wrote: > I've been using sphinx for general documentation for years and am > still under the impression it's the best out there, even for languages > other than Python. > > If there's a half way decent way to generate API docs from Erlang > source, I'd suggest starting there and contributing improvements as > needed. I'd be very happy to help with that! > > I'm currently just linking to edoc generated docs for Erlang, which is > okay, but it'd be so much better to be integrated with sphinx! > > On Tue, Aug 21, 2012 at 3:25 PM, Zabrane Mickael wrote: >> Hi guys, >> >> Do you've any experience on generating Erlang doc using Sphinx >> (http://sphinx.pocoo.org/)? >> >> I found this Sphinx mode for Erlang: >> http://packages.python.org/sphinxcontrib-erlangdomain/ >> It seems a bit old (2010) but still sable >> >> Do you know something similar to Sphinx or better (not edoc please ;-))? >> >> Regards, >> Zabrane >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From desired.mta@REDACTED Wed Aug 22 08:48:13 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Wed, 22 Aug 2012 08:48:13 +0200 Subject: [erlang-questions] calling Erlang functions from linked-in driver Message-ID: Hi, We are looking for a way to call Erlang functions from linked-in port driver (or NIF). From documentation I cannot see any way to do it. We want to call some Erlang functions from Lua code[1], which is executed in linked-in driver (can be re-implemented in NIF). In C node case this would be simple. However, for this C node is a bit overkill (we want erlualib side effects to be as small as possible). Linked-in drivers and NIFs are conceptually perfect for it. Is there a way to call Erlang function from linked-in driver or NIF *before* returning the result to emulator? Thanks, Motiejus Jak?tys [1]: https://github.com/Motiejus/erlualib From freeakk@REDACTED Wed Aug 22 09:01:50 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Wed, 22 Aug 2012 11:01:50 +0400 Subject: [erlang-questions] calling Erlang functions from linked-in driver In-Reply-To: References: Message-ID: It is not possible. You cannot pass funs to a driver. From max.lapshin@REDACTED Wed Aug 22 09:14:37 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 22 Aug 2012 11:14:37 +0400 Subject: [erlang-questions] calling Erlang functions from linked-in driver In-Reply-To: References: Message-ID: No. You need to pass some message up to erlang level, call function from there and return back result into driver and return back result from driver to caller. From mjtruog@REDACTED Wed Aug 22 09:17:38 2012 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 22 Aug 2012 00:17:38 -0700 Subject: [erlang-questions] calling Erlang functions from linked-in driver In-Reply-To: References: Message-ID: <50348792.7030501@gmail.com> You could use http://www.erlang.org/doc/man/erl_driver.html#driver_send_term or http://www.erlang.org/doc/man/erl_nif.html#enif_send to call Erlang code by sending an Erlang process a message (which is then handled so that it calls the Erlang code you want to call). On 08/21/2012 11:48 PM, Motiejus Jak?tys wrote: > Hi, > > We are looking for a way to call Erlang functions from linked-in port > driver (or NIF). From documentation I cannot see any way to do it. We > want to call some Erlang functions from Lua code[1], which is executed > in linked-in driver (can be re-implemented in NIF). > > In C node case this would be simple. However, for this C node is a bit > overkill (we want erlualib side effects to be as small as possible). > Linked-in drivers and NIFs are conceptually perfect for it. > > Is there a way to call Erlang function from linked-in driver or NIF > *before* returning the result to emulator? > > Thanks, > Motiejus Jak?tys > > [1]: https://github.com/Motiejus/erlualib > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From matthias@REDACTED Wed Aug 22 10:03:28 2012 From: matthias@REDACTED (Matthias Lang) Date: Wed, 22 Aug 2012 10:03:28 +0200 Subject: [erlang-questions] Function Dependency In-Reply-To: <25BB25D0-B346-47D1-88BF-2241AB49A2B9@cs.otago.ac.nz> References: <3783E7DF-FEBB-4E2C-9570-4F6F6DC13BA9@cs.otago.ac.nz> <25BB25D0-B346-47D1-88BF-2241AB49A2B9@cs.otago.ac.nz> Message-ID: <20120822080328.GA3248@corelatus.se> On Wednesday, August 22, Richard O'Keefe wrote: > All it provides is function *names* (module:atom/arity triples). > Those are *NOT* what computer science calls signatures. I tried to get a clearer idea of what "function signatures" are in general (i.e. not just in C++) by looking at http://en.wikipedia.org/wiki/Type_signature "In computer science, a type signature or type annotation defines the inputs and outputs for a function, subroutine or method. A type signature includes at least the function name and the number of its arguments. In some programming languages, it may also specify the function's return type, the types of its arguments, or errors it may pass back." The "at least" part is a bit odd, and seems to be contradicted by text further down in the same article, which is more in line with what I was expecting: "The signature of a function is a way of describing the parameters and parameter types with which a legal call to the function can be made." Hard to know if whoever wrote that was trying to be clever and left out the function's name because it's implied by "the function" or if it's hastily written. Perhaps there is a language which has a concept of a "function signature" or a "type signature" which is limited to name + arity. I can't think of one. That wikipedia article looks like a good candidate for tightening up; in particular it doesn't have any good references (which is what I went there for in the first place). Maybe someone here can suggest some. Matt From desired.mta@REDACTED Wed Aug 22 10:06:59 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Wed, 22 Aug 2012 10:06:59 +0200 Subject: [erlang-questions] calling Erlang functions from linked-in driver In-Reply-To: <50348792.7030501@gmail.com> References: <50348792.7030501@gmail.com> Message-ID: On Wed, Aug 22, 2012 at 9:17 AM, Michael Truog wrote: > You could use > http://www.erlang.org/doc/man/erl_driver.html#driver_send_term > or > http://www.erlang.org/doc/man/erl_nif.html#enif_send Thanks. Somehow I missed these. This is getting really interesting. Comments from more experienced users than me highly appreciated :) > to call Erlang code by sending an Erlang process a message (which is then handled so that it calls the Erlang code you want to call). OK, this is a chance. So creating an Erlang processor process is trivial, executing functions - too. Now I need some way to pass the return value back to the caller. Is there a way to *receive* a message in NIF or port driver? I can't find one in the manual. If not, I could do this: When I need to call another Erlang function from C: * create an Erlang resource* protected by a mutex and condition variable (should be per-port) * lock a mutex * send {rpc, Resource, M, F, Args} to another Erlang process * hold on condition variable (perhaps in another thread?)** Then another Erlang process (implemented partially in C, partially in Erlang): * do the RPC call, fetch the results * lock the mutex * put the results to the Resource * signal condition variable For performance, The first process continues execution with Resource set to the returned value. Is there a simpler way to do this? [*] in NIF case, it would be Resource. In port driver case, it would be a void*... [**] how could driver_async help me here not to block the scheduler while this is happening? Is this a valid use case for driver_async? Thanks! Motiejus Jak?tys From kostis@REDACTED Wed Aug 22 00:37:01 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 22 Aug 2012 00:37:01 +0200 Subject: [erlang-questions] CFP: Practical Aspects of Declarative Languages (PADL 2013) Message-ID: <50340D8D.6070707@it.uu.se> Apologies for cross-posting Call for Papers =============== 15th International Symposium on Practical Aspects of Declarative Languages (PADL 2013) http://www.it.uu.se/conf/padl2013/ Rome, Italy, January 21-22, 2013 Co-located with ACM POPL 2013 Conference Description ====================== Declarative languages build on sound theoretical bases to provide attractive frameworks for application development. These languages have been successfully applied to many different real-world situations, ranging from data base management to active networks to software engineering to decision support systems. New developments in theory and implementation have opened up new application areas. At the same time, applications of declarative languages to novel problems raise numerous interesting research issues. Well-known questions include designing for scalability, language extensions for application deployment, and programming environments. Thus, applications drive the progress in the theory and implementation of declarative systems, and benefit from this progress as well. PADL is a forum for researchers and practitioners to present original work emphasizing novel applications and implementation techniques for all forms of declarative concepts, including, functional, logic, constraints, etc. Topics of interest include, but are not limited to: * Innovative applications of declarative languages * Declarative domain-specific languages and applications * Practical applications of theoretical results * New language developments and their impact on applications * Declarative languages and software engineering * Evaluation of implementation techniques on practical applications * Practical experiences and industrial applications * Novel uses of declarative languages in the classroom * Practical extensions such as constraint-based, probabilistic, and reactive languages. PADL 2013 welcomes new ideas and approaches pertaining to applications and implementation of declarative languages. In this occasion PADL is co-located, as traditionally, with ACM POPL, which will be held immediately following PADL. The symposium will be held in beautiful Rome, Italy. Important Dates and Submission Guidelines ========================================= Abstract Submission: September 10, 2012 Paper Submission: September 17, 2012 Notification: October 28, 2012 Camera-ready: November 12, 2012 Symposium: January 21-22, 2013 Authors should submit an electronic copy of the full paper in PDF using the Springer LNCS format. The submission will be done through EasyChair conference system. If electronic submission is impossible, please contact the program chair long in advance for information on how to submit hard copies. All submissions must be original work written in English. Submissions must be unpublished and not submitted for publication elsewhere. Work that already appeared in unpublished or informally published workshops proceedings may be submitted but the authors should notify the program chair about the place on which it has previously appeared. PADL 2013 will accept both technical and application papers: * Technical papers must describe original, previously unpublished research results. Technical papers must not exceed 16 pages in Springer LNCS format. * Application papers are a mechanism to present important practical applications of declarative languages that occur in industry or in areas of research other than Computer Science. Application papers will be published in the Springer-Verlag conference proceedings, and will be presented in a separate session. Application papers are expected to describe complex and/or real-world applications that rely on an innovative use of declarative languages. Application descriptions, engineering solutions and real-world experiences (both positive and negative) are solicited. The limit for application papers is 6 pages in Springer LNCS format but such papers can also point to sites with supplemental information about the application or the system that they describe. Program Committee ================= Elvira Albert, Complutense University of Madrid, Spain Maria Garcia de la Banda, Monash University, Australia Adam Chlipala, Massachusetts Institute of Technology, U.S.A. Bart Demoen, K.U. Leuven, Belgium Maurizio Gabbrielli, University of Bologna, Italy Jurriaan Hage, Utrecht University, The Netherlands Kevin Hammond, University of St. Andrews, U.K. Ralf Hinze, University of Oxford, U.K. Jacob Howe, City University London, U.K. Joxan Jaffar, National University of Singapore, Singapore Gabriele Keller, University of New South Wales, Australia Naoki Kobayashi, University of Tokyo, Japan Kim Nguyen, LRI, Universit? Paris-Sud, France Norman Ramsey, Tufts University, U.S.A. Kostis Sagonas, Uppsala University, Sweden; NTUA, Greece (chair) Vitor Santos Costa, University of Porto, Portugal Terrance Swift, Univ. Nova de Lisboa, Portugal; Johns Hopkins Univ., USA Dimitrios Vytiniotis, Microsoft Research, U.K Contacts ======== For additional information about papers and submissions, please contact the Program Chairs: Kostis Sagonas Uppsala University, Sweden Email: kostis it uu se From f@REDACTED Wed Aug 22 13:18:28 2012 From: f@REDACTED (Francesco Mazzoli) Date: Wed, 22 Aug 2012 12:18:28 +0100 Subject: [erlang-questions] `mnesia:change_table_copy_type/3' issues when changing from disc to RAM Message-ID: <87r4qzkwe3.wl%f@mazzo.li> Hi list, For some reason, `mnesia:change_table_copy_type/3' fails when changing a table replica type from ram to disc while some node containing a disc replica of the same table is offline, with the error {aborted, {not_active, "All replicas on diskfull nodes are not active yet", rabbit_user, [hare@REDACTED]}} where `rabbit_user' is the table in question, and `hare@REDACTED' is the offline node that contains a disc replica. Note that there are other nodes with disc replicas online. Why does this happen? I've looked at `mnesia_schema.erl' and it seems to do this check (in `ensure_active/2') when the type transition (in this case disc -> RAM) is marked as `incompatible' by `compare_storage_type/3'. One solutions would be to delete the local replica and then re-create it, but that's inefficient and I'm still curious on why does the problem arise in the first place. -- Francesco * Often in error, never in doubt From essen@REDACTED Wed Aug 22 13:30:38 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 22 Aug 2012 13:30:38 +0200 Subject: [erlang-questions] [ANN] Ranch 0.4.0 Message-ID: <5034C2DE.10007@ninenines.eu> Hello, The new Ranch is here. Ranch is an acceptor pool for TCP protocols. https://github.com/extend/ranch The main work in this version was to remove the need for a very busy central gen_server, which was set in priority high in previous versions. This improved things for some and caused issues for others. To fix this I took inspiration from my frequent use of gproc and ended up using a similar but simpler approach (because we don't need to handle that many use cases). Many tests have been added, a simple example has been added, and most importantly a complete user guide has been written. https://github.com/extend/ranch/blob/master/guide/toc.md Please forward any comment, criticism, swearing or candy to me in any way you prefer. This version is what's going to be used in Cowboy 0.8.0. Cowboy master will be updated in a few days with the new code. Be careful if you still depend on "master" instead of "0.6.1"! -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From andre@REDACTED Wed Aug 22 13:44:46 2012 From: andre@REDACTED (=?ISO-8859-1?Q?Andr=E9_Graf?=) Date: Wed, 22 Aug 2012 13:44:46 +0200 Subject: [erlang-questions] [ANN] Ranch 0.4.0 In-Reply-To: <5034C2DE.10007@ninenines.eu> References: <5034C2DE.10007@ninenines.eu> Message-ID: Hi Lo?c Wow that's awesome news, and thank you for the documentation! :) One question remains: In https://github.com/extend/ranch/blob/master/guide/listeners.md#listening-on-a-port--1024 you mention that it isn't possible to listen on well known ports, why this limitation? If you start the node under a privileged user, or use tools such as privbind or authbind it shouldn't be a problem at all, right? Cheers! Andr? On 22 August 2012 13:30, Lo?c Hoguin wrote: > Hello, > > The new Ranch is here. Ranch is an acceptor pool for TCP protocols. > > https://github.com/extend/ranch > > The main work in this version was to remove the need for a very busy central > gen_server, which was set in priority high in previous versions. This > improved things for some and caused issues for others. > > To fix this I took inspiration from my frequent use of gproc and ended up > using a similar but simpler approach (because we don't need to handle that > many use cases). > > Many tests have been added, a simple example has been added, and most > importantly a complete user guide has been written. > > https://github.com/extend/ranch/blob/master/guide/toc.md > > Please forward any comment, criticism, swearing or candy to me in any way > you prefer. > > This version is what's going to be used in Cowboy 0.8.0. Cowboy master will > be updated in a few days with the new code. Be careful if you still depend > on "master" instead of "0.6.1"! > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Wed Aug 22 13:56:01 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 22 Aug 2012 13:56:01 +0200 Subject: [erlang-questions] [ANN] Ranch 0.4.0 In-Reply-To: References: <5034C2DE.10007@ninenines.eu> Message-ID: <5034C8D1.8030601@ninenines.eu> On 08/22/2012 01:44 PM, Andr? Graf wrote: > Hi Lo?c > > Wow that's awesome news, and thank you for the documentation! :) > > One question remains: In > https://github.com/extend/ranch/blob/master/guide/listeners.md#listening-on-a-port--1024 > you mention that it isn't possible to listen on well known ports, why > this limitation? If you start the node under a privileged user, or use > tools such as privbind or authbind it shouldn't be a problem at all, > right? Yeah of course there is a number of solutions involving various third party softwares or system configuration, but there is nothing specific in Ranch to take care of that. So I'm not sure how to go with that topic, can't document every possible solution. > Cheers! > Andr? > > > On 22 August 2012 13:30, Lo?c Hoguin wrote: >> Hello, >> >> The new Ranch is here. Ranch is an acceptor pool for TCP protocols. >> >> https://github.com/extend/ranch >> >> The main work in this version was to remove the need for a very busy central >> gen_server, which was set in priority high in previous versions. This >> improved things for some and caused issues for others. >> >> To fix this I took inspiration from my frequent use of gproc and ended up >> using a similar but simpler approach (because we don't need to handle that >> many use cases). >> >> Many tests have been added, a simple example has been added, and most >> importantly a complete user guide has been written. >> >> https://github.com/extend/ranch/blob/master/guide/toc.md >> >> Please forward any comment, criticism, swearing or candy to me in any way >> you prefer. >> >> This version is what's going to be used in Cowboy 0.8.0. Cowboy master will >> be updated in a few days with the new code. Be careful if you still depend >> on "master" instead of "0.6.1"! >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From langxianzhe@REDACTED Wed Aug 22 14:06:43 2012 From: langxianzhe@REDACTED (=?GB2312?B?wMnPzM7k?=) Date: Wed, 22 Aug 2012 20:06:43 +0800 Subject: [erlang-questions] =?gb2312?b?ZXRzse3X7rTzyv3T2m1uZXNpYcrCzvE=?= =?gb2312?b?zsrM4g==?= Message-ID: ???? ??????????????? ????????? "Cannot create an ets table for the local transaction store", ?????? -env ERL_MAX_ETS_TABLES 24000 ?????????????? ??????? 1????????1400,?????????????????????? 2??????????24000 ???? 3???mnesia?????ets????ets????????????????????????? Thanks ??? The number of tables stored at one Erlang node is limited. The current default limit is approximately 1400 tables. The upper limit can be increased by setting the environment variable ERL_MAX_ETS_TABLES before starting the Erlang runtime system -- ??????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre@REDACTED Wed Aug 22 14:11:03 2012 From: andre@REDACTED (=?ISO-8859-1?Q?Andr=E9_Graf?=) Date: Wed, 22 Aug 2012 14:11:03 +0200 Subject: [erlang-questions] [ANN] Ranch 0.4.0 In-Reply-To: <5034C8D1.8030601@ninenines.eu> References: <5034C2DE.10007@ninenines.eu> <5034C8D1.8030601@ninenines.eu> Message-ID: On 22 August 2012 13:56, Lo?c Hoguin wrote: > On 08/22/2012 01:44 PM, Andr? Graf wrote: >> >> Hi Lo?c >> >> >> Wow that's awesome news, and thank you for the documentation! :) >> >> One question remains: In >> >> https://github.com/extend/ranch/blob/master/guide/listeners.md#listening-on-a-port--1024 >> you mention that it isn't possible to listen on well known ports, why >> this limitation? If you start the node under a privileged user, or use >> tools such as privbind or authbind it shouldn't be a problem at all, >> right? > > > Yeah of course there is a number of solutions involving various third party > softwares or system configuration, but there is nothing specific in Ranch to > take care of that. So I'm not sure how to go with that topic, can't document > every possible solution. Agree, maybe a documentation page like yaws http://yaws.hyber.org/privbind.yaws can be useful. But, since ranch is just a library it may be not even necessary to keep the whole well-known-port story at all. Anyhow, if you feel it is necessary to keep it I am happy to contribute a little bit more documentation on that one. > >> Cheers! >> Andr? >> >> >> >> On 22 August 2012 13:30, Lo?c Hoguin wrote: >>> >>> Hello, >>> >>> The new Ranch is here. Ranch is an acceptor pool for TCP protocols. >>> >>> https://github.com/extend/ranch >>> >>> The main work in this version was to remove the need for a very busy >>> central >>> gen_server, which was set in priority high in previous versions. This >>> improved things for some and caused issues for others. >>> >>> To fix this I took inspiration from my frequent use of gproc and ended up >>> using a similar but simpler approach (because we don't need to handle >>> that >>> many use cases). >>> >>> Many tests have been added, a simple example has been added, and most >>> importantly a complete user guide has been written. >>> >>> https://github.com/extend/ranch/blob/master/guide/toc.md >>> >>> Please forward any comment, criticism, swearing or candy to me in any way >>> you prefer. >>> >>> This version is what's going to be used in Cowboy 0.8.0. Cowboy master >>> will >>> be updated in a few days with the new code. Be careful if you still >>> depend >>> on "master" instead of "0.6.1"! >>> >>> -- >>> Lo?c Hoguin >>> >>> Erlang Cowboy >>> Nine Nines >>> http://ninenines.eu >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Lo?c Hoguin > > Erlang Cowboy > Nine Nines > http://ninenines.eu From dang@REDACTED Wed Aug 22 17:16:51 2012 From: dang@REDACTED (Daniel Goertzen) Date: Wed, 22 Aug 2012 10:16:51 -0500 Subject: [erlang-questions] calling Erlang functions from linked-in driver In-Reply-To: References: Message-ID: Take a look at... http://www.erlang-factory.com/upload/presentations/377/RickardGreen-NativeInterface.pdf Native processes are presently not available, but the above pdf may give you insights about how to design your own system. Dan. On Wed, Aug 22, 2012 at 1:48 AM, Motiejus Jak?tys wrote: > Hi, > > We are looking for a way to call Erlang functions from linked-in port > driver (or NIF). From documentation I cannot see any way to do it. We > want to call some Erlang functions from Lua code[1], which is executed > in linked-in driver (can be re-implemented in NIF). > > In C node case this would be simple. However, for this C node is a bit > overkill (we want erlualib side effects to be as small as possible). > Linked-in drivers and NIFs are conceptually perfect for it. > > Is there a way to call Erlang function from linked-in driver or NIF > *before* returning the result to emulator? > > Thanks, > Motiejus Jak?tys > > [1]: https://github.com/Motiejus/erlualib > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- *Daniel Goertzen | Senior Software Engineer* Office: 828.610.4596 | Fax: 828.322.5294 | dang@REDACTED *Network Integrity Systems | We Bring Security To Light?* 1937 Tate Blvd. SE**** Hickory, North Carolina, USA 28602 *Network Integrity Systems? INTERCEPTOR? Optical Network Security System is a Smart-PDS? that ensures superior protection and cost effectiveness of classified networks. For more information, visit our website at: www.networkintegritysystems.com.* __________________________________________ INTERCEPTOR? Optical Network Security System is made in the USA for the USA. Although not an export controlled item, because of the role it plays in the assurance of the safety and integrity of National Security Information, Network Integrity Systems (NIS) is committed to compliance with the U.S. Export Administration Act. Accordingly, NIS will not ship INTERCEPTOR products to certain foreign government end users without U.S. government approval and will refuse transactions with individuals or entities that have been denied export privileges. -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Wed Aug 22 17:43:04 2012 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Wed, 22 Aug 2012 17:43:04 +0200 Subject: [erlang-questions] calling Erlang functions from linked-in driver In-Reply-To: References: Message-ID: <20120822154304.GA6969@precise> On Wed, Aug 22, 2012 at 10:16:51AM -0500, Daniel Goertzen wrote: > Take a look at... > > http://www.erlang-factory.com/upload/presentations/377/RickardGreen-NativeInterface.pdf > > Native processes are presently not available, but the above pdf may give > you insights about how to design your own system. Thanks! This is exactly what I am looking for: void enif_nproc_schedule_ecall(ErlNifEnv *proc_env, ERL_NIF_TERM module, ERL_NIF_TERM function, ERL_NIF_TERM arguments, void (*result_func)(ErlNifEnv *proc_env, void *state, ErlNProcResultType result_type, ERL_NIF_TERM result)); And the concept seems to be very tempting. And talking about this (from the one slide before the last): When will these features show up? ?Hopefully R15, but this is no promise Any update? :) For now, we decided to stick to NIFs and execute C-to-Erlang calls in a separate thread. Motiejus -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From mailparmalat@REDACTED Wed Aug 22 21:00:36 2012 From: mailparmalat@REDACTED (=?GB2312?B?087Wx4Kl?=) Date: Wed, 22 Aug 2012 21:00:36 +0200 Subject: [erlang-questions] =?gb2312?b?ZXRzse3X7rTzyv3T2m1uZXNpYcrCzvE=?= =?gb2312?b?zsrM4g==?= In-Reply-To: References: Message-ID: Hi, 1 -> ?????O????????????????atoms???????????????_??ets table??????????atoms. Atom????????1048576 per runtime. 2 -> ??????????????: (test@REDACTED)4> length(ets:all()). 90019 3 -> ???_runtimes???? Regards, 2012/8/22 ?????? > > ???????? > ?????????????????????????????? > ?????????????????? "Cannot create an ets table for the local transaction store", > ???????????? -env ERL_MAX_ETS_TABLES 24000 ???????????????????????????? > > ?????????????? > 1????????????????1400,???????????????????????????????????????????? > 2????????????????????24000 ???????? > 3??????mnesia??????????ets????????ets?????????????????????????????????????????????????? > > Thanks > ?????? > > > > The number of tables stored at one Erlang node is limited. The current > default limit is approximately 1400 tables. The upper limit can be > increased by setting the environment variable ERL_MAX_ETS_TABLES before > starting the Erlang runtime system > > > -- > ?????????????????????????????? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Aug 23 01:31:18 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 23 Aug 2012 11:31:18 +1200 Subject: [erlang-questions] Function Dependency In-Reply-To: <20120822080328.GA3248@corelatus.se> References: <3783E7DF-FEBB-4E2C-9570-4F6F6DC13BA9@cs.otago.ac.nz> <25BB25D0-B346-47D1-88BF-2241AB49A2B9@cs.otago.ac.nz> <20120822080328.GA3248@corelatus.se> Message-ID: <706BF901-DBCC-4DA7-B199-57F9E07D289E@cs.otago.ac.nz> A signature is basically a type for a module or a(n ML-style) functor. The 'class signatures' that G++ has (or had) are that sort of thing. In Python 3.3 http://www.python.org/dev/peps/pep-0362/ a signature is an object describing everything that is known about the interface of a function: two functions may well have equal signatures. C++ is seriously weird here (as ever): 1.3.10 signature [defns.signature] the information about a function that participates in overload resolution (13.3): the types of its parameters and, if the function is a class member, the cv- qualifiers (if any) on the function itself and the class in which the member function is declared. 2) The signature of a function template specialization includes the types of its template arguments (14.5.5.1). ... 2) Function signatures do not include return type, because that does not participate in overload resolution. Like Python, it does *NOT* include the name of the function. Unlike practically everyone, it also doesn't include the result type. From wgwi@REDACTED Thu Aug 23 02:24:40 2012 From: wgwi@REDACTED (Wang Wei) Date: Thu, 23 Aug 2012 08:24:40 +0800 Subject: [erlang-questions] =?utf-8?b?ZXRz6KGo5pyA5aSn5pWw5LqObW5lc2lh5LqL?= =?utf-8?b?5Yqh6Zeu6aKY?= In-Reply-To: References: Message-ID: <50357848.5080605@sxu.edu.cn> hello, 1. ??????????,??ets?????. 2. 24000?????,????????.?????. 3. ????,?????redis?session???,????. ?? ? 2012/8/22 20:06, ??? ??: > > ???? > ??????????????? > ????????? "Cannot create an ets table for the local > transaction store", > ?????? -env ERL_MAX_ETS_TABLES 24000 ?????????????? > > ??????? > 1????????1400,?????????????????????? > 2??????????24000 ???? > 3???mnesia?????ets????ets???????????????? > ????????? > > Thanks > ??? > > > > The number of tables stored at one Erlang node is limited. The current > default limit is approximately 1400 tables. The upper limit can be > increased by setting the environment variable > ERL_MAX_ETS_TABLES before starting the Erlang runtime system > > > -- > ??????????????? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From tmr@REDACTED Thu Aug 23 14:24:08 2012 From: tmr@REDACTED (Tomas Morstein) Date: Thu, 23 Aug 2012 14:24:08 +0200 (CEST) Subject: [erlang-questions] [ANN] EGTM: embedded ACID compliant NoSQL engine for Erlang Message-ID: <96d705fa-93c6-4476-9e80-9967e80deb90@zimbra> Dear Erlang community, It's finally here! We've just released first public version of IDEA EGTM. EGTM is an Erlang application built on the top of FIS GT.M engine what is a complete implementation of M[UMPS] technology. The main characteristics of M[UMPS] is that it is both language and database, so one can directly access persistent multidimensional arrays from M language. Main benefits of GT.M itself: - it's ANSI/ISO M[UMPS] compatible - fully ACID compliant - it's _very_ fast even if your data layout is not so well optimized - it's small and embedded - it has native replication - it can be distributed across multiple nodes (remote data may be treated as local) - it's mature -- as a heart of PROFILE core-banking system, it is used in hundreds of financial institutions around the world - its Linux version is free Main benefits of EGTM: - inherits all the properties of GT.M - allows Erlang to freely call MUMPS routines and share data with M[UMPS] applications without any limitations - can be used as a data-only storage for Erlang, without a single line of M[UMPS] code, so you don't have to learn M language that may not look so friendly (but in fact, is very simple and powerful!) - EGTM can be deployed on a private secondary GT.M replica instance just to make some data mining (web reporting via ChicagoBoss, for example) without affecting master database - can be used with any Erlang application as well as with ChicagoBoss or any other web framework - through layered software (from IDEA :) ) like IODB, it's possible to map EGTM tree structures to Erlang objects and optionally access them via SQL - IODB is not publicly available yet (coming soon!), but we use it in production and at the moment in conjunction with ChicagoBoss integration module, although the model compiler is ready to be used standalone - another layered software is EGTM HAC what is EGTM HighAvailibility Cluster extension -- the best of HA properties of native GT.M and Erlang distributed properties Even if the performance of any external binding will never be as good as native M code, it should be enough for many applications. If the performance is a primary requirement, it's possible to implement complex tasks with native M and simply call the native routine from EGTM. An example of application that is written in ChicagoBoss framework with pure EGTM (completely without IODB objects) is our website http://www.idea.cz. I am not able to do it immediately, but the source code may be released as a sample CB+EGTM reference project later. Some resources: - EGTM on GitHub: http://github.com/ztmr/egtm - EGTM "homepage": http://labs.idea.cz/egtm - GT.M documentation: http://www.mumps.cz/gtm/ - slides about GT.M: http://www.mumps.cz/gtm/misc/120730-1agtmasnosqldatabase.pdf - introduction to M-based systems for relational people: http://www.fooboo.org/~tmr/gtm/UniversalNoSQL.pdf In meantime of posting this message, there were started several discussions in another Erlang-related groups: - https://groups.google.com/forum/#!forum/chicagoboss - https://www.linkedin.com/groups/EGTM-embedded-ACID-compliant-NoSQL-90878.S.149660834 Enjoy and feel free to ask! Tom From Sergey_Zhemzhitsky@REDACTED Thu Aug 23 14:48:08 2012 From: Sergey_Zhemzhitsky@REDACTED (Zhemzhitsky Sergey) Date: Thu, 23 Aug 2012 12:48:08 +0000 Subject: [erlang-questions] How to convert erlang_pid structure into ErlDrvTermData? Message-ID: <06139A918ACCA041BF46A0F36940C7FA46291FC1@exch-mbx1.msk.trd.ru> Hi guys, I need to convert erlang_pid structure into ErlDrvTermData that will be passed to driver_send_term(ErlDrvPort ix, ErlDrvTermData to, ErlDrvTermData* data, int len) function in linked-in driver. Is it possible to achieve? Best Regards, Sergey _______________________________________________________ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Thu Aug 23 14:53:49 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 23 Aug 2012 14:53:49 +0200 Subject: [erlang-questions] [ANN] EGTM: embedded ACID compliant NoSQL engine for Erlang In-Reply-To: <96d705fa-93c6-4476-9e80-9967e80deb90@zimbra> References: <96d705fa-93c6-4476-9e80-9967e80deb90@zimbra> Message-ID: Hi Tomas, Didn't hear about MUPS nor GT.M before. It's simply awesome. Thanks for sharing !!! Can I imagine "egmt" as a good replacement to my RIAK store (for fast key/value retrieval)? Any pratical benchs? Finally, for when are you planning to release "iodb" (http://www.idea.cz/technology)? Regards, Zabrane On Aug 23, 2012, at 2:24 PM, Tomas Morstein wrote: > Dear Erlang community, > > It's finally here! We've just released first public version of IDEA EGTM. > > EGTM is an Erlang application built on the top of FIS GT.M engine what is a complete implementation of M[UMPS] technology. > The main characteristics of M[UMPS] is that it is both language and database, so one can directly access persistent multidimensional arrays from M language. > > Main benefits of GT.M itself: > - it's ANSI/ISO M[UMPS] compatible > - fully ACID compliant > - it's _very_ fast even if your data layout is not so well optimized > - it's small and embedded > - it has native replication > - it can be distributed across multiple nodes (remote data may be treated as local) > - it's mature -- as a heart of PROFILE core-banking system, it is used in hundreds of financial institutions around the world > - its Linux version is free > > Main benefits of EGTM: > - inherits all the properties of GT.M > - allows Erlang to freely call MUMPS routines and share data with M[UMPS] applications without any limitations > - can be used as a data-only storage for Erlang, without a single line of M[UMPS] code, so you don't have to learn M language that may not look so friendly (but in fact, is very simple and powerful!) > - EGTM can be deployed on a private secondary GT.M replica instance just to make some data mining (web reporting via ChicagoBoss, for example) without affecting master database > - can be used with any Erlang application as well as with ChicagoBoss or any other web framework > - through layered software (from IDEA :) ) like IODB, it's possible to map EGTM tree structures to Erlang objects and optionally access them via SQL > - IODB is not publicly available yet (coming soon!), but we use it in production and at the moment in conjunction with ChicagoBoss integration module, although the model compiler is ready to be used standalone > - another layered software is EGTM HAC what is EGTM HighAvailibility Cluster extension -- the best of HA properties of native GT.M and Erlang distributed properties > > Even if the performance of any external binding will never be as good as native M code, it should be enough for many applications. If the performance is a primary requirement, it's possible to implement complex tasks with native M and simply call the native routine from EGTM. > > An example of application that is written in ChicagoBoss framework with pure EGTM (completely without IODB objects) is our website http://www.idea.cz. > I am not able to do it immediately, but the source code may be released as a sample CB+EGTM reference project later. > > Some resources: > - EGTM on GitHub: http://github.com/ztmr/egtm > - EGTM "homepage": http://labs.idea.cz/egtm > - GT.M documentation: http://www.mumps.cz/gtm/ > - slides about GT.M: http://www.mumps.cz/gtm/misc/120730-1agtmasnosqldatabase.pdf > - introduction to M-based systems for relational people: http://www.fooboo.org/~tmr/gtm/UniversalNoSQL.pdf > > In meantime of posting this message, there were started several discussions in another Erlang-related groups: > - https://groups.google.com/forum/#!forum/chicagoboss > - https://www.linkedin.com/groups/EGTM-embedded-ACID-compliant-NoSQL-90878.S.149660834 > > Enjoy and feel free to ask! > > Tom > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivier.girondel@REDACTED Thu Aug 23 17:02:17 2012 From: olivier.girondel@REDACTED (Olivier Girondel) Date: Thu, 23 Aug 2012 17:02:17 +0200 Subject: [erlang-questions] Exit status of C-port Message-ID: Dear all, I have a C executable which is run with open_port (spawn_executable, streaming binary and getting exit_status) If/when the C code crashes, I (most of the time) correctly get an exit message (eg: {exit_status, 134}), which is exactly what I want. Sometimes though, I get an "exception exit" with the reason "epipe", which means the port really crashed in a nasty way; but this exception is not catchable. Apart from doing this another way (-yes, I really do want to spawn()/run a program that might crash-), what are the options ? Thanks, -- Olivier From tmr@REDACTED Fri Aug 24 02:20:19 2012 From: tmr@REDACTED (Tomas Morstein) Date: Fri, 24 Aug 2012 02:20:19 +0200 (CEST) Subject: [erlang-questions] [ANN] EGTM: embedded ACID compliant NoSQL engine for Erlang In-Reply-To: Message-ID: <889eb799-5d25-4297-8006-01b28e73f48d@zimbra> Hi, Yes, MUMPS is somehow "hidden" from mainstream for more over 40 years (it was operating system for DEC PDP minicomputers, similarly like early UNIX versions). Maybe it was also the reason why one of its commercial implementations is called Cach? (= hidden in French) :-) To answer your questions: (1) to be honest, I don't like benchmarks so much. We don't know RIAK well enough to compare to. I also think that RIAK is much more high-level DB than GT.M/EGTM. If you're asking for speed, I can tell you only stories about M compared to relational databases (MySQL, PostgreSQL, Oracle, MS-SQL). In short, even if we wanted to be fair and artificially slowed down the M task by significant anti-optimalizations, the SQL did the job for hours where it took a few minutes in MUMPS. I will not share these "benchmarks" publicly as they're good only for marketing, not for any fair, serious and meaningful technology competition. The number of factors and input variables is not small and especially when it comes to NoSQL and distributed environment, I'd say it's not comparable at all. But because I am interested too, let's open a competition like this: - try to define a sensible test scenario (what's the goal, what we're going to measure); - implement it with RIAK; - I will implement the same scenario with GT.M in meantime (in both variants: native M code and EGTM-powered Erlang) - we can compare then... (2) according to IODB, it is something we're using in production, but it's not ready to be released to public yet. Actually, we're working on a brand new model compiler (what unlocks a lot of new features as well as much better performance) and we will probably change some general API functions to make it more comfortable to use. So the only I can say at the moment is: stay tuned! :-) Since we use some IODB objects from JavaScript, we have also a simple IODB REST API implemented as a ChicagoBoss controller what is a part of IDEA CloudOS where we also plan some web GUI management (EGTM Global browser, IODB model manager, etc.), but this is definitely on our wishlist... (3) what actually came into my mind (but I am not sure about its value in practical life): - EGTM seems to be a good adept to act as a Mnesia backend (it can break all the DETS limits I am aware of) - what about employing EGTM as a RIAK storage backend? :-) (For example, I have experimented with GT.M as a backend for OpenLDAP around three years ago.) Regards, Tom. ----- Original Message ----- > Od: "Zabrane Mickael" > Komu: "Tomas Morstein" > Kopie: erlang-questions@REDACTED > Odeslan?: ?tvrtek, 23. Srpen 2012 14:53:49 > P?edm?t: Re: [erlang-questions] [ANN] EGTM: embedded ACID compliant NoSQL engine for Erlang > > Hi Tomas, > > > Didn't hear about MUPS nor GT.M before. > It's simply awesome. Thanks for sharing !!! > > > Can I imagine "egmt" as a good replacement to my RIAK store (for fast > key/value retrieval)? > Any pratical benchs? > > > Finally, for when are you planning to release "iodb" ( > http://www.idea.cz/technology )? > > > > Regards, > Zabrane > > > > On Aug 23, 2012, at 2:24 PM, Tomas Morstein wrote: > > > > Dear Erlang community, > > It's finally here! We've just released first public version of IDEA > EGTM. > > EGTM is an Erlang application built on the top of FIS GT.M engine > what is a complete implementation of M[UMPS] technology. > The main characteristics of M[UMPS] is that it is both language and > database, so one can directly access persistent multidimensional > arrays from M language. > > Main benefits of GT.M itself: > - it's ANSI/ISO M[UMPS] compatible > - fully ACID compliant > - it's _very_ fast even if your data layout is not so well optimized > - it's small and embedded > - it has native replication > - it can be distributed across multiple nodes (remote data may be > treated as local) > - it's mature -- as a heart of PROFILE core-banking system, it is > used in hundreds of financial institutions around the world > - its Linux version is free > > Main benefits of EGTM: > - inherits all the properties of GT.M > - allows Erlang to freely call MUMPS routines and share data with > M[UMPS] applications without any limitations > - can be used as a data-only storage for Erlang, without a single > line of M[UMPS] code, so you don't have to learn M language that may > not look so friendly (but in fact, is very simple and powerful!) > - EGTM can be deployed on a private secondary GT.M replica instance > just to make some data mining (web reporting via ChicagoBoss, for > example) without affecting master database > - can be used with any Erlang application as well as with ChicagoBoss > or any other web framework > - through layered software (from IDEA :) ) like IODB, it's possible > to map EGTM tree structures to Erlang objects and optionally access > them via SQL > - IODB is not publicly available yet (coming soon!), but we use it in > production and at the moment in conjunction with ChicagoBoss > integration module, although the model compiler is ready to be used > standalone > - another layered software is EGTM HAC what is EGTM HighAvailibility > Cluster extension -- the best of HA properties of native GT.M and > Erlang distributed properties > > Even if the performance of any external binding will never be as good > as native M code, it should be enough for many applications. If the > performance is a primary requirement, it's possible to implement > complex tasks with native M and simply call the native routine from > EGTM. > > An example of application that is written in ChicagoBoss framework > with pure EGTM (completely without IODB objects) is our website > http://www.idea.cz . > I am not able to do it immediately, but the source code may be > released as a sample CB+EGTM reference project later. > > Some resources: > - EGTM on GitHub: http://github.com/ztmr/egtm > - EGTM "homepage": http://labs.idea.cz/egtm > - GT.M documentation: http://www.mumps.cz/gtm/ > - slides about GT.M: > http://www.mumps.cz/gtm/misc/120730-1agtmasnosqldatabase.pdf > - introduction to M-based systems for relational people: > http://www.fooboo.org/~tmr/gtm/UniversalNoSQL.pdf > > In meantime of posting this message, there were started several > discussions in another Erlang-related groups: > - https://groups.google.com/forum/#!forum/chicagoboss > - > https://www.linkedin.com/groups/EGTM-embedded-ACID-compliant-NoSQL-90878.S.149660834 > > Enjoy and feel free to ask! > > Tom > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vincent.dephily@REDACTED Fri Aug 24 10:56:27 2012 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Fri, 24 Aug 2012 10:56:27 +0200 Subject: [erlang-questions] Function Dependency In-Reply-To: References: Message-ID: <4495250.HsF5vaJLeP@moltowork> On Tuesday 21 August 2012 08:23:17 Tyron Zerafa wrote: > I need to > transfer only the code that a function makes use of between nodes, nothing > more. Xref is giving me the signature of all those functions that I need. > Now, I want to use these signatures to extract the remaining body of the > functions. You might want to look at systools:make_tar/2, which does something fairly close to what you describe. You'll still want to make sure that both machines have the same erlang release installed beforehand. -- Vincent de Phily Mobile Devices +33 (0) 142 119 325 +353 (0) 85 710 6320 Warning This message (and any associated files) is intended only for the use of its intended recipient and may contain information that is confidential, subject to copyright or constitutes a trade secret. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this message, or files associated with this message, is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. Any views or opinions presented are solely those of the author vincent.dephily@REDACTED and do not necessarily represent those of the company. Although the company has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. From jozsef.berces@REDACTED Fri Aug 24 11:01:25 2012 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces?=) Date: Fri, 24 Aug 2012 11:01:25 +0200 Subject: [erlang-questions] Why Erlang Message-ID: <3717CEF51B134A4F8073627839A9DF412A689F78D1@ESESSCMS0353.eemea.ericsson.se> Hi, I would like to suggest Erlang for an internal project. Do we have any good presentation that - can be shown to relatively high level people so it is not too long - lists the main characteristics and advantages of Erlang - has a few arguments why Erlang is a better choice than others (e.g. than a Perl script) Thanks, Jozsef -------------- next part -------------- An HTML attachment was scrubbed... URL: From vincent.dephily@REDACTED Fri Aug 24 11:16:28 2012 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Fri, 24 Aug 2012 11:16:28 +0200 Subject: [erlang-questions] [ANN] EGTM: embedded ACID compliant NoSQL engine for Erlang In-Reply-To: References: <96d705fa-93c6-4476-9e80-9967e80deb90@zimbra> Message-ID: <5030896.bX6DiK1xmJ@moltowork> On Thursday 23 August 2012 14:53:49 Zabrane Mickael wrote: > Hi Tomas, > > Didn't hear about MUPS nor GT.M before. > It's simply awesome. Thanks for sharing !!! You might want to avoid googling for "wtf mumps" to keep that spirit up then :p Sorry, couldn't resist, and I fully admit that I know very little about mumps and that there's certainly good reasons why people are using it. -- Vincent de Phily From essen@REDACTED Fri Aug 24 12:07:07 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Fri, 24 Aug 2012 12:07:07 +0200 Subject: [erlang-questions] Why Erlang In-Reply-To: <3717CEF51B134A4F8073627839A9DF412A689F78D1@ESESSCMS0353.eemea.ericsson.se> References: <3717CEF51B134A4F8073627839A9DF412A689F78D1@ESESSCMS0353.eemea.ericsson.se> Message-ID: <5037524B.30708@ninenines.eu> I'd suggest looking at this talk: http://www.erlang-factory.com/conference/London2011/speakers/MikeWilliams On 08/24/2012 11:01 AM, J?zsef B?rces wrote: > Hi, > I would like to suggest Erlang for an internal project. Do we have any > good presentation that > - can be shown to relatively high level people so it is not too long > - lists the main characteristics and advantages of Erlang > - has a few arguments why Erlang is a better choice than others (e.g. > than a Perl script) > Thanks, > Jozsef > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From f@REDACTED Fri Aug 24 12:18:30 2012 From: f@REDACTED (Francesco Mazzoli) Date: Fri, 24 Aug 2012 11:18:30 +0100 Subject: [erlang-questions] `mnesia:change_table_copy_type/3' issues when changing from disc to RAM In-Reply-To: <87r4qzkwe3.wl%f@mazzo.li> References: <87r4qzkwe3.wl%f@mazzo.li> Message-ID: <87393c61ah.wl%f@mazzo.li> It turns out that this limitation also affects deleting disc replicas (if other disc replicas are down, you'll get an error). There is no mention if this limitation in the docs for `change_table_copy/3' or `del_table_copy/3'. -- Francesco * Often in error, never in doubt From tomas.morstein@REDACTED Fri Aug 24 13:02:55 2012 From: tomas.morstein@REDACTED (Tomas Morstein) Date: Fri, 24 Aug 2012 13:02:55 +0200 (CEST) Subject: [erlang-questions] [ANN] EGTM: embedded ACID compliant NoSQL engine for Erlang In-Reply-To: <5030896.bX6DiK1xmJ@moltowork> Message-ID: <06f98646-5d29-446f-9b5e-b83677996082@zimbra> > > Didn't hear about MUPS nor GT.M before. > > It's simply awesome. Thanks for sharing !!! > > You might want to avoid googling for "wtf mumps" to keep that spirit > up then > :p At first, just a refreshment: EGTM's goal was to put the best from Erlang and MUMPS together :-) So well, it's true that many people hate the M _language_ because for historical reasons, it has some features resulting in situations where a M begineer is not able to understand the code [*]. The EGTM is quite different because you don't need to touch such a MUMPS code! Just write a regular Erlang code and use MUMPS only for its good database properties, don't care about the rest. If you need to do some performance tuning, you can write M code in style of your own preference, but you don't have to. If you want to do Oracle tuning, you don't need to learn a new, maybe crazy, language, but you need to become an Oracle internals wizard. And from my point of view, I'd go for learning a new language :) ----- [*] back to these "language features": Sure, it is possible to write inline code with a commands of single character, but it is only _possible_ shortcut. Many modern M developers use VeryLongCamelNaming.Possibly.With.Some.NameSpaces :-) For example, Cach? Studio (their IDE) has a button that will expand/collapse all the code in your buffer to short/long form. So you open a routine with code like this: S X="" F S X=$O(^Foo(X)) Q:X="" W X,! and if you click to expand button, it makes: Set X="" For Set X=$Order(^Foo(X)) Quit:X="" Write X,! and vice versa. The original short code was good when transfering source code over serial lines in past. Imagine, that for example OpenVista healthcare project has around 25.000 routines, what is 331 MB of raw source code with over than 2.000.000 LoC. (Not counting their GUI client application.) I guess the expanded source can grow up to 1G what's not much today, but for example on VAX computer with 32M RAM, 25MHz CPU and 1G disk, serving to a big hospital with hundreds of users... (And since we do migrations of legacy hardware, I can confirm there're lots of mission critical servers with similar or lower configration still in production!) > Sorry, couldn't resist, and I fully admit that I know very little > about mumps > and that there's certainly good reasons why people are using it. Yep, this is a realistic point of view. Many people judge what they never saw and that's an infinite source of many rumours. From spawn.think@REDACTED Fri Aug 24 14:28:56 2012 From: spawn.think@REDACTED (Ahmed Omar) Date: Fri, 24 Aug 2012 14:28:56 +0200 Subject: [erlang-questions] How to convert erlang_pid structure into ErlDrvTermData? In-Reply-To: <06139A918ACCA041BF46A0F36940C7FA46291FC1@exch-mbx1.msk.trd.ru> References: <06139A918ACCA041BF46A0F36940C7FA46291FC1@exch-mbx1.msk.trd.ru> Message-ID: Hi Sergey, I think what you are looking for is http://www.erlang.org/doc/man/erl_driver.html#driver_caller "This function returns the process id of the process that made the current call to the driver. The process id can be used withdriver_send_term to send back data to the caller". or http://www.erlang.org/doc/man/erl_driver.html#driver_connected On Thu, Aug 23, 2012 at 2:48 PM, Zhemzhitsky Sergey < Sergey_Zhemzhitsky@REDACTED> wrote: > Hi guys,**** > > ** ** > > I need to convert erlang_pid structure into ErlDrvTermData that will be > passed to **** > > driver_send_term(ErlDrvPort ix, ErlDrvTermData to, ErlDrvTermData* data, > int len) function in linked-in driver.**** > > ** ** > > Is it possible to achieve? **** > > ** ** > > Best Regards,**** > > Sergey**** > > ** ** > > _______________________________________________________ > > > > The information contained in this message may be privileged and conf > idential and protected from disclosure. If you are not the original > intended recipient, you are hereby notified that any review, > retransmission, dissemination, or other use of, or taking of any action in > reliance upon, this information is prohibited. If you have received this > communication in error, please notify the sender immediately by replying to > this message and delete it from your computer. Thank you for your > cooperation. Troika Dialog, Russia. > > If you need assistance please contact our Contact Center (+7495) 258 0500or go to > www.troika.ru/eng/Contacts/system.wbp > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Fri Aug 24 14:45:08 2012 From: erlangsiri@REDACTED (Siri Hansen) Date: Fri, 24 Aug 2012 14:45:08 +0200 Subject: [erlang-questions] "Empty" appups on the form {"", [], []} in OTP In-Reply-To: References: Message-ID: Hi Ulf! In general type of empty appup files indicate that the application is not prepared to be upgraded in a live system, e.g. because it is a tool which is not expected to be included in a live system at all. However, we have lately learned that there are applications with empty appup files that *should* be possible to upgrade, and these must be corrected. We are working on the priorities. Are there any application(s) in particular that you would flag as high prio? Regards /siri 2012/8/20 Ulf Leopold > Hi All, > > I see that many apps in OTP (R14 & R15) have appups on the form: > > {"",[],[]} > > systools:make_relup does not seem to like them. How should they be > interpreted? Does it mean that this particular app has no sanctioned > upgrade/downgrade instruction? > > Thanks, > > Ulf > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Fri Aug 24 15:33:31 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 24 Aug 2012 09:33:31 -0400 Subject: [erlang-questions] Poll on Erlang maintenance, according to Erlang developers Message-ID: <503782AB.4050507@ferd.ca> Hi everyone, in an attempt to better understand the ecosystem we Erlang developers live in, I've decided to create a poll on what fellow developers consider important when working with an Erlang code base that isn't theirs. The poll can be filled at http://ferd.ca/poll/ and I will be thankful for all answers we get. I'll come back with the results later, if enough people want to know about them. Thanks for your (potential) participation, Regards, Fred. From bourinov@REDACTED Fri Aug 24 16:16:51 2012 From: bourinov@REDACTED (Max Bourinov) Date: Fri, 24 Aug 2012 18:16:51 +0400 Subject: [erlang-questions] count events within last XX minutes Message-ID: Dear Erlangers, Does anybody know best memory efficient way to count events within last XX minutes? So far I have the following idea: For each counter I have a process. The counter process has a list of XX items. Each item represents a certain minute, so I always know where is my current counter. There is also must be a mechanism to remove last item from the list and add a new one. Summing values from all items is a number of events within last XX minutes. This is it. Maybe there is a ready lib that does the same in a better way? Any suggestions are welcome! Best regards, Max -------------- next part -------------- An HTML attachment was scrubbed... URL: From f@REDACTED Fri Aug 24 16:45:47 2012 From: f@REDACTED (Francesco Mazzoli) Date: Fri, 24 Aug 2012 15:45:47 +0100 Subject: [erlang-questions] edoc: documenting behaviours callback and preprocessor macros Message-ID: <87y5l44ack.wl%f@mazzo.li> Hi list, I have two questions about edoc: 1. Is it possible to document behaviour callbacks? I have to use the old `behaviour_info/1' form for compatibility reason. 2. Is there any nice way to embed values defined in preprocessor macros in edoc comments? I'm pretty sure that the answer is "no", but you never know. -- Francesco * Often in error, never in doubt From essen@REDACTED Fri Aug 24 16:46:12 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Fri, 24 Aug 2012 16:46:12 +0200 Subject: [erlang-questions] edoc: documenting behaviours callback and preprocessor macros In-Reply-To: <87y5l44ack.wl%f@mazzo.li> References: <87y5l44ack.wl%f@mazzo.li> Message-ID: <503793B4.5050503@ninenines.eu> On 08/24/2012 04:45 PM, Francesco Mazzoli wrote: > Hi list, > > I have two questions about edoc: > > 1. Is it possible to document behaviour callbacks? I have to use the old > `behaviour_info/1' form for compatibility reason. Not answering but related: I got an error when trying to @doc a -callback (R15B01), it would be really nice if I could do that. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From anders.nygren@REDACTED Fri Aug 24 17:14:51 2012 From: anders.nygren@REDACTED (Anders Nygren) Date: Fri, 24 Aug 2012 10:14:51 -0500 Subject: [erlang-questions] count events within last XX minutes In-Reply-To: References: Message-ID: Use an ETS table with {Counter,{YYYY,MM,DD,HH,MM}} as key, and ets:update_counter/2,3. update_counter returns the new counter value, so if it is 1, (or the increment used) You know that a new minute has been entered so You can delete the oldest. /Anders On Fri, Aug 24, 2012 at 9:16 AM, Max Bourinov wrote: > Dear Erlangers, > > Does anybody know best memory efficient way to count events within last XX > minutes? > > So far I have the following idea: For each counter I have a process. The > counter process has a list of XX items. Each item represents a certain > minute, so I always know where is my current counter. There is also must be > a mechanism to remove last item from the list and add a new one. Summing > values from all items is a number of events within last XX minutes. This is > it. > > Maybe there is a ready lib that does the same in a better way? > > Any suggestions are welcome! > > Best regards, > Max > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From brum76@REDACTED Fri Aug 24 17:53:20 2012 From: brum76@REDACTED (Radu Brumariu) Date: Fri, 24 Aug 2012 11:53:20 -0400 Subject: [erlang-questions] Why Erlang In-Reply-To: <5037524B.30708@ninenines.eu> References: <3717CEF51B134A4F8073627839A9DF412A689F78D1@ESESSCMS0353.eemea.ericsson.se> <5037524B.30708@ninenines.eu> Message-ID: This talk is how Erlang was pitched and used inside Yahoo : http://www.erlang-factory.com/conference/SFBay2010/speakers/MarkZweifel Radu On Fri, Aug 24, 2012 at 6:07 AM, Lo?c Hoguin wrote: > I'd suggest looking at this talk: > > http://www.erlang-factory.com/conference/London2011/speakers/MikeWilliams > > > On 08/24/2012 11:01 AM, J?zsef B?rces wrote: >> >> Hi, >> I would like to suggest Erlang for an internal project. Do we have any >> good presentation that >> - can be shown to relatively high level people so it is not too long >> - lists the main characteristics and advantages of Erlang >> - has a few arguments why Erlang is a better choice than others (e.g. >> than a Perl script) >> Thanks, >> Jozsef >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From magnus.henoch@REDACTED Fri Aug 24 17:55:12 2012 From: magnus.henoch@REDACTED (Magnus Henoch) Date: Fri, 24 Aug 2012 16:55:12 +0100 Subject: [erlang-questions] edoc: documenting behaviours callback and preprocessor macros In-Reply-To: <87y5l44ack.wl%f@mazzo.li> (Francesco Mazzoli's message of "Fri, 24 Aug 2012 15:45:47 +0100") References: <87y5l44ack.wl%f@mazzo.li> Message-ID: Francesco Mazzoli writes: > 1. Is it possible to document behaviour callbacks? I have to use the old > `behaviour_info/1' form for compatibility reason. There is a patch for making edoc list the callbacks, so that you're no worse off than with behaviour_info: https://github.com/erlang/otp/commit/715bb8044600ec16ad6ba97de9d033f6a5c06bfe It is lined up to be released in R15B02, due 5 September. Of course, it would be nice if edoc would display the callback types, and allow documenting callbacks like it does with -type declarations, but I'm not aware of anyone working on that. Regards, Magnus From gleber.p@REDACTED Fri Aug 24 19:41:54 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 24 Aug 2012 19:41:54 +0200 Subject: [erlang-questions] Semantic Versioning 2.0 library - erlsemver Message-ID: Hello folks! I am glad to present a Semantic Versioning parsing library which is compatible with 2.0.0-rc.1 version of SemVer. Feel free to grab it from: https://github.com/gleber/erlsemver or with git clone http://github.com/gleber/erlsemver.git README contains examples of usage. Interesting features: - allows for direct comparison of #semver{} records to determine version precedence - just by using > and < operators - supports SemVerTag specification in form of "to_tag/1" and "from_tag/1" functions - supports parsing of 'git describe' output and treats it as build version - has basic unit tests Thanks for LivePress Inc. for allowing to open source this library! Cheers, Gleb P.S. I am aware of semver library by nebularis, but it targets 1.0 specification and erlsemver was much earlier than semver by nebularis was announced. From moxford@REDACTED Fri Aug 24 22:05:03 2012 From: moxford@REDACTED (Mike Oxford) Date: Fri, 24 Aug 2012 13:05:03 -0700 Subject: [erlang-questions] count events within last XX minutes In-Reply-To: References: Message-ID: If you care about, as you say, "certain minute" you can use Ulf's gproc. If you want "last minute" then the ETS solution below will work well, with the modification on the Counter to use a ms/us timer like erlang:now(). -mox On Fri, Aug 24, 2012 at 8:14 AM, Anders Nygren wrote: > Use an ETS table with {Counter,{YYYY,MM,DD,HH,MM}} as key, and > ets:update_counter/2,3. > update_counter returns the new counter value, so if it is 1, (or the > increment used) You know that a new minute has been entered so You can > delete the oldest. > > /Anders > > On Fri, Aug 24, 2012 at 9:16 AM, Max Bourinov wrote: > > Dear Erlangers, > > > > Does anybody know best memory efficient way to count events within last > XX > > minutes? > > > > So far I have the following idea: For each counter I have a process. The > > counter process has a list of XX items. Each item represents a certain > > minute, so I always know where is my current counter. There is also must > be > > a mechanism to remove last item from the list and add a new one. Summing > > values from all items is a number of events within last XX minutes. This > is > > it. > > > > Maybe there is a ready lib that does the same in a better way? > > > > Any suggestions are welcome! > > > > Best regards, > > Max > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonas.falkevik@REDACTED Fri Aug 24 22:08:04 2012 From: jonas.falkevik@REDACTED (Jonas Falkevik) Date: Fri, 24 Aug 2012 22:08:04 +0200 Subject: [erlang-questions] Exit status of C-port In-Reply-To: References: Message-ID: The exception you see might be that you received a exit signal from the port. Try setting the process flag "trap_exit" to true, then you will exit signal as a message to process linked to the port. /Jonas On Aug 23, 2012, at 17:02 , Olivier Girondel wrote: > Dear all, > > I have a C executable which is run with open_port (spawn_executable, > streaming binary > and getting exit_status) > > If/when the C code crashes, I (most of the time) correctly get an exit message > (eg: {exit_status, 134}), which is exactly what I want. > > Sometimes though, I get an "exception exit" with the reason "epipe", which means > the port really crashed in a nasty way; but this exception is not catchable. > > Apart from doing this another way (-yes, I really do want to > spawn()/run a program that might crash-), > what are the options ? > > Thanks, > > -- > Olivier > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From desired.mta@REDACTED Fri Aug 24 22:22:29 2012 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Fri, 24 Aug 2012 22:22:29 +0200 Subject: [erlang-questions] Semantic Versioning 2.0 library - erlsemver In-Reply-To: References: Message-ID: <20120824202229.GA2038@precise> On Fri, Aug 24, 2012 at 07:41:54PM +0200, Gleb Peregud wrote: > Hello folks! > > I am glad to present a Semantic Versioning parsing library which is > compatible with 2.0.0-rc.1 version of SemVer. Feel free to grab it > from: > > https://github.com/gleber/erlsemver > > or with > > git clone http://github.com/gleber/erlsemver.git > > README contains examples of usage. Interesting features: > - allows for direct comparison of #semver{} records to determine > version precedence - just by using > and < operators > - supports SemVerTag specification in form of "to_tag/1" and > "from_tag/1" functions > - supports parsing of 'git describe' output and treats it as build version > - has basic unit tests > > Thanks for LivePress Inc. for allowing to open source this library! > > Cheers, > Gleb > > P.S. I am aware of semver library by nebularis, but it targets 1.0 > specification and erlsemver was much earlier than semver by nebularis > was announced. So now we have three ;) * erlsemver * mojombo (by nebularis) * one waiting for approval in rebar: https://github.com/basho/rebar/pull/263 /Motiejus -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From gleber.p@REDACTED Fri Aug 24 22:33:45 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 24 Aug 2012 22:33:45 +0200 Subject: [erlang-questions] Semantic Versioning 2.0 library - erlsemver In-Reply-To: <20120824202229.GA2038@precise> References: <20120824202229.GA2038@precise> Message-ID: On Fri, Aug 24, 2012 at 10:22 PM, Motiejus Jak?tys wrote: > So now we have three ;) > > * erlsemver > * mojombo (by nebularis) > * one waiting for approval in rebar: https://github.com/basho/rebar/pull/263 Hehe. Didn't knew about rebar's version. We probably should merge them all somehow into one. I believe that currently my version is the fullest implementation of SemVer specification. I'll be glad to improve it to make it useful for rebar as it seems that rebar_version.erl has a very limited implementation of SemVer :) Thijs, ping? From gleber.p@REDACTED Fri Aug 24 22:44:47 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 24 Aug 2012 22:44:47 +0200 Subject: [erlang-questions] erlgit - convenience Erlang wrapper around git executable Message-ID: Hello again folks! I am glad to present yet another project open sourced by LivePress Inc. This time it is a standalone convenience wrapper around git executable: https://github.com/gleber/erlgit Here's an example of usage, which says everything about the project itself: 1> Local = "/tmp/erlgit". "/tmp/erlgit" 2> git:clone("http://github.com/gleber/erlgit.git", Local). {ok,"Cloning into '/tmp/erlgit'...\n"} 3> git:branches(Local). ["master"] 4> git:refs(Local). [{"HEAD",'HEAD',"097428dda56dec6085747387645d6020f86cfae7"}, {"master",head,"097428dda56dec6085747387645d6020f86cfae7"}, {"origin/HEAD",remote, "097428dda56dec6085747387645d6020f86cfae7"}, {"origin/master",remote, "097428dda56dec6085747387645d6020f86cfae7"}, {"v0.3.0",tag,"be3b0ade881666286801ee8b218ecbf29da97558"}, {"v0.3.1",tag,"89f2d81be12f6034db52fd6d71d8a4b96f4ee9de"}, {"v0.3.2",tag,"3dc42981d9e4677654370614888ec3f368421240"}, {"v0.5.0",tag,"097428dda56dec6085747387645d6020f86cfae7"}] 5> git:branch(Local). "master" 6> git:tags(Local). ["v0.3.0","v0.3.1","v0.3.2","v0.5.0"] 7> os:cmd("touch "++Local++"/new_file"). [] 8> git:status_is_dirty(Local). true 9> git:status_changed_files(Local). [{untracked,"./new_file"}] 10> os:cmd("echo >> "++Local++"/README.md"). [] 11> git:status_changed_files(Local). [{modified,"./README.md"},{untracked,"./new_file"}] 12> git:add_files(Local, ["new_file"]). {ok,[]} 13> git:status_changed_files(Local). [{modified,"./README.md"},{indexed_added,"./new_file"}] 14> git:commit(Local, "update README and add file"). {ok,"[master 3825b3c] update README and add file\n 0 files changed\n create mode 100644 new_file\n"} 15> git:head(Local). "3825b3cb5e713c5145b4b74445a4d5e6187e567d" and it has some support for semantic versioning using erlsemver library: 16> rr("deps/erlsemver/include/*"). [semver] 17> git:version_tags(Local). [#semver{x = 0,y = 3,z = 0,pre = <<>>,build = undefined}, #semver{x = 0,y = 3,z = 1,pre = <<>>,build = undefined}, #semver{x = 0,y = 3,z = 2,pre = <<>>,build = undefined}, #semver{x = 0,y = 5,z = 0,pre = <<>>,build = undefined}] 18> git:tag(Local, semver:inc_z(#semver{x = 0,y = 5,z = 0,pre = <<>>,build = undefined})). #semver{x = 0,y = 5,z = 1,pre = <<>>,build = undefined} 19> lists:last(git:version_tags_commits(Local)). {#semver{x = 0,y = 5,z = 1,pre = <<>>,build = undefined}, "3825b3cb5e713c5145b4b74445a4d5e6187e567d"} I hope it will be useful for someone except for me. Cheers, Gleb Peregud From desired.mta@REDACTED Fri Aug 24 22:56:26 2012 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Fri, 24 Aug 2012 22:56:26 +0200 Subject: [erlang-questions] erlgit - convenience Erlang wrapper around git executable In-Reply-To: References: Message-ID: <20120824205626.GA2765@precise> On Fri, Aug 24, 2012 at 10:44:47PM +0200, Gleb Peregud wrote: > Hello again folks! > > I am glad to present yet another project open sourced by LivePress > Inc. This time it is a standalone convenience wrapper around git > executable: > > https://github.com/gleber/erlgit > > > I hope it will be useful for someone except for me. Funny coincidence. Today I was thinking about making libgit2[1] bindings for Erlang (I need some time-consuming project for my final year university project, ideas welcome). Would that be useful? Probably not.. And here you go :) If somebody will convince me that libgit2 bindings to Erlang is a valid/necessary use case.. I will do that. Anyone? Motiejus [1]: http://libgit2.github.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From gleber.p@REDACTED Fri Aug 24 23:00:51 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 24 Aug 2012 23:00:51 +0200 Subject: [erlang-questions] erlgit - convenience Erlang wrapper around git executable In-Reply-To: <20120824205626.GA2765@precise> References: <20120824205626.GA2765@precise> Message-ID: On Fri, Aug 24, 2012 at 10:56 PM, Motiejus Jak?tys wrote: > On Fri, Aug 24, 2012 at 10:44:47PM +0200, Gleb Peregud wrote: >> Hello again folks! >> >> I am glad to present yet another project open sourced by LivePress >> Inc. This time it is a standalone convenience wrapper around git >> executable: >> >> https://github.com/gleber/erlgit >> >> >> I hope it will be useful for someone except for me. > > Funny coincidence. Today I was thinking about making libgit2[1] bindings > for Erlang (I need some time-consuming project for my final year > university project, ideas welcome). Would that be useful? Probably > not.. And here you go :) > > If somebody will convince me that libgit2 bindings to Erlang is a > valid/necessary use case.. I will do that. Anyone? Actually there is one which is in it's infancy: https://github.com/schacon/geef - it needs a lot of love to be useful for any high-level git repository manipulations. erlgit uses git executable, which is always a bad solution in a long run, so in a long run I had a vision where geef implements low level libgit2 features and erlgit uses geef to provide a easy to use high level API. From steven.charles.davis@REDACTED Sat Aug 25 02:38:01 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Fri, 24 Aug 2012 17:38:01 -0700 (PDT) Subject: [erlang-questions] Why Erlang In-Reply-To: <3717CEF51B134A4F8073627839A9DF412A689F78D1@ESESSCMS0353.eemea.ericsson.se> References: <3717CEF51B134A4F8073627839A9DF412A689F78D1@ESESSCMS0353.eemea.ericsson.se> Message-ID: Working prototypes tend to speak volumes. On Friday, August 24, 2012 4:01:25 AM UTC-5, J?zsef B?rces wrote: > > Hi, > > I would like to suggest Erlang for an internal project. Do we have any > good presentation that > - can be shown to relatively high level people so it is not too long > - lists the main characteristics and advantages of Erlang > - has a few arguments why Erlang is a better choice than others (e.g. than > a Perl script) > > Thanks, > Jozsef > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Aug 25 08:14:10 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 25 Aug 2012 10:14:10 +0400 Subject: [erlang-questions] erlgit - convenience Erlang wrapper around git executable In-Reply-To: References: <20120824205626.GA2765@precise> Message-ID: I've just looked for something like this yesterday =) I consider, that grit https://github.com/mojombo/grit approach is best: for frequently used operations, like "read head", you should unpack git repo by yourself: read pack files, decode them and parse their content. Usage of libgit2 is nice, but it is nif. It should be used for speed. From hal.casteel@REDACTED Sat Aug 25 11:13:59 2012 From: hal.casteel@REDACTED (halcasteel) Date: Sat, 25 Aug 2012 02:13:59 -0700 (PDT) Subject: [erlang-questions] [ANN] euuid 1.0.0 - an erlang UUID implementation In-Reply-To: References: <42f23e60d04addac0a1ce88ae3aa09ee.squirrel@secure.bunix.org> Message-ID: <1345886039481-4655143.post@n4.nabble.com> http://erlang.biniou.org/erluuid/ This url is no longer valid. Do you still have your uuid code accessible somewhere? -- View this message in context: http://erlang.2086793.n4.nabble.com/ANN-euuid-1-0-0-an-erlang-UUID-implementation-tp3710485p4655143.html Sent from the Erlang Questions mailing list archive at Nabble.com. From zabrane3@REDACTED Sat Aug 25 11:26:38 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Sat, 25 Aug 2012 11:26:38 +0200 Subject: [erlang-questions] [ANN] euuid 1.0.0 - an erlang UUID implementation In-Reply-To: <1345886039481-4655143.post@n4.nabble.com> References: <42f23e60d04addac0a1ce88ae3aa09ee.squirrel@secure.bunix.org> <1345886039481-4655143.post@n4.nabble.com> Message-ID: <61974678-BDD4-4A96-99DC-3C601DABA967@gmail.com> The best implementation ever: https://github.com/okeuday/uuid Regards, Zabrane On Aug 25, 2012, at 11:13 AM, halcasteel wrote: > http://erlang.biniou.org/erluuid/ > > This url is no longer valid. > > Do you still have your uuid code accessible somewhere? > > > > -- > View this message in context: http://erlang.2086793.n4.nabble.com/ANN-euuid-1-0-0-an-erlang-UUID-implementation-tp3710485p4655143.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From szh.subs@REDACTED Sat Aug 25 12:39:19 2012 From: szh.subs@REDACTED (Sergey Zhemzhitsky) Date: Sat, 25 Aug 2012 14:39:19 +0400 Subject: [erlang-questions] How to convert erlang_pid structure into ErlDrvTermData? In-Reply-To: References: <06139A918ACCA041BF46A0F36940C7FA46291FC1@exch-mbx1.msk.trd.ru> Message-ID: <1951751844.20120825143919@gmail.com> An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Sat Aug 25 14:19:46 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Sat, 25 Aug 2012 14:19:46 +0200 Subject: [erlang-questions] erlgit - convenience Erlang wrapper around git executable In-Reply-To: References: <20120824205626.GA2765@precise> Message-ID: On Sat, Aug 25, 2012 at 8:14 AM, Max Lapshin wrote: > I've just looked for something like this yesterday =) > > I consider, that grit https://github.com/mojombo/grit approach is > best: for frequently used operations, like "read head", > you should unpack git repo by yourself: read pack files, decode them > and parse their content. > > Usage of libgit2 is nice, but it is nif. It should be used for speed. I guess all approaches are feasible depending on the needs. Currently just a wrapper around git executable is good enough for my needs, but in theory it is the worst approach, since it depends on output of user facing executable. I am glad to accept patches which uses any method. I like libgit2 approach better than manually parsing git repo, since it is much less work - and yet it works for anyone who isn't concerned about concurrency aspect. From olivier.girondel@REDACTED Sat Aug 25 17:23:33 2012 From: olivier.girondel@REDACTED (Olivier Girondel) Date: Sat, 25 Aug 2012 17:23:33 +0200 Subject: [erlang-questions] [ANN] euuid 1.0.0 - an erlang UUID implementation In-Reply-To: <1345886039481-4655143.post@n4.nabble.com> References: <42f23e60d04addac0a1ce88ae3aa09ee.squirrel@secure.bunix.org> <1345886039481-4655143.post@n4.nabble.com> Message-ID: On Sat, Aug 25, 2012 at 11:13 AM, halcasteel wrote: > http://erlang.biniou.org/erluuid/ > > This url is no longer valid. > > Do you still have your uuid code accessible somewhere? The website was moved to biniou.net: http://erlang.biniou.net/erluuid/ Things to note: this code is *very* old, it was written as a port-driver before NIFs were added, to learn how to write drivers. It might not be perfect. Also, there has been at least 3 or 4 other implementations of UUIDs since, in other projects, which may be of better quality/completeness https://github.com/okeuday/uuid -as Zabrane mentioned- seems quite good So I just put my version back online for reference :) -- Olivier From hal.casteel@REDACTED Sat Aug 25 20:41:58 2012 From: hal.casteel@REDACTED (Hal Casteel) Date: Sat, 25 Aug 2012 14:41:58 -0400 Subject: [erlang-questions] [ANN] euuid 1.0.0 - an erlang UUID implementation In-Reply-To: References: <42f23e60d04addac0a1ce88ae3aa09ee.squirrel@secure.bunix.org> <1345886039481-4655143.post@n4.nabble.com> Message-ID: Thanks I will read yours as a reference. HAL Hal Casteel Email (work related): hal.casteel@REDACTED Cellular: 980-475-5158 Direct Line (work): 704-413-6906 Toll-Free (work): 888-568-5991 May our thoughts be kind and clear; May our words and communication be kind and clear; May our actions and intentions be for the greater good of all human beings. May all be well with mankind. May the leaders of the earth protect in every way by keeping to the right path. May there be goodness for those who know the earth to be sacred. May all the worlds be happy. May the force be with you! On Sat, Aug 25, 2012 at 11:23 AM, Olivier Girondel wrote: > On Sat, Aug 25, 2012 at 11:13 AM, halcasteel wrote: >> http://erlang.biniou.org/erluuid/ >> >> This url is no longer valid. >> >> Do you still have your uuid code accessible somewhere? > > The website was moved to biniou.net: http://erlang.biniou.net/erluuid/ > > Things to note: this code is *very* old, it was written as a port-driver before > NIFs were added, to learn how to write drivers. It might not be perfect. > > Also, there has been at least 3 or 4 other implementations of UUIDs since, > in other projects, which may be of better quality/completeness > > https://github.com/okeuday/uuid -as Zabrane mentioned- seems quite good > > So I just put my version back online for reference :) > > -- > Olivier From carlsson.richard@REDACTED Sat Aug 25 21:39:05 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Sat, 25 Aug 2012 21:39:05 +0200 Subject: [erlang-questions] heart prevents beam from creating crash dumps Message-ID: <503929D9.6000003@gmail.com> We have had a long-standing problems with not getting any Erlang crash dumps at all on our live servers. I finally figured out why it happens. I have already reported this to the OTP folks, but I thought I should send a summary to the mailing lists for documentation and to give people a heads-up. The problem occurs when you start Erlang with the -heart flag (http://www.erlang.org/doc/man/heart.html). This spawns a small external C program connected through a port. From Erlang's point of view it's like any other port program. The heart program pings the Erlang side every now and then, and if it gets no reply within HEART_BEAT_TIMEOUT seconds, or if the connection to Erlang breaks, it assumes the Beam process has gone bad and kills it off with a SIGKILL, and then restarts Erlang using whatever HEART_COMMAND is set to. So far so good. Normally, when Beam detects a critical situation (e.g., out of memory) and decides to shut down, it will create an erl_crash.dump file (or whatever ERL_CRASH_DUMP is set to). This information can greatly help figuring out what went wrong. But if the system that crashed was large, the crash dump file can take quite a long time to create. In order to make it possible to restart the node (reusing the node name) while the old defunct system is still writing the crash dump, Beam wants to drop its connection to the EPMD service before it starts writing the dump, making it look like the old node has disappeared. The code that does this is the function prepare_crash_dump() in erts/emulator/sys/unix/sys.c. The problem from the perspective of the C code is that the connection to EPMD is on some unknown file descriptor (just like heart, this has been started as a port from Erlang code). The solution they chose, and which has been part of the OTP system for years, is to close _all_ file descriptors except 0-2. This certainly has the desired effect that EPMD releases the node name for reuse. But it also, when the loop gets to file descriptor 10 or thereabouts (probably depending on your system), has the effect of breaking the connection to the heart program. In these multicore days, the effect is almost instantaneous. The heart program immediately wakes up due to the broken pipe and sends SIGKILL to Beam for good measure, to make sure it's really gone, and then it starts a new Erlang node. Meanwhile, the old node is still busy closing file descriptors. Sometimes it makes it as far as 12 before SIGKILL arrives. The poor thing never has a chance to even open the crash dump file for writing. And your operations people only see a weird restart without any further clues. I don't have a good solution right now, except "don't use -heart". And it might be that one wants to separate the automatic restarting of a crashed node from the automatic killing of an unresponsive node anyway. Suggestions are welcome. /Richard From vinoski@REDACTED Sat Aug 25 21:48:42 2012 From: vinoski@REDACTED (Steve Vinoski) Date: Sat, 25 Aug 2012 15:48:42 -0400 Subject: [erlang-questions] [erlang-bugs] heart prevents beam from creating crash dumps In-Reply-To: <503929D9.6000003@gmail.com> References: <503929D9.6000003@gmail.com> Message-ID: On Sat, Aug 25, 2012 at 3:39 PM, Richard Carlsson wrote: > We have had a long-standing problems with not getting any Erlang crash dumps > at all on our live servers. I finally figured out why it happens. I have > already reported this to the OTP folks, but I thought I should send a > summary to the mailing lists for documentation and to give people a > heads-up. > > The problem occurs when you start Erlang with the -heart flag > (http://www.erlang.org/doc/man/heart.html). This spawns a small external C > program connected through a port. From Erlang's point of view it's like any > other port program. The heart program pings the Erlang side every now and > then, and if it gets no reply within HEART_BEAT_TIMEOUT seconds, or if the > connection to Erlang breaks, it assumes the Beam process has gone bad and > kills it off with a SIGKILL, and then restarts Erlang using whatever > HEART_COMMAND is set to. So far so good. > > Normally, when Beam detects a critical situation (e.g., out of memory) and > decides to shut down, it will create an erl_crash.dump file (or whatever > ERL_CRASH_DUMP is set to). This information can greatly help figuring out > what went wrong. But if the system that crashed was large, the crash dump > file can take quite a long time to create. In order to make it possible to > restart the node (reusing the node name) while the old defunct system is > still writing the crash dump, Beam wants to drop its connection to the EPMD > service before it starts writing the dump, making it look like the old node > has disappeared. > > The code that does this is the function prepare_crash_dump() in > erts/emulator/sys/unix/sys.c. The problem from the perspective of the C code > is that the connection to EPMD is on some unknown file descriptor (just like > heart, this has been started as a port from Erlang code). The solution they > chose, and which has been part of the OTP system for years, is to close > _all_ file descriptors except 0-2. This certainly has the desired effect > that EPMD releases the node name for reuse. But it also, when the loop gets > to file descriptor 10 or thereabouts (probably depending on your system), > has the effect of breaking the connection to the heart program. > > In these multicore days, the effect is almost instantaneous. The heart > program immediately wakes up due to the broken pipe and sends SIGKILL to > Beam for good measure, to make sure it's really gone, and then it starts a > new Erlang node. Meanwhile, the old node is still busy closing file > descriptors. Sometimes it makes it as far as 12 before SIGKILL arrives. The > poor thing never has a chance to even open the crash dump file for writing. > And your operations people only see a weird restart without any further > clues. > > I don't have a good solution right now, except "don't use -heart". And it > might be that one wants to separate the automatic restarting of a crashed > node from the automatic killing of an unresponsive node anyway. Suggestions > are welcome. Hi Richard, I hit this problem a few years ago. Here's the thread starting from where I posted a temporary solution: http://erlang.org/pipermail/erlang-questions/2010-August/052970.html Unfortunately no patches came out of that conversation, but Ulf had an idea that might be worth exploring in a followup to the post linked above. --steve From carlsson.richard@REDACTED Sat Aug 25 22:03:02 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Sat, 25 Aug 2012 22:03:02 +0200 Subject: [erlang-questions] [erlang-bugs] heart prevents beam from creating crash dumps In-Reply-To: References: <503929D9.6000003@gmail.com> Message-ID: <50392F76.4020501@gmail.com> On 08/25/2012 09:48 PM, Steve Vinoski wrote: > On Sat, Aug 25, 2012 at 3:39 PM, Richard Carlsson > wrote: >> We have had a long-standing problems with not getting any Erlang crash dumps >> at all on our live servers. I finally figured out why it happens. I have >> already reported this to the OTP folks, but I thought I should send a >> summary to the mailing lists for documentation and to give people a >> heads-up. >> >> The problem occurs when you start Erlang with the -heart flag >> (http://www.erlang.org/doc/man/heart.html). This spawns a small external C >> program connected through a port. From Erlang's point of view it's like any >> other port program. The heart program pings the Erlang side every now and >> then, and if it gets no reply within HEART_BEAT_TIMEOUT seconds, or if the >> connection to Erlang breaks, it assumes the Beam process has gone bad and >> kills it off with a SIGKILL, and then restarts Erlang using whatever >> HEART_COMMAND is set to. So far so good. >> >> Normally, when Beam detects a critical situation (e.g., out of memory) and >> decides to shut down, it will create an erl_crash.dump file (or whatever >> ERL_CRASH_DUMP is set to). This information can greatly help figuring out >> what went wrong. But if the system that crashed was large, the crash dump >> file can take quite a long time to create. In order to make it possible to >> restart the node (reusing the node name) while the old defunct system is >> still writing the crash dump, Beam wants to drop its connection to the EPMD >> service before it starts writing the dump, making it look like the old node >> has disappeared. >> >> The code that does this is the function prepare_crash_dump() in >> erts/emulator/sys/unix/sys.c. The problem from the perspective of the C code >> is that the connection to EPMD is on some unknown file descriptor (just like >> heart, this has been started as a port from Erlang code). The solution they >> chose, and which has been part of the OTP system for years, is to close >> _all_ file descriptors except 0-2. This certainly has the desired effect >> that EPMD releases the node name for reuse. But it also, when the loop gets >> to file descriptor 10 or thereabouts (probably depending on your system), >> has the effect of breaking the connection to the heart program. >> >> In these multicore days, the effect is almost instantaneous. The heart >> program immediately wakes up due to the broken pipe and sends SIGKILL to >> Beam for good measure, to make sure it's really gone, and then it starts a >> new Erlang node. Meanwhile, the old node is still busy closing file >> descriptors. Sometimes it makes it as far as 12 before SIGKILL arrives. The >> poor thing never has a chance to even open the crash dump file for writing. >> And your operations people only see a weird restart without any further >> clues. >> >> I don't have a good solution right now, except "don't use -heart". And it >> might be that one wants to separate the automatic restarting of a crashed >> node from the automatic killing of an unresponsive node anyway. Suggestions >> are welcome. > > Hi Richard, I hit this problem a few years ago. Here's the thread > starting from where I posted a temporary solution: > > http://erlang.org/pipermail/erlang-questions/2010-August/052970.html Yes, I had seen that. (It was pretty much the only thing that Google came up with for this particular topic.) But the key point that was missing from that discussion was that it ironically enough is the act of preparing to write a crash dump that ends up killing the system before it can write the crash dump. It would be great if there was a way of simply figuring out the file descriptor numbers used for EPMD and/or heart from the C code. Then it would be easy to fix this. One possibility is to add a new BIF that stores the current EPMD port in a C variable. Then the loop that closes all ports could be replaced with a single close. /Richard From thomas.elsgaard@REDACTED Sat Aug 25 22:12:00 2012 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Sat, 25 Aug 2012 22:12:00 +0200 Subject: [erlang-questions] Best way of automatic Yaws start on FreeBSD 9.0? Message-ID: Hi list Any real life exemples of automatic start of yaws on FreeBSD 9? Maybe a working rc.d script? Currently i am via rc.d starting screen, which then starts yaws. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivier.girondel@REDACTED Sat Aug 25 22:21:16 2012 From: olivier.girondel@REDACTED (Olivier Girondel) Date: Sat, 25 Aug 2012 22:21:16 +0200 Subject: [erlang-questions] Exit status of C-port In-Reply-To: References: Message-ID: On Fri, Aug 24, 2012 at 10:08 PM, Jonas Falkevik wrote: > The exception you see might be that you received a exit signal from the port. > Try setting the process flag "trap_exit" to true, then you will exit signal as a message to process linked to the port. /me slaps his head.. Maybe I came back too early from my holidays in the South of France, sorry for the noise ;) -- Olivier From olivier.girondel@REDACTED Sat Aug 25 22:27:09 2012 From: olivier.girondel@REDACTED (Olivier Girondel) Date: Sat, 25 Aug 2012 22:27:09 +0200 Subject: [erlang-questions] [ANN] euuid 1.0.0 - an erlang UUID implementation In-Reply-To: References: <42f23e60d04addac0a1ce88ae3aa09ee.squirrel@secure.bunix.org> <1345886039481-4655143.post@n4.nabble.com> Message-ID: On Sat, Aug 25, 2012 at 8:41 PM, Hal Casteel wrote: > Thanks I will read yours as a reference. Historical reference, I meant. Mine is definitely outdated/not the best available :) -- Olivier From hal.casteel@REDACTED Sat Aug 25 23:00:54 2012 From: hal.casteel@REDACTED (Hal Casteel) Date: Sat, 25 Aug 2012 17:00:54 -0400 Subject: [erlang-questions] [ANN] euuid 1.0.0 - an erlang UUID implementation In-Reply-To: References: <42f23e60d04addac0a1ce88ae3aa09ee.squirrel@secure.bunix.org> <1345886039481-4655143.post@n4.nabble.com> Message-ID: Correct, thanks again. HAL Hal Casteel Email (work related): hal.casteel@REDACTED Cellular: 980-475-5158 Direct Line (work): 704-413-6906 Toll-Free (work): 888-568-5991 May our thoughts be kind and clear; May our words and communication be kind and clear; May our actions and intentions be for the greater good of all human beings. May all be well with mankind. May the leaders of the earth protect in every way by keeping to the right path. May there be goodness for those who know the earth to be sacred. May all the worlds be happy. May the force be with you! On Sat, Aug 25, 2012 at 4:27 PM, Olivier Girondel wrote: > On Sat, Aug 25, 2012 at 8:41 PM, Hal Casteel wrote: >> Thanks I will read yours as a reference. > > Historical reference, I meant. Mine is definitely outdated/not the > best available :) > > -- > Olivier From vinoski@REDACTED Sun Aug 26 00:17:10 2012 From: vinoski@REDACTED (Steve Vinoski) Date: Sat, 25 Aug 2012 18:17:10 -0400 Subject: [erlang-questions] Best way of automatic Yaws start on FreeBSD 9.0? In-Reply-To: References: Message-ID: On Sat, Aug 25, 2012 at 4:12 PM, Thomas Elsgaard wrote: > Hi list > > Any real life exemples of automatic start of yaws on FreeBSD 9? Maybe a > working rc.d script? > > Currently i am via rc.d starting screen, which then starts yaws. The yaws sources include the following under scripts/freebsd: https://github.com/klacke/yaws/blob/master/scripts/freebsd/yaws.sh If that doesn't help or work, patches gladly accepted. :) --steve From thomas.elsgaard@REDACTED Sun Aug 26 01:37:49 2012 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Sun, 26 Aug 2012 01:37:49 +0200 Subject: [erlang-questions] Best way of automatic Yaws start on FreeBSD 9.0? In-Reply-To: References: Message-ID: Hi Steve It seems like the format of the rc.d scripts has changed, i am not sure in which FreeBSD version, but i am using latest stable (9.0) I have just made below (and attached), which seems to work fine Best regards Thomas #!/bin/sh # # Start up for the Yaws daemon. # # PROVIDE: yaws # BEFORE: DAEMON # KEYWORD: shutdown . /etc/rc.subr name="yaws" yaws_id="default" # By default we run with the default id start_precmd="${name}_prestart" stop_cmd="yaws_stop" status_cmd="yaws_status" command="/usr/local/bin/${name}" rcvar="yaws_enable" command_args="" required_files="/etc/${name}/${name}.conf" yaws_prestart() { rc_flags="--id ${yaws_id} --daemon --heart --conf /etc/${name}/${name}.conf ${rc_flags}" } yaws_stop() { /usr/local/bin/yaws --id ${yaws_id} --stop } yaws_status() { /usr/local/bin/yaws --id ${yaws_id} --status } load_rc_config $name run_rc_command "$1" On Sun, Aug 26, 2012 at 12:17 AM, Steve Vinoski wrote: > On Sat, Aug 25, 2012 at 4:12 PM, Thomas Elsgaard > wrote: >> Hi list >> >> Any real life exemples of automatic start of yaws on FreeBSD 9? Maybe a >> working rc.d script? >> >> Currently i am via rc.d starting screen, which then starts yaws. > > The yaws sources include the following under scripts/freebsd: > > https://github.com/klacke/yaws/blob/master/scripts/freebsd/yaws.sh > > If that doesn't help or work, patches gladly accepted. :) > > --steve -------------- next part -------------- A non-text attachment was scrubbed... Name: yaws Type: application/octet-stream Size: 657 bytes Desc: not available URL: From vinoski@REDACTED Sun Aug 26 02:08:05 2012 From: vinoski@REDACTED (Steve Vinoski) Date: Sat, 25 Aug 2012 20:08:05 -0400 Subject: [erlang-questions] Best way of automatic Yaws start on FreeBSD 9.0? In-Reply-To: References: Message-ID: Thanks, I'll add this to the yaws repo on github. --steve On Sat, Aug 25, 2012 at 7:37 PM, Thomas Elsgaard wrote: > Hi Steve > > It seems like the format of the rc.d scripts has changed, i am not > sure in which FreeBSD version, but i am using latest stable (9.0) > > I have just made below (and attached), which seems to work fine > > Best regards > > Thomas > > > #!/bin/sh > # > # Start up for the Yaws daemon. > # > > # PROVIDE: yaws > # BEFORE: DAEMON > # KEYWORD: shutdown > > . /etc/rc.subr > > name="yaws" > yaws_id="default" # By default we run with the default id > start_precmd="${name}_prestart" > stop_cmd="yaws_stop" > status_cmd="yaws_status" > command="/usr/local/bin/${name}" > rcvar="yaws_enable" > command_args="" > required_files="/etc/${name}/${name}.conf" > > yaws_prestart() { > rc_flags="--id ${yaws_id} --daemon --heart --conf > /etc/${name}/${name}.conf ${rc_flags}" > } > > yaws_stop() { > /usr/local/bin/yaws --id ${yaws_id} --stop > } > > yaws_status() { > /usr/local/bin/yaws --id ${yaws_id} --status > } > > load_rc_config $name > run_rc_command "$1" > > > > > > On Sun, Aug 26, 2012 at 12:17 AM, Steve Vinoski wrote: >> On Sat, Aug 25, 2012 at 4:12 PM, Thomas Elsgaard >> wrote: >>> Hi list >>> >>> Any real life exemples of automatic start of yaws on FreeBSD 9? Maybe a >>> working rc.d script? >>> >>> Currently i am via rc.d starting screen, which then starts yaws. >> >> The yaws sources include the following under scripts/freebsd: >> >> https://github.com/klacke/yaws/blob/master/scripts/freebsd/yaws.sh >> >> If that doesn't help or work, patches gladly accepted. :) >> >> --steve From kenji.rikitake@REDACTED Sun Aug 26 06:54:45 2012 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Sun, 26 Aug 2012 13:54:45 +0900 Subject: [erlang-questions] Best way of automatic Yaws start on FreeBSD 9.0? In-Reply-To: References: Message-ID: <20120826045445.GA25869@k2r.org> The Yaws 1.94 Port on the FreeBSD Ports tree works fine. http://www.freshports.org/www/yaws/ And I'm using this: https://github.com/jj1bdx/yaws-freebsd-port FYI Kenji Rikitake ++> Steve Vinoski [2012-08-25 20:08:05 -0400]: > Date: Sat, 25 Aug 2012 20:08:05 -0400 > From: Steve Vinoski > To: Thomas Elsgaard > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Best way of automatic Yaws start on FreeBSD > 9.0? > > Thanks, I'll add this to the yaws repo on github. > > --steve > > On Sat, Aug 25, 2012 at 7:37 PM, Thomas Elsgaard > wrote: > > Hi Steve > > > > It seems like the format of the rc.d scripts has changed, i am not > > sure in which FreeBSD version, but i am using latest stable (9.0) > > > > I have just made below (and attached), which seems to work fine > > > > Best regards > > > > Thomas > > > > > > #!/bin/sh > > # > > # Start up for the Yaws daemon. > > # > > > > # PROVIDE: yaws > > # BEFORE: DAEMON > > # KEYWORD: shutdown > > > > . /etc/rc.subr > > > > name="yaws" > > yaws_id="default" # By default we run with the default id > > start_precmd="${name}_prestart" > > stop_cmd="yaws_stop" > > status_cmd="yaws_status" > > command="/usr/local/bin/${name}" > > rcvar="yaws_enable" > > command_args="" > > required_files="/etc/${name}/${name}.conf" > > > > yaws_prestart() { > > rc_flags="--id ${yaws_id} --daemon --heart --conf > > /etc/${name}/${name}.conf ${rc_flags}" > > } > > > > yaws_stop() { > > /usr/local/bin/yaws --id ${yaws_id} --stop > > } > > > > yaws_status() { > > /usr/local/bin/yaws --id ${yaws_id} --status > > } > > > > load_rc_config $name > > run_rc_command "$1" > > > > > > > > > > > > On Sun, Aug 26, 2012 at 12:17 AM, Steve Vinoski wrote: > >> On Sat, Aug 25, 2012 at 4:12 PM, Thomas Elsgaard > >> wrote: > >>> Hi list > >>> > >>> Any real life exemples of automatic start of yaws on FreeBSD 9? Maybe a > >>> working rc.d script? > >>> > >>> Currently i am via rc.d starting screen, which then starts yaws. > >> > >> The yaws sources include the following under scripts/freebsd: > >> > >> https://github.com/klacke/yaws/blob/master/scripts/freebsd/yaws.sh > >> > >> If that doesn't help or work, patches gladly accepted. :) > >> > >> --steve > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bourinov@REDACTED Sun Aug 26 08:33:35 2012 From: bourinov@REDACTED (Max Bourinov) Date: Sun, 26 Aug 2012 10:33:35 +0400 Subject: [erlang-questions] count events within last XX minutes In-Reply-To: References: Message-ID: Hi guys, Thank you guys for all your replies. I think I will go with my own implementation. The only thing I cannot understand, why you suggest using ETS for this? Why keeping data in the state is not ok? I think on heavy load the state approach will perform better than ETS. Moreover, I think ETS is overkill for this task. p.s. In my case XX minutes wont exceed 120. Best regards, Max On Sat, Aug 25, 2012 at 12:05 AM, Mike Oxford wrote: > If you care about, as you say, "certain minute" you can use Ulf's gproc. > If you want "last minute" then the ETS solution below will work well, with > the modification on the Counter to use a ms/us timer like erlang:now(). > > -mox > > > On Fri, Aug 24, 2012 at 8:14 AM, Anders Nygren wrote: > >> Use an ETS table with {Counter,{YYYY,MM,DD,HH,MM}} as key, and >> ets:update_counter/2,3. >> update_counter returns the new counter value, so if it is 1, (or the >> increment used) You know that a new minute has been entered so You can >> delete the oldest. >> >> /Anders >> >> On Fri, Aug 24, 2012 at 9:16 AM, Max Bourinov wrote: >> > Dear Erlangers, >> > >> > Does anybody know best memory efficient way to count events within last >> XX >> > minutes? >> > >> > So far I have the following idea: For each counter I have a process. The >> > counter process has a list of XX items. Each item represents a certain >> > minute, so I always know where is my current counter. There is also >> must be >> > a mechanism to remove last item from the list and add a new one. Summing >> > values from all items is a number of events within last XX minutes. >> This is >> > it. >> > >> > Maybe there is a ready lib that does the same in a better way? >> > >> > Any suggestions are welcome! >> > >> > Best regards, >> > Max >> > >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Aug 26 09:22:19 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 26 Aug 2012 11:22:19 +0400 Subject: [erlang-questions] count events within last XX minutes In-Reply-To: References: Message-ID: On Sun, Aug 26, 2012 at 10:33 AM, Max Bourinov wrote: > Hi guys, > > Thank you guys for all your replies. > > I think I will go with my own implementation. The only thing I cannot > understand, why you suggest using ETS for this? because ets have update_counter API > Why keeping data in the state is not ok? because you will have giant state with all problems > I think on heavy load the state approach will perform > better than ETS. Moreover, I think ETS is overkill for this task. "overkill" here is an emotion without any exact results. Keeping this info in state will lead to copy of growing amount of memory. > p.s. In my case XX minutes wont exceed 120. > > Best regards, > Max > > > > > On Sat, Aug 25, 2012 at 12:05 AM, Mike Oxford wrote: >> >> If you care about, as you say, "certain minute" you can use Ulf's gproc. >> If you want "last minute" then the ETS solution below will work well, with >> the modification on the Counter to use a ms/us timer like erlang:now(). >> >> -mox >> >> >> On Fri, Aug 24, 2012 at 8:14 AM, Anders Nygren >> wrote: >>> >>> Use an ETS table with {Counter,{YYYY,MM,DD,HH,MM}} as key, and >>> ets:update_counter/2,3. >>> update_counter returns the new counter value, so if it is 1, (or the >>> increment used) You know that a new minute has been entered so You can >>> delete the oldest. >>> >>> /Anders >>> >>> On Fri, Aug 24, 2012 at 9:16 AM, Max Bourinov wrote: >>> > Dear Erlangers, >>> > >>> > Does anybody know best memory efficient way to count events within last >>> > XX >>> > minutes? >>> > >>> > So far I have the following idea: For each counter I have a process. >>> > The >>> > counter process has a list of XX items. Each item represents a certain >>> > minute, so I always know where is my current counter. There is also >>> > must be >>> > a mechanism to remove last item from the list and add a new one. >>> > Summing >>> > values from all items is a number of events within last XX minutes. >>> > This is >>> > it. >>> > >>> > Maybe there is a ready lib that does the same in a better way? >>> > >>> > Any suggestions are welcome! >>> > >>> > Best regards, >>> > Max >>> > >>> > >>> > >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From bourinov@REDACTED Sun Aug 26 09:54:48 2012 From: bourinov@REDACTED (Max Bourinov) Date: Sun, 26 Aug 2012 11:54:48 +0400 Subject: [erlang-questions] count events within last XX minutes In-Reply-To: References: Message-ID: Hi Max, Thank you for you comments and explanation. Did you saw that I need only 120 minutes? So it will be a list with 121 element at most. In this case memory won't grow at all and size of the state will be the same. I agree about coping but I don't understand why state will grow. Could you please explain it? Best regards, Max On Sun, Aug 26, 2012 at 11:22 AM, Max Lapshin wrote: > On Sun, Aug 26, 2012 at 10:33 AM, Max Bourinov wrote: > > Hi guys, > > > > Thank you guys for all your replies. > > > > I think I will go with my own implementation. The only thing I cannot > > understand, why you suggest using ETS for this? > > because ets have update_counter API > > > > Why keeping data in the state is not ok? > > because you will have giant state with all problems > > > I think on heavy load the state approach will perform > > better than ETS. Moreover, I think ETS is overkill for this task. > > "overkill" here is an emotion without any exact results. > Keeping this info in state will lead to copy of growing amount of memory. > > > > p.s. In my case XX minutes wont exceed 120. > > > > Best regards, > > Max > > > > > > > > > > On Sat, Aug 25, 2012 at 12:05 AM, Mike Oxford wrote: > >> > >> If you care about, as you say, "certain minute" you can use Ulf's gproc. > >> If you want "last minute" then the ETS solution below will work well, > with > >> the modification on the Counter to use a ms/us timer like erlang:now(). > >> > >> -mox > >> > >> > >> On Fri, Aug 24, 2012 at 8:14 AM, Anders Nygren > > >> wrote: > >>> > >>> Use an ETS table with {Counter,{YYYY,MM,DD,HH,MM}} as key, and > >>> ets:update_counter/2,3. > >>> update_counter returns the new counter value, so if it is 1, (or the > >>> increment used) You know that a new minute has been entered so You can > >>> delete the oldest. > >>> > >>> /Anders > >>> > >>> On Fri, Aug 24, 2012 at 9:16 AM, Max Bourinov > wrote: > >>> > Dear Erlangers, > >>> > > >>> > Does anybody know best memory efficient way to count events within > last > >>> > XX > >>> > minutes? > >>> > > >>> > So far I have the following idea: For each counter I have a process. > >>> > The > >>> > counter process has a list of XX items. Each item represents a > certain > >>> > minute, so I always know where is my current counter. There is also > >>> > must be > >>> > a mechanism to remove last item from the list and add a new one. > >>> > Summing > >>> > values from all items is a number of events within last XX minutes. > >>> > This is > >>> > it. > >>> > > >>> > Maybe there is a ready lib that does the same in a better way? > >>> > > >>> > Any suggestions are welcome! > >>> > > >>> > Best regards, > >>> > Max > >>> > > >>> > > >>> > > >>> > _______________________________________________ > >>> > erlang-questions mailing list > >>> > erlang-questions@REDACTED > >>> > http://erlang.org/mailman/listinfo/erlang-questions > >>> > > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Aug 26 09:56:45 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 26 Aug 2012 11:56:45 +0400 Subject: [erlang-questions] count events within last XX minutes In-Reply-To: References: Message-ID: with 120 minute limit, it will not grow and you can use in-state memory storage. But ets is not an overkill. It is just another solution. On Sun, Aug 26, 2012 at 11:54 AM, Max Bourinov wrote: > Hi Max, > > Thank you for you comments and explanation. > > Did you saw that I need only 120 minutes? So it will be a list with 121 > element at most. In this case memory won't grow at all and size of the state > will be the same. > > I agree about coping but I don't understand why state will grow. Could you > please explain it? > > Best regards, > Max > > > > > On Sun, Aug 26, 2012 at 11:22 AM, Max Lapshin wrote: >> >> On Sun, Aug 26, 2012 at 10:33 AM, Max Bourinov wrote: >> > Hi guys, >> > >> > Thank you guys for all your replies. >> > >> > I think I will go with my own implementation. The only thing I cannot >> > understand, why you suggest using ETS for this? >> >> because ets have update_counter API >> >> >> > Why keeping data in the state is not ok? >> >> because you will have giant state with all problems >> >> > I think on heavy load the state approach will perform >> > better than ETS. Moreover, I think ETS is overkill for this task. >> >> "overkill" here is an emotion without any exact results. >> Keeping this info in state will lead to copy of growing amount of memory. >> >> >> > p.s. In my case XX minutes wont exceed 120. >> > >> > Best regards, >> > Max >> > >> > >> > >> > >> > On Sat, Aug 25, 2012 at 12:05 AM, Mike Oxford wrote: >> >> >> >> If you care about, as you say, "certain minute" you can use Ulf's >> >> gproc. >> >> If you want "last minute" then the ETS solution below will work well, >> >> with >> >> the modification on the Counter to use a ms/us timer like erlang:now(). >> >> >> >> -mox >> >> >> >> >> >> On Fri, Aug 24, 2012 at 8:14 AM, Anders Nygren >> >> >> >> wrote: >> >>> >> >>> Use an ETS table with {Counter,{YYYY,MM,DD,HH,MM}} as key, and >> >>> ets:update_counter/2,3. >> >>> update_counter returns the new counter value, so if it is 1, (or the >> >>> increment used) You know that a new minute has been entered so You can >> >>> delete the oldest. >> >>> >> >>> /Anders >> >>> >> >>> On Fri, Aug 24, 2012 at 9:16 AM, Max Bourinov >> >>> wrote: >> >>> > Dear Erlangers, >> >>> > >> >>> > Does anybody know best memory efficient way to count events within >> >>> > last >> >>> > XX >> >>> > minutes? >> >>> > >> >>> > So far I have the following idea: For each counter I have a process. >> >>> > The >> >>> > counter process has a list of XX items. Each item represents a >> >>> > certain >> >>> > minute, so I always know where is my current counter. There is also >> >>> > must be >> >>> > a mechanism to remove last item from the list and add a new one. >> >>> > Summing >> >>> > values from all items is a number of events within last XX minutes. >> >>> > This is >> >>> > it. >> >>> > >> >>> > Maybe there is a ready lib that does the same in a better way? >> >>> > >> >>> > Any suggestions are welcome! >> >>> > >> >>> > Best regards, >> >>> > Max >> >>> > >> >>> > >> >>> > >> >>> > _______________________________________________ >> >>> > erlang-questions mailing list >> >>> > erlang-questions@REDACTED >> >>> > http://erlang.org/mailman/listinfo/erlang-questions >> >>> > >> >>> _______________________________________________ >> >>> erlang-questions mailing list >> >>> erlang-questions@REDACTED >> >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > > > From mjtruog@REDACTED Sun Aug 26 10:02:09 2012 From: mjtruog@REDACTED (Michael Truog) Date: Sun, 26 Aug 2012 01:02:09 -0700 Subject: [erlang-questions] count events within last XX minutes In-Reply-To: References: Message-ID: <5039D801.5050505@gmail.com> There often is a habit with quick projects to throw data in ets, since it is easy to access the data as global data. This helps people coming from an imperative programming background. I don't see a good reason in the email thread that shows that ets is the best solution, simply because the amount of data is not clear. ets can be used to limit the memory consumption, but so far, memory consumption was not mentioned as a concern. Having an update_counter function doesn't sound too convincing because programmers are generally capable of a fetch-increment-store routine. So, the moral of the story is to test and experiment. You could try internal state with a dict as compared to ets, and see which make sense based on your requirements. On 08/25/2012 11:33 PM, Max Bourinov wrote: > Hi guys, > > Thank you guys for all your replies. > > I think I will go with my own implementation. The only thing I cannot understand, why you suggest using ETS for this? Why keeping data in the state is not ok? I think on heavy load the state approach will perform better than ETS. Moreover, I think ETS is overkill for this task. > > p.s. In my case XX minutes wont exceed 120. > > Best regards, > Max > > > > > On Sat, Aug 25, 2012 at 12:05 AM, Mike Oxford > wrote: > > If you care about, as you say, "certain minute" you can use Ulf's gproc. If you want "last minute" then the ETS solution below will work well, with the modification on the Counter to use a ms/us timer like erlang:now(). > > -mox > > > On Fri, Aug 24, 2012 at 8:14 AM, Anders Nygren > wrote: > > Use an ETS table with {Counter,{YYYY,MM,DD,HH,MM}} as key, and > ets:update_counter/2,3. > update_counter returns the new counter value, so if it is 1, (or the > increment used) You know that a new minute has been entered so You can > delete the oldest. > > /Anders > > On Fri, Aug 24, 2012 at 9:16 AM, Max Bourinov > wrote: > > Dear Erlangers, > > > > Does anybody know best memory efficient way to count events within last XX > > minutes? > > > > So far I have the following idea: For each counter I have a process. The > > counter process has a list of XX items. Each item represents a certain > > minute, so I always know where is my current counter. There is also must be > > a mechanism to remove last item from the list and add a new one. Summing > > values from all items is a number of events within last XX minutes. This is > > it. > > > > Maybe there is a ready lib that does the same in a better way? > > > > Any suggestions are welcome! > > > > Best regards, > > Max > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Aug 26 10:04:17 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 26 Aug 2012 12:04:17 +0400 Subject: [erlang-questions] count events within last XX minutes In-Reply-To: <5039D801.5050505@gmail.com> References: <5039D801.5050505@gmail.com> Message-ID: On Sun, Aug 26, 2012 at 12:02 PM, Michael Truog wrote: > There often is a habit with quick projects to throw data in ets, since it is > easy to access the data as global data. This helps people coming from an > imperative programming background. I don't see a good reason in the email > thread that shows that ets is the best solution, But there is a good reason. It is performance. For example, in erlyvideo all major statistics data are collected not via gen_server:cal: you cannot ask process to tell its statistics, because it is very easy to DOS your server with such replies. If you put stats into public ets, than collector will not be overloaded with requests. From bourinov@REDACTED Sun Aug 26 10:21:40 2012 From: bourinov@REDACTED (Max Bourinov) Date: Sun, 26 Aug 2012 12:21:40 +0400 Subject: [erlang-questions] count events within last XX minutes In-Reply-To: References: <5039D801.5050505@gmail.com> Message-ID: >If you put stats into public ets, than collector will not be >overloaded with requests. This is interesting. I also have thoughts about avoid DOS. But I don't understand how ETS will help here. Here what I have. I have only one process that process clients requests. It really doesn't make sense to have more processes because the process does nothing but returns some pre-calculated value. If I store data in ETS my process will have to do additional work - extracting data from ETS. Or maybe I am missing something? Please give the right direction here. Or maybe in my case it doesn't matter because my case is trivial? Best regards, Max On Sun, Aug 26, 2012 at 12:04 PM, Max Lapshin wrote: > On Sun, Aug 26, 2012 at 12:02 PM, Michael Truog wrote: > > There often is a habit with quick projects to throw data in ets, since > it is > > easy to access the data as global data. This helps people coming from an > > imperative programming background. I don't see a good reason in the > email > > thread that shows that ets is the best solution, > > > But there is a good reason. It is performance. For example, in > erlyvideo all major statistics data are > collected not via gen_server:cal: you cannot ask process to tell its > statistics, because > it is very easy to DOS your server with such replies. > > If you put stats into public ets, than collector will not be > overloaded with requests. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Sun Aug 26 10:30:51 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Sun, 26 Aug 2012 10:30:51 +0200 Subject: [erlang-questions] count events within last XX minutes In-Reply-To: References: <5039D801.5050505@gmail.com> Message-ID: Lately when working on project with up to a million concurrent clients connected I had to replace few instances of central servers with a public ets'es to avoid bottlenecks under heavy concurrent access. So choice of data storage may depend on what are access patterns of your data - i.e. how concurrent reads and writes are. On Aug 26, 2012 10:22 AM, "Max Bourinov" wrote: > > >If you put stats into public ets, than collector will not be > >overloaded with requests. > > This is interesting. I also have thoughts about avoid DOS. But I don't understand how ETS will help here. > > Here what I have. I have only one process that process clients requests. It really doesn't make sense to have more processes because the process does nothing but returns some pre-calculated value. If I store data in ETS my process will have to do additional work - extracting data from ETS. > > Or maybe I am missing something? Please give the right direction here. Or maybe in my case it doesn't matter because my case is trivial? > > Best regards, > Max > > > > On Sun, Aug 26, 2012 at 12:04 PM, Max Lapshin wrote: >> >> On Sun, Aug 26, 2012 at 12:02 PM, Michael Truog wrote: >> > There often is a habit with quick projects to throw data in ets, since it is >> > easy to access the data as global data. This helps people coming from an >> > imperative programming background. I don't see a good reason in the email >> > thread that shows that ets is the best solution, >> >> >> But there is a good reason. It is performance. For example, in >> erlyvideo all major statistics data are >> collected not via gen_server:cal: you cannot ask process to tell its >> statistics, because >> it is very easy to DOS your server with such replies. >> >> If you put stats into public ets, than collector will not be >> overloaded with requests. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Sun Aug 26 10:41:43 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Sun, 26 Aug 2012 10:41:43 +0200 Subject: [erlang-questions] count events within last XX minutes In-Reply-To: References: <5039D801.5050505@gmail.com> Message-ID: On Sunday, August 26, 2012, Max Bourinov wrote: > > Lately when working on project with up to a million concurrent clients >> connected I had to replace few instances of central servers with a public >> ets'es to avoid bottlenecks under heavy concurrent access. So choice of >> data storage may depend on what are access patterns of your data - i.e. how >> concurrent reads and writes are. >> > That is a good point. > > I have idea how to utilize ETS in my case too: one process will write to > ETS, and all other will read from ETS. > > The question now: will it benefit me or not? > Probably yes, if read_concurrency is enabled and you have enough concurrent processes reading data. The best way to answer for sure is to implement both and measure under some high load situation. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Sun Aug 26 11:02:05 2012 From: bourinov@REDACTED (Max Bourinov) Date: Sun, 26 Aug 2012 13:02:05 +0400 Subject: [erlang-questions] count events within last XX minutes In-Reply-To: References: <5039D801.5050505@gmail.com> Message-ID: Thank you! +1 for testing :-) Best regards, Max On Sun, Aug 26, 2012 at 12:41 PM, Gleb Peregud wrote: > On Sunday, August 26, 2012, Max Bourinov wrote: >> >> Lately when working on project with up to a million concurrent clients >>> connected I had to replace few instances of central servers with a public >>> ets'es to avoid bottlenecks under heavy concurrent access. So choice of >>> data storage may depend on what are access patterns of your data - i.e. how >>> concurrent reads and writes are. >>> >> That is a good point. >> >> I have idea how to utilize ETS in my case too: one process will write to >> ETS, and all other will read from ETS. >> >> The question now: will it benefit me or not? >> > Probably yes, if read_concurrency is enabled and you have enough > concurrent processes reading data. The best way to answer for sure is to > implement both and measure under some high load situation. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sun Aug 26 11:45:21 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Sun, 26 Aug 2012 11:45:21 +0200 Subject: [erlang-questions] count events within last XX minutes In-Reply-To: References: <5039D801.5050505@gmail.com> Message-ID: <5039F031.2030007@ninenines.eu> On 08/26/2012 10:41 AM, Gleb Peregud wrote: > On Sunday, August 26, 2012, Max Bourinov wrote: > > Lately when working on project with up to a million concurrent > clients connected I had to replace few instances of central > servers with a public ets'es to avoid bottlenecks under heavy > concurrent access. So choice of data storage may depend on what > are access patterns of your data - i.e. how concurrent reads and > writes are. > > That is a good point. > > I have idea how to utilize ETS in my case too: one process will > write to ETS, and all other will read from ETS. > > The question now: will it benefit me or not? > > Probably yes, if read_concurrency is enabled and you have enough > concurrent processes reading data. The best way to answer for sure is to > implement both and measure under some high load situation. If he's using update_counter then he only needs write_concurrency. update_counter is a write operation that also returns the new value; if you update_counter with an increment of 0, then you just happened to read the value with a write context. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From kilgoretrout62@REDACTED Mon Aug 27 14:15:38 2012 From: kilgoretrout62@REDACTED (kilgore trout) Date: Mon, 27 Aug 2012 13:15:38 +0100 Subject: [erlang-questions] Graphing tools Message-ID: Hello Langers! I'm looking for a graphing tool, anyone know of any useful libraries? I have piles of performance data in .csv format and a parser to read it into records. I have been trying to find patterns and draw some meaningful conclusions but it would be so much easier if I could represent it graphically. A quick google search shows up erlycairo, which I'm about to experiment with, but I see the latest downloadable is from 2009, anyone have any experience with this? Please save me from having to paste everything into (shudder) MS Excel! ;-) Thanks in advance. //KT. -------------- next part -------------- An HTML attachment was scrubbed... URL: From armstrong.whit@REDACTED Mon Aug 27 14:23:32 2012 From: armstrong.whit@REDACTED (Whit Armstrong) Date: Mon, 27 Aug 2012 08:23:32 -0400 Subject: [erlang-questions] Graphing tools In-Reply-To: References: Message-ID: http://www.r-project.org/ On Mon, Aug 27, 2012 at 8:15 AM, kilgore trout wrote: > Hello Langers! > I'm looking for a graphing tool, anyone know of any useful libraries? > I have piles of performance data in .csv format and a parser to read it into > records. I have been trying to find patterns and draw some meaningful > conclusions but it would be so much easier if I could represent it > graphically. > A quick google search shows up erlycairo, which I'm about to experiment > with, but I see the latest downloadable is from 2009, anyone have any > experience with this? > Please save me from having to paste everything into (shudder) MS Excel! ;-) > Thanks in advance. > //KT. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From dmkolesnikov@REDACTED Mon Aug 27 14:45:57 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Mon, 27 Aug 2012 15:45:57 +0300 Subject: [erlang-questions] Graphing tools In-Reply-To: References: Message-ID: <3505154021464762445@unknownmsgid> Hello, For Web application Google Visualization API is your friend. Personally, I have been integrating rrdtools with erlang for same use-cases (perf data visualization) but it becomes unscalable solution due to rrd overhead, I've trashed it. I do have a simple SVG toolkit for SVG rendering on Erlang side http://github.com/fogfish/svg.git It might be useful if you are ready to invest into visualization sugar... Best Regards, Dmitry >-|-|-*> On 27.8.2012, at 15.15, kilgore trout wrote: > Hello Langers! > I'm looking for a graphing tool, anyone know of any useful libraries? > I have piles of performance data in .csv format and a parser to read it into records. I have been trying to find patterns and draw some meaningful conclusions but it would be so much easier if I could represent it graphically. > A quick google search shows up erlycairo, which I'm about to experiment with, but I see the latest downloadable is from 2009, anyone have any experience with this? > Please save me from having to paste everything into (shudder) MS Excel! ;-) > Thanks in advance. > //KT. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From Sergey_Zhemzhitsky@REDACTED Mon Aug 27 17:12:30 2012 From: Sergey_Zhemzhitsky@REDACTED (Zhemzhitsky Sergey) Date: Mon, 27 Aug 2012 15:12:30 +0000 Subject: [erlang-questions] Logger to log terms, that can contain unicode chars Message-ID: <06139A918ACCA041BF46A0F36940C7FA4FE199CF@exch-mbx2.msk.trd.ru> Hi erlangers, Could you suggest any logger that can accept erlang terms which contains strings with Unicode characters? Or could you suggest how the lager should be configured properly. I was not able to get correct string presentation with lager doing like so lager:info(?Info: ~p?, [{tuple, ?string with umlauts u?}]) Best Regards, Sergey _______________________________________________________ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp -------------- next part -------------- An HTML attachment was scrubbed... URL: From jared.kofron@REDACTED Mon Aug 27 17:13:54 2012 From: jared.kofron@REDACTED (Jared Kofron) Date: Mon, 27 Aug 2012 08:13:54 -0700 Subject: [erlang-questions] Graphing tools In-Reply-To: <3505154021464762445@unknownmsgid> References: <3505154021464762445@unknownmsgid> Message-ID: Gnuplot is great for simple plotting as well. JK On Aug 27, 2012 5:46 AM, "dmitry kolesnikov" wrote: > Hello, > > For Web application Google Visualization API is your friend. > > Personally, I have been integrating rrdtools with erlang for same > use-cases (perf data visualization) but it becomes unscalable solution > due to rrd overhead, I've trashed it. > > I do have a simple SVG toolkit for SVG rendering on Erlang side > http://github.com/fogfish/svg.git > It might be useful if you are ready to invest into visualization sugar... > > Best Regards, > Dmitry >-|-|-*> > > > On 27.8.2012, at 15.15, kilgore trout wrote: > > > Hello Langers! > > I'm looking for a graphing tool, anyone know of any useful libraries? > > I have piles of performance data in .csv format and a parser to read it > into records. I have been trying to find patterns and draw some meaningful > conclusions but it would be so much easier if I could represent it > graphically. > > A quick google search shows up erlycairo, which I'm about to experiment > with, but I see the latest downloadable is from 2009, anyone have any > experience with this? > > Please save me from having to paste everything into (shudder) MS Excel! > ;-) > > Thanks in advance. > > //KT. > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew@REDACTED Mon Aug 27 17:40:59 2012 From: andrew@REDACTED (Andrew Thompson) Date: Mon, 27 Aug 2012 11:40:59 -0400 Subject: [erlang-questions] Logger to log terms, that can contain unicode chars In-Reply-To: <06139A918ACCA041BF46A0F36940C7FA4FE199CF@exch-mbx2.msk.trd.ru> References: <06139A918ACCA041BF46A0F36940C7FA4FE199CF@exch-mbx2.msk.trd.ru> Message-ID: <20120827154059.GL5837@hijacked.us> On Mon, Aug 27, 2012 at 03:12:30PM +0000, Zhemzhitsky Sergey wrote: > Hi erlangers, > > Could you suggest any logger that can accept erlang terms which contains strings with Unicode characters? > Or could you suggest how the lager should be configured properly. > > I was not able to get correct string presentation with lager doing like so > lager:info(?Info: ~p?, [{tuple, ?string with umlauts u?}]) Looks like a bug to do with printing unicode with the ~p formatter, things seem to work ok if you use ~s or ~ts. Could you file a bug so I can look into it more when I have time? Andrew From essen@REDACTED Mon Aug 27 22:34:38 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 27 Aug 2012 22:34:38 +0200 Subject: [erlang-questions] What do you like the most about Erlang/OTP? Message-ID: <503BD9DE.2080206@ninenines.eu> Hello, I am wondering what you guys like the most about Erlang/OTP, especially newcomers, maybe it changed your life, allowed you to climb the Everest (or at least sleep at night). I'll be synthesizing that into a few key points that I am hopeful will be reusable by anyone wanting to sell Erlang to their friends/work/clients. So help me by telling us what's so good about it! Thanks. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From desired.mta@REDACTED Mon Aug 27 22:53:07 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Mon, 27 Aug 2012 22:53:07 +0200 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503BD9DE.2080206@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> Message-ID: On Mon, Aug 27, 2012 at 10:34 PM, Lo?c Hoguin wrote: > Hello, > > I am wondering what you guys like the most about Erlang/OTP, especially > newcomers, maybe it changed your life, allowed you to climb the Everest (or > at least sleep at night). > Hi, killer feature for me is possibilities to interact with running system. You can send arbitrary messages to arbitraty processes, call arbitraty functions, restart anything by hand. This gives utmost flexibility to inspect and fix a running system. $ erl -setcookie company_name -sname d > net_kernel:connect_node(db@REDACTED). true > rpc:call(db@REDACTED, yodadb, how_are_you, []). {still_alive, [some, stuff], but_soon_will_die, [why]}. ... -- Motiejus Jak?tys From jayson.barley@REDACTED Mon Aug 27 22:56:37 2012 From: jayson.barley@REDACTED (Jayson Barley) Date: Mon, 27 Aug 2012 13:56:37 -0700 Subject: [erlang-questions] Why does Erlang have control structures? Message-ID: I am not sure I understand why we have them. For instance I can take the following code is_greater_than(X, Y) -> if X>Y -> true; true -> % works as an 'else' branch false end. And make it is_true(true) -> true; is_true(false) -> false. is_greater_than(X, Y) -> is_true(X>Y). I can do the same thing with case statements is_valid_signal(Signal) -> case Signal of {signal, _What, _From, _To} -> true; {signal, _What, _To} -> true; _Else -> false end. Becomes switch_signal({signal, _What, _From, _To}) -> true; switch_signal({signal, _What, _To}) -> true; switch_signal(_Else) -> false. is_valid_signal(Signal) -> switch_signal(Signal). I know that the control structures are a little bit faster, not much, but I find that the function form is more readable. -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Mon Aug 27 22:57:43 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Mon, 27 Aug 2012 22:57:43 +0200 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503BD9DE.2080206@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> Message-ID: On Mon, Aug 27, 2012 at 10:34 PM, Lo?c Hoguin wrote: > Hello, > > I am wondering what you guys like the most about Erlang/OTP, especially > newcomers, maybe it changed your life, allowed you to climb the Everest (or > at least sleep at night). > Hi, killer feature for me is possibilities to interact with running system. You can send arbitrary messages to arbitraty processes, call arbitraty functions, restart anything by hand. This gives utmost flexibility to inspect and fix a running system. $ erl -setcookie company_name -sname d > net_kernel:connect_node(db@REDACTED). true > rpc:call(db@REDACTED, yodadb, how_are_you, []). {still_alive, [some, stuff], but_soon_will_die, [why]}. ... -- Motiejus Jak?tys From andrzej.sliwa@REDACTED Mon Aug 27 23:00:14 2012 From: andrzej.sliwa@REDACTED (Andrzej Sliwa) Date: Mon, 27 Aug 2012 23:00:14 +0200 Subject: [erlang-questions] Why does Erlang have control structures? In-Reply-To: References: Message-ID: there is no difference in speed at all, > switch_signal({signal, _What, _From, _To}) -> > true; > switch_signal({signal, _What, _To}) -> > true; > switch_signal(_Else) -> > false. is compiled internally to one function with case so code speed is equal there is only difference in readability. On Aug 27, 2012, at 10:56 PM, Jayson Barley wrote: > I am not sure I understand why we have them. For instance I can take the following code > is_greater_than(X, Y) -> > if > X>Y -> > true; > true -> % works as an 'else' branch > false > end. > And make it > is_true(true) -> > true; > is_true(false) -> > false. > > is_greater_than(X, Y) -> > is_true(X>Y). > I can do the same thing with case statements > is_valid_signal(Signal) -> > case Signal of > {signal, _What, _From, _To} -> > true; > {signal, _What, _To} -> > true; > _Else -> > false > end. > Becomes > switch_signal({signal, _What, _From, _To}) -> > true; > switch_signal({signal, _What, _To}) -> > true; > switch_signal(_Else) -> > false. > > is_valid_signal(Signal) -> > switch_signal(Signal). > > I know that the control structures are a little bit faster, not much, but I find that the function form is more readable. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bourinov@REDACTED Mon Aug 27 23:00:30 2012 From: bourinov@REDACTED (Max Bourinov) Date: Tue, 28 Aug 2012 01:00:30 +0400 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: References: <503BD9DE.2080206@ninenines.eu> Message-ID: <5742923679170897044@unknownmsgid> Hot code update. This is super cool. Sent from my iPhone On 28.08.2012, at 0:53, "Motiejus Jak?tys" wrote: > On Mon, Aug 27, 2012 at 10:34 PM, Lo?c Hoguin wrote: >> Hello, >> >> I am wondering what you guys like the most about Erlang/OTP, especially >> newcomers, maybe it changed your life, allowed you to climb the Everest (or >> at least sleep at night). >> > > Hi, > killer feature for me is possibilities to interact with running system. > > You can send arbitrary messages to arbitraty processes, call arbitraty > functions, restart anything by hand. This gives utmost flexibility to > inspect and fix a running system. > > $ erl -setcookie company_name -sname d >> net_kernel:connect_node(db@REDACTED). > true >> rpc:call(db@REDACTED, yodadb, how_are_you, []). > {still_alive, [some, stuff], but_soon_will_die, [why]}. ... > > -- > Motiejus Jak?tys > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From olav@REDACTED Mon Aug 27 23:06:54 2012 From: olav@REDACTED (Olav Frengstad) Date: Mon, 27 Aug 2012 23:06:54 +0200 Subject: [erlang-questions] Dialyzer and binary() supertypes Message-ID: Hey, I have the following snippet of code which I'm trying to get Dialyzer to pass: -module(test). -compile([export_all]). -spec a({binary(), binary()} | binary()) -> atom(). a(<<"abc/def">>) -> ok1. a(<<"hijk/lmno">>) -> ok2. When running the above I get an error regarding the type specification: [/tmp] erlc +debug_info test.erl && dialyzer test.beam -Wunderspecs .... test.erl:7: Invalid type specification for function test:a/1. The success typing is (<<_:56,_:_*16>>) -> 'ok1' | 'ok2' Am I expected to create a custom type for to match static binary patterns, or is there some other type I can use for binaries with length() > 0? Regards, Olav -- Med Vennlig Hilsen Olav Frengstad Systemutvikler // FWT +47 920 42 090 From james@REDACTED Mon Aug 27 23:10:42 2012 From: james@REDACTED (James Aimonetti) Date: Mon, 27 Aug 2012 14:10:42 -0700 Subject: [erlang-questions] Dialyzer and binary() supertypes In-Reply-To: References: Message-ID: <503BE252.8010607@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 My assumption is that Dialyzer is letting you know that {binary(), binary()} is superfluous and can be removed (at least in the example you provided), and that you can be even more specific with your binary() declaration. - -type ne_binary() :: <<_:8,_:_*8>>. We use the above type to indicate the function does not accept empty binaries. See if that helps? On 08/27/2012 02:06 PM, Olav Frengstad wrote: > Hey, > > I have the following snippet of code which I'm trying to get > Dialyzer to pass: > > -module(test). > > -compile([export_all]). > > -spec a({binary(), binary()} | binary()) -> atom(). > a(<<"abc/def">>) -> ok1. a(<<"hijk/lmno">>) -> ok2. > > When running the above I get an error regarding the type > specification: [/tmp] erlc +debug_info test.erl && dialyzer > test.beam -Wunderspecs .... test.erl:7: Invalid type specification > for function test:a/1. The success typing is (<<_:56,_:_*16>>) -> > 'ok1' | 'ok2' > > Am I expected to create a custom type for to match static binary > patterns, or is there some other type I can use for binaries with > length() > 0? > > Regards, Olav > - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iQEcBAEBAgAGBQJQO+JSAAoJENc77s1OYoGg05UIAKs6YrgXUuiL02t1T5Ihj1yo muu9q69DJsO/0kH+Xi7vcUrSpudeApay6OxCl1RrNR2nsuQA4xtJBZtaEya52wi5 U+gZ/bIptSkG6T3UsNzxlWrNoL21+OGU42KK89R+d/DXOQYJjFC0izZGz1qqIqLs Mdz6qfTcuPPPMnpUYvxCpwmUEAZe5DyQNvKttQXOQBDx6tH6QNiLE2O3NZ5W+3iU KIke56oFppGGLsix5Rh8/QZdFEOPxeg5xOMsYujToWINYVLLrnNPHOoJ3joa6KGe ocoQZJuihY2gCDysA56fXC1yh1ElN5FO8CAB9EFHljAFdiVmSOD+1fv8GVHTO3Q= =d3gL -----END PGP SIGNATURE----- From max.lapshin@REDACTED Mon Aug 27 23:15:27 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 28 Aug 2012 01:15:27 +0400 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <5742923679170897044@unknownmsgid> References: <503BD9DE.2080206@ninenines.eu> <5742923679170897044@unknownmsgid> Message-ID: Simplicity together with performance and rock solid code from the very first line of code. From freza@REDACTED Mon Aug 27 23:07:42 2012 From: freza@REDACTED (freza@REDACTED) Date: Mon, 27 Aug 2012 17:07:42 -0400 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503BD9DE.2080206@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> Message-ID: <20120827210742.GA11096@circlewave.net> On Mon, Aug 27, 2012 at 10:34:38PM +0200, Lo?c Hoguin wrote: > I am wondering what you guys like the most about Erlang/OTP, especially > newcomers, maybe it changed your life, allowed you to climb the Everest > (or at least sleep at night). As newcomer: pattern matching and super-simple base language. A few years later: links & monitors, ability to effortlessly combine many processing stacks into single solution without silly interferences. Hard to list just a handful of properties though, it all fits together so nicely. BR, -- Jachym From james@REDACTED Mon Aug 27 23:32:17 2012 From: james@REDACTED (James Aimonetti) Date: Mon, 27 Aug 2012 14:32:17 -0700 Subject: [erlang-questions] Dialyzer and binary() supertypes In-Reply-To: References: <503BE252.8010607@2600hz.com> Message-ID: <503BE761.9040009@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Right, but you have -Wunderspecs on, which tells Dialyzer to "Warn about underspecified functions (the -spec is strictly more allowing than the success typing)". binary() and ne_binary() allow a wider range of binary constructions than the detected static binaries. Remove -Wunderspecs and you won't get this warning (or have a catch-all clause). On 08/27/2012 02:25 PM, Olav Frengstad wrote: > Hey, > > Sorry the spec in the first post should be: -spec a(binary()) -> > ok. > > Regarding the ne_binary() type, this does not work as expected > unless you have something like a(<<_>>) -> ok. I guess dialyzer > infers the length of all the possible static matches and expects a > equal length type. > > This is not an issue when binding a variable: a(<>) -> > ok. > > Cheers, Olav > > 2012/8/27 James Aimonetti : My assumption is that > Dialyzer is letting you know that {binary(), binary()} is > superfluous and can be removed (at least in the example you > provided), and that you can be even more specific with your > binary() declaration. > > -type ne_binary() :: <<_:8,_:_*8>>. > > We use the above type to indicate the function does not accept > empty binaries. See if that helps? > > On 08/27/2012 02:06 PM, Olav Frengstad wrote: >>>> Hey, >>>> >>>> I have the following snippet of code which I'm trying to get >>>> Dialyzer to pass: >>>> >>>> -module(test). >>>> >>>> -compile([export_all]). >>>> >>>> -spec a({binary(), binary()} | binary()) -> atom(). >>>> a(<<"abc/def">>) -> ok1. a(<<"hijk/lmno">>) -> ok2. >>>> >>>> When running the above I get an error regarding the type >>>> specification: [/tmp] erlc +debug_info test.erl && dialyzer >>>> test.beam -Wunderspecs .... test.erl:7: Invalid type >>>> specification for function test:a/1. The success typing is >>>> (<<_:56,_:_*16>>) -> 'ok1' | 'ok2' >>>> >>>> Am I expected to create a custom type for to match static >>>> binary patterns, or is there some other type I can use for >>>> binaries with length() > 0? >>>> >>>> Regards, Olav >>>> > > >> _______________________________________________ erlang-questions >> mailing list erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iQEcBAEBAgAGBQJQO+dhAAoJENc77s1OYoGgbxQH/jqTNa+niZ7MOTaaUXs95Zs0 /lKwXT5RQlK/kkvK7gzkloGkBQ0b8s35ESfP6iORjbyMqw56h5dXyS9+x8Ssv2w9 QhavboHDZQ4trjJ0q02ZwWphFRvN2nzekpys3rSYXxhFCUM3tJlq8D1YvTzk7WRP Kz98Mxa/K8LXplKq99JS4GFyGVZFtDX3XG4kTpAeKlVxSrYpACFgR9tTD2CDVALf t1nosvRSE/ZsIinTag2+/gMSiE5NBGR+ocDcFVaGEdDtw3WSn513do2lVihpi2uy vwYHcbqDDFN7afjlD8FOb5lJgCmLs8pc+2r52mKx3g5w28wgqvkihpQHRZDRsIM= =UYFM -----END PGP SIGNATURE----- From jayson.barley@REDACTED Mon Aug 27 23:44:53 2012 From: jayson.barley@REDACTED (Jayson Barley) Date: Mon, 27 Aug 2012 14:44:53 -0700 Subject: [erlang-questions] Why does Erlang have control structures? In-Reply-To: References: Message-ID: That is interesting. I didn't know that. I wonder why over 100 million iterations I consistently get between 1 to 2 second differences in timing. I know that there will be differences due to garbage collection, other processes running, etc, but the is_greater_than_control/2 function always outperforms the is_greater_than/2 function. Am I making a wrong assumption and eventually the opposite would be true? Could you test the below code on your system and let me know if you get similar results? -module(test). -compile(export_all). is_true(true) -> true; is_true(false) -> false. is_greater_than(X, Y) -> is_true(X>Y). is_greater_than_control(X, Y) -> if X>Y -> true; true -> % works as an 'else' branch false end. test_if(X, X, _) -> done; test_if(X, Max, Fun) -> Fun(X, Max), test_if(X + 1, Max, Fun). test_if(Max) -> [timer:tc(test, test_if, [0, Max, fun is_greater_than/2]), timer:tc(test, test_if, [0, Max, fun is_greater_than_control/2])]. 8> test:test_if(100000000). [{7234000,done},{5718999,done}] On Mon, Aug 27, 2012 at 2:00 PM, Andrzej Sliwa wrote: > there is no difference in speed at all, > > > switch_signal({signal, _What, _From, _To}) -> > > true; > > switch_signal({signal, _What, _To}) -> > > true; > > switch_signal(_Else) -> > > false. > > is compiled internally to one function with case > so code speed is equal > > there is only difference in readability. > > On Aug 27, 2012, at 10:56 PM, Jayson Barley > wrote: > > > I am not sure I understand why we have them. For instance I can take the > following code > > is_greater_than(X, Y) -> > > if > > X>Y -> > > true; > > true -> % works as an 'else' branch > > false > > end. > > And make it > > is_true(true) -> > > true; > > is_true(false) -> > > false. > > > > is_greater_than(X, Y) -> > > is_true(X>Y). > > I can do the same thing with case statements > > is_valid_signal(Signal) -> > > case Signal of > > {signal, _What, _From, _To} -> > > true; > > {signal, _What, _To} -> > > true; > > _Else -> > > false > > end. > > Becomes > > switch_signal({signal, _What, _From, _To}) -> > > true; > > switch_signal({signal, _What, _To}) -> > > true; > > switch_signal(_Else) -> > > false. > > > > is_valid_signal(Signal) -> > > switch_signal(Signal). > > > > I know that the control structures are a little bit faster, not much, > but I find that the function form is more readable. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chan.sisowath@REDACTED Tue Aug 28 00:17:10 2012 From: chan.sisowath@REDACTED (chan sisowath) Date: Tue, 28 Aug 2012 00:17:10 +0200 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503BD9DE.2080206@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> Message-ID: as a newcomer, learn cool stuff, base code for a given project is small compare to other language, then you can dig in easely to learn. so plaisant to do stuff you never could imagine to do in language like PHP or java without pain and also programming in erlang is more fun than before. the main probleme, there are few position in erlang programming ;). 2012/8/27 Lo?c Hoguin > Hello, > > I am wondering what you guys like the most about Erlang/OTP, especially > newcomers, maybe it changed your life, allowed you to climb the Everest (or > at least sleep at night). > > I'll be synthesizing that into a few key points that I am hopeful will be > reusable by anyone wanting to sell Erlang to their friends/work/clients. > > So help me by telling us what's so good about it! > > Thanks. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From puzza007@REDACTED Tue Aug 28 00:35:58 2012 From: puzza007@REDACTED (Paul Oliver) Date: Mon, 27 Aug 2012 18:35:58 -0400 Subject: [erlang-questions] Why does Erlang have control structures? In-Reply-To: References: Message-ID: You can use erlc +to_core to get the core erlang: module 'test' ['is_greater_than'/2, 'is_greater_than_control'/2, 'is_true'/1, 'module_info'/0, 'module_info'/1] attributes [] 'is_true'/1 = %% Line 5 fun (_cor0) -> case _cor0 of <'true'> when 'true' -> %% Line 6 'true' %% Line 7 <'false'> when 'true' -> %% Line 8 'false' ( <_cor1> when 'true' -> ( primop 'match_fail' ({'function_clause',_cor1}) -| [{'function_name',{'is_true',1}}] ) -| ['compiler_generated'] ) end 'is_greater_than'/2 = %% Line 10 fun (_cor1,_cor0) -> let <_cor2> = %% Line 11 call 'erlang':'>' (_cor1, _cor0) in %% Line 11 apply 'is_true'/1 (_cor2) 'is_greater_than_control'/2 = %% Line 13 fun (_cor1,_cor0) -> %% Line 15 case <> of %% Line 16 <> when call 'erlang':'>' (_cor1, _cor0) -> %% Line 17 'true' %% Line 18 <> when 'true' -> %% Line 19 'false' end 'module_info'/0 = fun () -> call 'erlang':'get_module_info' ('test') 'module_info'/1 = fun (_cor0) -> call 'erlang':'get_module_info' ('test', _cor0) end On Mon, Aug 27, 2012 at 5:44 PM, Jayson Barley wrote: > That is interesting. I didn't know that. I wonder why over 100 million > iterations I consistently get between 1 to 2 second differences in timing. > I know that there will be differences due to garbage collection, other > processes running, etc, but the is_greater_than_control/2 function always > outperforms the is_greater_than/2 function. Am I making a wrong assumption > and eventually the opposite would be true? Could you test the below code on > your system and let me know if you get similar results? > > -module(test). > -compile(export_all). > > > is_true(true) -> > true; > is_true(false) -> > false. > > is_greater_than(X, Y) -> > is_true(X>Y). > > is_greater_than_control(X, Y) -> > > if > X>Y -> > true; > true -> % works as an 'else' branch > false > end. > > test_if(X, X, _) -> > done; > test_if(X, Max, Fun) -> > Fun(X, Max), > test_if(X + 1, Max, Fun). > > test_if(Max) -> > [timer:tc(test, test_if, [0, Max, fun is_greater_than/2]), > timer:tc(test, test_if, [0, Max, fun is_greater_than_control/2])]. > > 8> test:test_if(100000000). > [{7234000,done},{5718999,done}] > > > On Mon, Aug 27, 2012 at 2:00 PM, Andrzej Sliwa wrote: > >> there is no difference in speed at all, >> >> > switch_signal({signal, _What, _From, _To}) -> >> > true; >> > switch_signal({signal, _What, _To}) -> >> > true; >> > switch_signal(_Else) -> >> > false. >> >> is compiled internally to one function with case >> so code speed is equal >> >> there is only difference in readability. >> >> On Aug 27, 2012, at 10:56 PM, Jayson Barley >> wrote: >> >> > I am not sure I understand why we have them. For instance I can take >> the following code >> > is_greater_than(X, Y) -> >> > if >> > X>Y -> >> > true; >> > true -> % works as an 'else' branch >> > false >> > end. >> > And make it >> > is_true(true) -> >> > true; >> > is_true(false) -> >> > false. >> > >> > is_greater_than(X, Y) -> >> > is_true(X>Y). >> > I can do the same thing with case statements >> > is_valid_signal(Signal) -> >> > case Signal of >> > {signal, _What, _From, _To} -> >> > true; >> > {signal, _What, _To} -> >> > true; >> > _Else -> >> > false >> > end. >> > Becomes >> > switch_signal({signal, _What, _From, _To}) -> >> > true; >> > switch_signal({signal, _What, _To}) -> >> > true; >> > switch_signal(_Else) -> >> > false. >> > >> > is_valid_signal(Signal) -> >> > switch_signal(Signal). >> > >> > I know that the control structures are a little bit faster, not much, >> but I find that the function form is more readable. >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chan.sisowath@REDACTED Tue Aug 28 00:40:00 2012 From: chan.sisowath@REDACTED (chan sisowath) Date: Tue, 28 Aug 2012 00:40:00 +0200 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: References: <503BD9DE.2080206@ninenines.eu> Message-ID: erlang itself 2012/8/28 chan sisowath > as a newcomer, > > learn cool stuff, base code for a given project is small compare to other > language, > then you can dig in easely to learn. > so plaisant to do stuff you never could imagine > to do in language like PHP or java without pain > and also programming in erlang is more fun than before. > > the main probleme, there are few position in erlang programming ;). > > 2012/8/27 Lo?c Hoguin > >> Hello, >> >> I am wondering what you guys like the most about Erlang/OTP, especially >> newcomers, maybe it changed your life, allowed you to climb the Everest (or >> at least sleep at night). >> >> I'll be synthesizing that into a few key points that I am hopeful will be >> reusable by anyone wanting to sell Erlang to their friends/work/clients. >> >> So help me by telling us what's so good about it! >> >> Thanks. >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayson.barley@REDACTED Tue Aug 28 00:47:27 2012 From: jayson.barley@REDACTED (Jayson Barley) Date: Mon, 27 Aug 2012 15:47:27 -0700 Subject: [erlang-questions] Why does Erlang have control structures? In-Reply-To: References: Message-ID: Thank you Paul. It appears to me that they don't compile down to the same code. Am I missing something? On Mon, Aug 27, 2012 at 3:35 PM, Paul Oliver wrote: > You can use erlc +to_core to get the core erlang: > > module 'test' ['is_greater_than'/2, > 'is_greater_than_control'/2, > 'is_true'/1, > 'module_info'/0, > 'module_info'/1] > attributes [] > 'is_true'/1 = > %% Line 5 > fun (_cor0) -> > case _cor0 of > <'true'> when 'true' -> > %% Line 6 > 'true' > %% Line 7 > <'false'> when 'true' -> > %% Line 8 > 'false' > ( <_cor1> when 'true' -> > ( primop 'match_fail' > ({'function_clause',_cor1}) > -| [{'function_name',{'is_true',1}}] ) > -| ['compiler_generated'] ) > end > 'is_greater_than'/2 = > %% Line 10 > fun (_cor1,_cor0) -> > let <_cor2> = > %% Line 11 > call 'erlang':'>' > (_cor1, _cor0) > in %% Line 11 > apply 'is_true'/1 > (_cor2) > 'is_greater_than_control'/2 = > %% Line 13 > fun (_cor1,_cor0) -> > %% Line 15 > case <> of > %% Line 16 > <> > when call 'erlang':'>' > (_cor1, > _cor0) -> > %% Line 17 > 'true' > %% Line 18 > <> when 'true' -> > %% Line 19 > 'false' > end > 'module_info'/0 = > fun () -> > call 'erlang':'get_module_info' > ('test') > 'module_info'/1 = > fun (_cor0) -> > call 'erlang':'get_module_info' > ('test', _cor0) > end > > On Mon, Aug 27, 2012 at 5:44 PM, Jayson Barley wrote: > >> That is interesting. I didn't know that. I wonder why over 100 million >> iterations I consistently get between 1 to 2 second differences in timing. >> I know that there will be differences due to garbage collection, other >> processes running, etc, but the is_greater_than_control/2 function always >> outperforms the is_greater_than/2 function. Am I making a wrong assumption >> and eventually the opposite would be true? Could you test the below code on >> your system and let me know if you get similar results? >> >> -module(test). >> -compile(export_all). >> >> >> is_true(true) -> >> true; >> is_true(false) -> >> false. >> >> is_greater_than(X, Y) -> >> is_true(X>Y). >> >> is_greater_than_control(X, Y) -> >> >> if >> X>Y -> >> true; >> true -> % works as an 'else' branch >> false >> end. >> >> test_if(X, X, _) -> >> done; >> test_if(X, Max, Fun) -> >> Fun(X, Max), >> test_if(X + 1, Max, Fun). >> >> test_if(Max) -> >> [timer:tc(test, test_if, [0, Max, fun is_greater_than/2]), >> timer:tc(test, test_if, [0, Max, fun is_greater_than_control/2])]. >> >> 8> test:test_if(100000000). >> [{7234000,done},{5718999,done}] >> >> >> On Mon, Aug 27, 2012 at 2:00 PM, Andrzej Sliwa wrote: >> >>> there is no difference in speed at all, >>> >>> > switch_signal({signal, _What, _From, _To}) -> >>> > true; >>> > switch_signal({signal, _What, _To}) -> >>> > true; >>> > switch_signal(_Else) -> >>> > false. >>> >>> is compiled internally to one function with case >>> so code speed is equal >>> >>> there is only difference in readability. >>> >>> On Aug 27, 2012, at 10:56 PM, Jayson Barley >>> wrote: >>> >>> > I am not sure I understand why we have them. For instance I can take >>> the following code >>> > is_greater_than(X, Y) -> >>> > if >>> > X>Y -> >>> > true; >>> > true -> % works as an 'else' branch >>> > false >>> > end. >>> > And make it >>> > is_true(true) -> >>> > true; >>> > is_true(false) -> >>> > false. >>> > >>> > is_greater_than(X, Y) -> >>> > is_true(X>Y). >>> > I can do the same thing with case statements >>> > is_valid_signal(Signal) -> >>> > case Signal of >>> > {signal, _What, _From, _To} -> >>> > true; >>> > {signal, _What, _To} -> >>> > true; >>> > _Else -> >>> > false >>> > end. >>> > Becomes >>> > switch_signal({signal, _What, _From, _To}) -> >>> > true; >>> > switch_signal({signal, _What, _To}) -> >>> > true; >>> > switch_signal(_Else) -> >>> > false. >>> > >>> > is_valid_signal(Signal) -> >>> > switch_signal(Signal). >>> > >>> > I know that the control structures are a little bit faster, not much, >>> but I find that the function form is more readable. >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Aug 28 00:50:40 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 28 Aug 2012 10:50:40 +1200 Subject: [erlang-questions] Graphing tools In-Reply-To: References: Message-ID: On 28/08/2012, at 12:15 AM, kilgore trout wrote: > Hello Langers! > I'm looking for a graphing tool, anyone know of any useful libraries? > I have piles of performance data in .csv format and a parser to read it into records. I have been trying to find patterns and draw some meaningful conclusions but it would be so much easier if I could represent it graphically. You should probably do it in R (http://www.r-project.org). It's an environment, not a library, but if you want to find patterns in data, it's hard to go past. I believe there is an Erlang/R bridge somewhere. From ok@REDACTED Tue Aug 28 01:32:32 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 28 Aug 2012 11:32:32 +1200 Subject: [erlang-questions] Why does Erlang have control structures? In-Reply-To: References: Message-ID: On 28/08/2012, at 8:56 AM, Jayson Barley wrote: > I am not sure I understand why we have them. "if" is indeed a special case of "case", and "case" is basically an in-lined function call. And there is a hint: if you are going to inline a function call, what are you going to inline it *as*? The possibility must exist in the core syntax, if nowhere else. But "try" and "receive" couldn't be anything else. Without "receive", we'd have to use channels. > is_greater_than(X, Y) -> > is_true(X>Y). That would never have worked in the Good Old Days, where X > Y was a guard test and not usable in an ordinary expression. > I know that the control structures are a little bit faster, not much, but I find that the function form is more readable. You will find it hard to get a consensus on that. From stu.bailey@REDACTED Tue Aug 28 01:42:31 2012 From: stu.bailey@REDACTED (Stu Bailey) Date: Mon, 27 Aug 2012 16:42:31 -0700 Subject: [erlang-questions] Graphing tools In-Reply-To: References: Message-ID: There are several nice javascript data visualization tools here: http://d3js.org/ It's straight forward to send lots of data from Erlang to Javascript running in a web browser using: * BERT (http://bert-rpc.org/) * BERT-JS (https://github.com/rustyio/BERT-JS) * Websockets (http://www.websocket.org/) * Yaws (http://yaws.hyber.org/websockets.yaws) The nice thing is that on the Erlang side you are just sending/receiving Erlang terms (i.e no complicated marshaling and un-marshaling on the Erlang side). So you can do the data reduction/analysis/computation on the Erlang side and render the data in some meaningful visual way on the Javascript (web browser) side. Have fun! Stu On Mon, Aug 27, 2012 at 3:50 PM, Richard O'Keefe wrote: > > On 28/08/2012, at 12:15 AM, kilgore trout wrote: > >> Hello Langers! >> I'm looking for a graphing tool, anyone know of any useful libraries? >> I have piles of performance data in .csv format and a parser to read it into records. I have been trying to find patterns and draw some meaningful conclusions but it would be so much easier if I could represent it graphically. > > You should probably do it in R (http://www.r-project.org). > It's an environment, not a library, but if you want to find > patterns in data, it's hard to go past. > > I believe there is an Erlang/R bridge somewhere. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Tue Aug 28 02:26:41 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 28 Aug 2012 12:26:41 +1200 Subject: [erlang-questions] Graphing tools In-Reply-To: References: Message-ID: <253B92DD-70DE-4EA8-B7B2-E36811966BF2@cs.otago.ac.nz> On 28/08/2012, at 11:42 AM, Stu Bailey wrote: > There are several nice javascript data visualization tools here: The reason that I recommended R is that there is a *vast* amount of stuff already available free for/in R to do all sorts of data mining and analysis stuff. There is also an R mailing list that has some very smart, informed, and helpful people on it, and it is even more active than the Erlang mailing list. If it is just about pretty pictures, javascript is cool, but so is rrd. If it's anything more than that, you need at least something like Matlab or R or Octave or ... Just out of curiosity, could the raw data be made available? From xiaopong.tran@REDACTED Tue Aug 28 04:45:18 2012 From: xiaopong.tran@REDACTED (Xiaopong Tran) Date: Tue, 28 Aug 2012 10:45:18 +0800 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: References: <503BD9DE.2080206@ninenines.eu> Message-ID: <503C30BE.6020600@gmail.com> On 08/28/2012 06:17 AM, chan sisowath wrote: > as a newcomer, > > learn cool stuff, base code for a given project is small compare to > other language, > then you can dig in easely to learn. > so plaisant to do stuff you never could imagine > to do in language like PHP or java without pain > and also programming in erlang is more fun than before. > > the main probleme, there are few position in erlang programming ;). > Not to hijack the discussion topic, we still have 4 to 5 erlang openings in Shanghai to fill :) We are in a new phase of expansion, and we are hiring more erlang devs to work on a distributed system. Please get in touch with me, if you are interested. Xiaopong From ttmrichter@REDACTED Tue Aug 28 04:50:40 2012 From: ttmrichter@REDACTED (Michael Richter) Date: Tue, 28 Aug 2012 10:50:40 +0800 Subject: [erlang-questions] Why does Erlang have control structures? In-Reply-To: References: Message-ID: On 28 August 2012 04:56, Jayson Barley wrote: > I can do the same thing with case statements > > is_valid_signal(Signal) -> > case Signal of > {signal, _What, _From, _To} -> > true; > {signal, _What, _To} -> > true; > _Else -> > false > end. > > Becomes > > switch_signal({signal, _What, _From, _To}) -> > true; > switch_signal({signal, _What, _To}) -> > true; > switch_signal(_Else) -> > false. > > is_valid_signal(Signal) -> > switch_signal(Signal). > > > I know that the control structures are a little bit faster, not much, but > I find that the function form is more readable. > A case structure is like an *anonymous* function in this. The use case for them is the same as the use case for funs. You had to come up with, even in your trivial example, two module-global symbols: is_valid_signal/1 and switch_signal/1. Take a non-trivial module, now, and multiply it by the number of public API functions. You'll start getting name clashes as you go on. For example, with your code above, what happens if I want to tell a device to change the signal it's sending? I can't use switch_signal/1 any longer, so I have to use something like change_signal/1 instead. Now I've got two functions, switch_signal/1 and change_signal/1, that look very similar by name. I have to dig deeper to figure out which does what if I'm maintaining the code base. You could argue that this is because switch_signal/1 is poorly named, but then it falls right back into why case structures are useful: they're *anonymous*. Anonymous constructs (cases, funs, etc.) are quite often very useful. Further, your version of is_valid_signal/1 has a larger cognitive load. In the first case, if I want to know what is_valid_signal does and how it works, I look up one function. All the information I need to understand the function is in one place. With your solution this is now divided up into multiple places. There's more work to decode what's going on. This being a trivial function disguises the effect somewhat, but in a larger, less trivial function this can be an impediment, again, to understanding the code and reasoning about its behaviour. -- "Perhaps people don't believe this, but throughout all of the discussions of entering China our focus has really been what's best for the Chinese people. It's not been about our revenue or profit or whatnot." --Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Tue Aug 28 06:17:58 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Tue, 28 Aug 2012 07:17:58 +0300 Subject: [erlang-questions] Graphing tools In-Reply-To: <253B92DD-70DE-4EA8-B7B2-E36811966BF2@cs.otago.ac.nz> References: <253B92DD-70DE-4EA8-B7B2-E36811966BF2@cs.otago.ac.nz> Message-ID: <-1987573917525763736@unknownmsgid> Hello, No doubt R is super for data analysis. It also includes various data visualization primitives. It provides you multiple data aggregation interfaces including csv, MySQL, etc. My issue with R was a performance for online analysis and visualization of data. It was a case for handling data from 128 sensors. Best Regards, Dmitry >-|-|-*> On 28.8.2012, at 3.32, Richard O'Keefe wrote: > > On 28/08/2012, at 11:42 AM, Stu Bailey wrote: > >> There are several nice javascript data visualization tools here: > > The reason that I recommended R is that there is a *vast* amount > of stuff already available free for/in R to do all sorts of data > mining and analysis stuff. There is also an R mailing list that > has some very smart, informed, and helpful people on it, and it > is even more active than the Erlang mailing list. If it is just > about pretty pictures, javascript is cool, but so is rrd. > If it's anything more than that, you need at least something like > Matlab or R or Octave or ... > > Just out of curiosity, could the raw data be made available? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ivan@REDACTED Tue Aug 28 07:43:29 2012 From: ivan@REDACTED (Ivan Uemlianin) Date: Tue, 28 Aug 2012 06:43:29 +0100 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503BD9DE.2080206@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> Message-ID: I loved the ease of working with binary data. Using that for the first time was "laugh out loud". I liked the fact that erlang was an "old" language --- not "flavour of the month" or "new kid on the block". Erlang had matured for so long quietly --- like good wine or cheese. This helped my client of the time go with erlang too --- the huge projects that erlang had been used for. Line-by-line erlang can feel difficult and slow to work with, but when you look back at what you've created (and consider how you might have created that in Your Other Favourite Language), you realise erlang is a very helpful language. I like the fact that "behaviour" is spelt properly. Ivan -- hilaritas excessum habere nequit. On 27 Aug 2012, at 21:34, Lo?c Hoguin wrote: > Hello, > > I am wondering what you guys like the most about Erlang/OTP, especially newcomers, maybe it changed your life, allowed you to climb the Everest (or at least sleep at night). > > I'll be synthesizing that into a few key points that I am hopeful will be reusable by anyone wanting to sell Erlang to their friends/work/clients. > > So help me by telling us what's so good about it! > > Thanks. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ivan@REDACTED Tue Aug 28 08:09:05 2012 From: ivan@REDACTED (Ivan Uemlianin) Date: Tue, 28 Aug 2012 07:09:05 +0100 Subject: [erlang-questions] Graphing tools In-Reply-To: References: Message-ID: +1 for R. Their visualisation libraries are second to none as well. Ivan -- hilaritas excessum habere nequit. On 27 Aug 2012, at 23:50, "Richard O'Keefe" wrote: > > On 28/08/2012, at 12:15 AM, kilgore trout wrote: > >> Hello Langers! >> I'm looking for a graphing tool, anyone know of any useful libraries? >> I have piles of performance data in .csv format and a parser to read it into records. I have been trying to find patterns and draw some meaningful conclusions but it would be so much easier if I could represent it graphically. > > You should probably do it in R (http://www.r-project.org). > It's an environment, not a library, but if you want to find > patterns in data, it's hard to go past. > > I believe there is an Erlang/R bridge somewhere. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From Sergey_Zhemzhitsky@REDACTED Tue Aug 28 08:46:54 2012 From: Sergey_Zhemzhitsky@REDACTED (Zhemzhitsky Sergey) Date: Tue, 28 Aug 2012 06:46:54 +0000 Subject: [erlang-questions] Logger to log terms, that can contain unicode chars In-Reply-To: <20120827154059.GL5837@hijacked.us> References: <06139A918ACCA041BF46A0F36940C7FA4FE199CF@exch-mbx2.msk.trd.ru> <20120827154059.GL5837@hijacked.us> Message-ID: <06139A918ACCA041BF46A0F36940C7FA4FE19D60@exch-mbx2.msk.trd.ru> Hi Andrew, I have raised an issue - https://github.com/basho/lager/issues/75 Best Regards, Sergey -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Andrew Thompson Sent: Monday, August 27, 2012 7:41 PM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] Logger to log terms, that can contain unicode chars On Mon, Aug 27, 2012 at 03:12:30PM +0000, Zhemzhitsky Sergey wrote: > Hi erlangers, > > Could you suggest any logger that can accept erlang terms which contains strings with Unicode characters? > Or could you suggest how the lager should be configured properly. > > I was not able to get correct string presentation with lager doing > like so > lager:info(?Info: ~p?, [{tuple, ?string with umlauts u?}]) Looks like a bug to do with printing unicode with the ~p formatter, things seem to work ok if you use ~s or ~ts. Could you file a bug so I can look into it more when I have time? Andrew _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________________ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp From chenbo09thss@REDACTED Tue Aug 28 11:17:29 2012 From: chenbo09thss@REDACTED (Bo Chen) Date: Tue, 28 Aug 2012 17:17:29 +0800 Subject: [erlang-questions] How to use dbg and fprof to profile a running server ? Message-ID: Hello, every one, I am a rookie in erlang. Recently I am trying to use the dbg and fprof module to profile a running server to observe the performance. Since the server is based on mochiweb, a quite popular web server framework, and I only wish to see the performance of my own code, I tried the dbg module. I have tried in the following two ways: 1: use dbg to write trace to file and use fprof to analyse the result dbg:stop_clear(), dbg:tracer(port,dbg:trace_port(file,"fprof.trace")), dbg:tpl(util,[]), dbg:tpl(...), dbg:p(all,[call,return_to, procs, timestamp,running,garbage_collection]), dbg:stop_clear(), fprof:profile(), Fname = lists:append("profile/",Name), fprof:analyse({dest, Fname}). 2: like fprof, but use dbg:tpl to confine the module fprof:trace([start, {procs, all}]), dbg:tpl(util,[]), dbg:tpl(...), fprof:trace([stop]), fprof:profile(), Fname = lists:append("profile/",Name), fprof:analyse({dest, Fname}). Sadly, neither way seems to work. So any idea about how to combine the dbg and fprof module? Or maybe someone would like to point out the mistake I have made? Thanks in advance for your reply~ -- ?? / Bill 2009???? ???????? ------------------------------------------------------------ Chen Bo School of Software Tsinghua University Add: Building Zijing 1# Room 210B, Tsinghua University Beijing 100084 P.R.CHINA Tel: +86-010-51534210 E-mail: chenbo09thss@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From armstrong.whit@REDACTED Tue Aug 28 11:48:47 2012 From: armstrong.whit@REDACTED (Whit Armstrong) Date: Tue, 28 Aug 2012 05:48:47 -0400 Subject: [erlang-questions] Graphing tools In-Reply-To: <-1987573917525763736@unknownmsgid> References: <253B92DD-70DE-4EA8-B7B2-E36811966BF2@cs.otago.ac.nz> <-1987573917525763736@unknownmsgid> Message-ID: R graphics are slowing imrpoving. Have a look at the iPlots. The Simon Urbanek's presentation from R/Fin is not online (http://www.rinfinance.com/agenda/). I'll post a link it if I can find a copy. Here are some other references: http://www.rosuda.org/iplots/ http://csgillespie.wordpress.com/2011/08/18/simon-urbanek-r-graphics-supercharged/ Also, a lot of people like ggplot2: http://had.co.nz/ggplot2/ -Whit On Tue, Aug 28, 2012 at 12:17 AM, dmitry kolesnikov wrote: > Hello, > > No doubt R is super for data analysis. It also includes various data > visualization primitives. It provides you multiple data aggregation > interfaces including csv, MySQL, etc. > > My issue with R was a performance for online analysis and > visualization of data. It was a case for handling data from 128 > sensors. > > > Best Regards, > Dmitry >-|-|-*> > > > On 28.8.2012, at 3.32, Richard O'Keefe wrote: > >> >> On 28/08/2012, at 11:42 AM, Stu Bailey wrote: >> >>> There are several nice javascript data visualization tools here: >> >> The reason that I recommended R is that there is a *vast* amount >> of stuff already available free for/in R to do all sorts of data >> mining and analysis stuff. There is also an R mailing list that >> has some very smart, informed, and helpful people on it, and it >> is even more active than the Erlang mailing list. If it is just >> about pretty pictures, javascript is cool, but so is rrd. >> If it's anything more than that, you need at least something like >> Matlab or R or Octave or ... >> >> Just out of curiosity, could the raw data be made available? >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andre@REDACTED Tue Aug 28 12:43:21 2012 From: andre@REDACTED (=?ISO-8859-1?Q?Andr=E9_Graf?=) Date: Tue, 28 Aug 2012 12:43:21 +0200 Subject: [erlang-questions] Graphing tools In-Reply-To: References: <253B92DD-70DE-4EA8-B7B2-E36811966BF2@cs.otago.ac.nz> <-1987573917525763736@unknownmsgid> Message-ID: R with ggplot2 +1 On 28 August 2012 11:48, Whit Armstrong wrote: > R graphics are slowing imrpoving. > > Have a look at the iPlots. > > The Simon Urbanek's presentation from R/Fin is not online > (http://www.rinfinance.com/agenda/). I'll post a link it if I can > find a copy. > > Here are some other references: > > http://www.rosuda.org/iplots/ > http://csgillespie.wordpress.com/2011/08/18/simon-urbanek-r-graphics-supercharged/ > > Also, a lot of people like ggplot2: > http://had.co.nz/ggplot2/ > > -Whit > > > On Tue, Aug 28, 2012 at 12:17 AM, dmitry kolesnikov > wrote: >> Hello, >> >> No doubt R is super for data analysis. It also includes various data >> visualization primitives. It provides you multiple data aggregation >> interfaces including csv, MySQL, etc. >> >> My issue with R was a performance for online analysis and >> visualization of data. It was a case for handling data from 128 >> sensors. >> >> >> Best Regards, >> Dmitry >-|-|-*> >> >> >> On 28.8.2012, at 3.32, Richard O'Keefe wrote: >> >>> >>> On 28/08/2012, at 11:42 AM, Stu Bailey wrote: >>> >>>> There are several nice javascript data visualization tools here: >>> >>> The reason that I recommended R is that there is a *vast* amount >>> of stuff already available free for/in R to do all sorts of data >>> mining and analysis stuff. There is also an R mailing list that >>> has some very smart, informed, and helpful people on it, and it >>> is even more active than the Erlang mailing list. If it is just >>> about pretty pictures, javascript is cool, but so is rrd. >>> If it's anything more than that, you need at least something like >>> Matlab or R or Octave or ... >>> >>> Just out of curiosity, could the raw data be made available? >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Tue Aug 28 13:12:30 2012 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 28 Aug 2012 13:12:30 +0200 Subject: [erlang-questions] Why does Erlang have control structures? In-Reply-To: References: Message-ID: On Mon, Aug 27, 2012 at 10:56 PM, Jayson Barley wrote: > I am not sure I understand why we have them. For instance I can take the > following code > > is_greater_than(X, Y) -> > if > X>Y -> > true; > true -> % works as an 'else' branch > false > end. > > And make it > > is_true(true) -> > true; > is_true(false) -> > false. Or even is_greater_than(X, Y) -> X > Y. /J > is_greater_than(X, Y) -> > is_true(X>Y). > > I can do the same thing with case statements > > is_valid_signal(Signal) -> > case Signal of > {signal, _What, _From, _To} -> > true; > {signal, _What, _To} -> > true; > _Else -> > false > end. > > Becomes > > switch_signal({signal, _What, _From, _To}) -> > true; > switch_signal({signal, _What, _To}) -> > true; > switch_signal(_Else) -> > false. > > is_valid_signal(Signal) -> > switch_signal(Signal). > > > I know that the control structures are a little bit faster, not much, but I > find that the function form is more readable. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From daniel@REDACTED Tue Aug 28 13:15:23 2012 From: daniel@REDACTED (Daniel Eliasson) Date: Tue, 28 Aug 2012 13:15:23 +0200 Subject: [erlang-questions] [ANN] erserve Erlang/R (Rserve) connector - 3.0.0 Message-ID: Dear all, I announce to you the existence of erserve 3.0.0. It is available at https://github.com/del/erserve and a connection pool library for it at https://github.com/del/erserve_pool What is it? === erserve is an Erlang application that lets you easily connect to an R instance via Rserve ( http://rforge.net/Rserve/ ) over a binary protocol. You can send strings to R to be evaluated, and also upload data to variables in R via binary transport. Who is it for? === erserve should come in handy if you want to perform some kind of calculations that are tedious/difficult in Erlang, for instance running data through machine learning models. It can also shorten your deployment times by allowing you to model things in R, and then just use your results from Erlang, instead of porting them there. Is it fast? === It's not bad. With 10 workers and a pool of 10 erserve connections, my workstation (Core i7 2.93 Ghz, 16 GB RAM) could run 10 000 individual classifications through a random forest in 8 seconds. That's about 1250 calls per second in throughput, with each invidivual call taking about 3 ms round-trip. === If you have any questions, please let me know. If anyone has use for this, I would greatly appreciate feedback on improvements. I'd also like to thank my employer, Klarna AB, for letting me do some of the work on this open source software during work hours. Best, Daniel Eliasson From hukl@REDACTED Tue Aug 28 15:07:10 2012 From: hukl@REDACTED (John-Paul Bader) Date: Tue, 28 Aug 2012 15:07:10 +0200 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: References: <503BD9DE.2080206@ninenines.eu> Message-ID: <503CC27E.5070404@berlin.ccc.de> I think the question would be more interesting the other way around. What do you not like about Erlang/OTP. Here is my list: * Inconsistencies/Missing functionality in the standard library: - The principle of least surprise does not apply - proplists:get_value(foo,[]) returns undefined orrdict:fetch(foo, []) throws and error. There are plenty of examples where similar operations in different modules either throw or just return undefined but never in a predictable way My favorite: calling split on a one element list throws an error - functions operating on lists sometimes expect the list as first argument, sometimes as the last argument - merge, explode, implode, traversing, filter, search for dictionaries would be nice - missing json parser - crappy http support! Its the 21st century and the web is a major thing by now and people want to build awesome backends with erlang but in terms of http erlang really is a ghetto. the supplied client and server modules fail at the slightest traffic blow - calender implementation is super weak and is missing a lot of functionality which I see people implementing over and over again - there are sooo soo many things that you could improve about the standard library its actually kind of sad. * OTP Releases: for most of the use cases other than big telcos or banks this is far more confusing then helpful especially when beginning erlang. While I'm sure there are valid use cases for it most of the time I thought it was bloated, cumbersome, slow and error prone. * No built in support / standard way for installing libraries - instead people have to find out about rebar themselves. Look at http://rubygems.org in contrast or https://www.ruby-toolbox.com/categories * Lack of namespaces (duck and cover) - Actually I'd be more than happy with a ruby / python way of doing it but I guess that won't happen * I also think the supplied test libraries are not that great to get started or to motivate anybody to test from day one * Error messages are often not helpful. BadMatch etc could include more context information to spot the errors faster There are a lot of things I like and even love about Erlang but these are the ones that constantly annoy me. ~ John From manprog@REDACTED Tue Aug 28 15:30:44 2012 From: manprog@REDACTED (Bernardo Alvez) Date: Tue, 28 Aug 2012 10:30:44 -0300 Subject: [erlang-questions] [ANN] erserve Erlang/R (Rserve) connector - 3.0.0 In-Reply-To: References: Message-ID: <503CC804.50809@adinet.com.uy> Thank you Daniel, very interesting! Bernardo Daniel Eliasson escribi?: > Dear all, > > I announce to you the existence of erserve 3.0.0. It is available at > https://github.com/del/erserve and a connection pool library for it at > https://github.com/del/erserve_pool > > What is it? > === > erserve is an Erlang application that lets you easily connect to an R > instance via Rserve ( http://rforge.net/Rserve/ ) over a binary > protocol. > You can send strings to R to be evaluated, and also upload data to > variables in R via binary transport. > > Who is it for? > === > erserve should come in handy if you want to perform some kind of > calculations that are tedious/difficult in Erlang, for instance > running data through machine learning models. It can also shorten your > deployment times by allowing you to model things in R, and then just > use your results from Erlang, instead of porting them there. > > Is it fast? > === > It's not bad. With 10 workers and a pool of 10 erserve connections, my > workstation (Core i7 2.93 Ghz, 16 GB RAM) could run 10 000 individual > classifications through a random forest in 8 seconds. That's about > 1250 calls per second in throughput, with each invidivual call taking > about 3 ms round-trip. > > === > If you have any questions, please let me know. If anyone has use for > this, I would greatly appreciate feedback on improvements. > > I'd also like to thank my employer, Klarna AB, for letting me do some > of the work on this open source software during work hours. > > Best, > Daniel Eliasson > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From essen@REDACTED Tue Aug 28 16:00:23 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 28 Aug 2012 16:00:23 +0200 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503CC27E.5070404@berlin.ccc.de> References: <503BD9DE.2080206@ninenines.eu> <503CC27E.5070404@berlin.ccc.de> Message-ID: <503CCEF7.4080405@ninenines.eu> On 08/28/2012 03:07 PM, John-Paul Bader wrote: > I think the question would be more interesting the other way around. > What do you not like about Erlang/OTP. Here is my list: Unfortunately that doesn't help me! I'll answer a few points anyway. > - missing json parser You are kidding, right? There's at least 10 of them available. (I'm not going to go into the debate of "included by default" vs otherwise, I'm a big proponent of a tiny OTP distribution and letting people choose what to use. So I wouldn't want a JSON parser by default.) > - crappy http support! Its the 21st century and the web is a major > thing by now and people want to build awesome backends with erlang > but in terms of http erlang really is a ghetto. the supplied client > and server modules fail at the slightest traffic blow Again there's a lot more available than what's in the OTP distribution. > - there are sooo soo many things that you could improve about the > standard library its actually kind of sad. Send patches and become happy. > * OTP Releases: for most of the use cases other than big telcos or > banks this is far more confusing then helpful especially when > beginning erlang. While I'm sure there are valid use cases for it > most of the time I thought it was bloated, cumbersome, slow and error > prone. rebar makes them really easy. They're much easier to build and deploy than custom solutions. > * No built in support / standard way for installing libraries - instead > people have to find out about rebar themselves. Look at > http://rubygems.org in contrast or > https://www.ruby-toolbox.com/categories That's because the common way of doing things doesn't go through library installation. Instead you get the source and build a release out of it. I overheard things about rebar getting in the OTP distribution, though. The only thing missing is a central repository for listing available application and libraries. > * Lack of namespaces (duck and cover) - Actually I'd be more than happy > with a ruby / python way of doing it but I guess that won't happen Hopefully it'll never happen. > * I also think the supplied test libraries are not that great to get > started or to motivate anybody to test from day one Are you kidding? Erlang has the most wonderful test libraries ever. Want to test that something(123) returns true? Include the eunit file and write: something_test() -> true = something(123). It can't get any easier. And I'm not even talking about the greatness of ct and proper... -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From dmercer@REDACTED Tue Aug 28 16:28:36 2012 From: dmercer@REDACTED (David Mercer) Date: Tue, 28 Aug 2012 09:28:36 -0500 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503CCEF7.4080405@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> <503CC27E.5070404@berlin.ccc.de> <503CCEF7.4080405@ninenines.eu> Message-ID: <001201cd8529$69874b00$3c95e100$@com> On Tuesday, August 28, 2012, Lo?c Hoguin wrote: > rebar makes them really easy. They're much easier to build and deploy > than custom solutions. Rebar doesn?t work on all the platforms Erlang works on, so it is not a general solution. DBM From zabrane3@REDACTED Tue Aug 28 16:34:07 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 28 Aug 2012 16:34:07 +0200 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <001201cd8529$69874b00$3c95e100$@com> References: <503BD9DE.2080206@ninenines.eu> <503CC27E.5070404@berlin.ccc.de> <503CCEF7.4080405@ninenines.eu> <001201cd8529$69874b00$3c95e100$@com> Message-ID: <571C776A-1FD9-40F2-B352-06CA77A106F2@gmail.com> On Aug 28, 2012, at 4:28 PM, David Mercer wrote: > On Tuesday, August 28, 2012, Lo?c Hoguin wrote: > >> rebar makes them really easy. They're much easier to build and deploy >> than custom solutions. > > Rebar doesn?t work on all the platforms Erlang works on, so it is not a > general solution. But a correctly written Makefile does (even on Windows). Regards, Zabrane From ulf.leopold@REDACTED Tue Aug 28 16:36:57 2012 From: ulf.leopold@REDACTED (Ulf Leopold) Date: Tue, 28 Aug 2012 16:36:57 +0200 Subject: [erlang-questions] "Empty" appups on the form {"", [], []} in OTP In-Reply-To: References: Message-ID: Hi Siri, Thanks for your answer! I figured that was the case. I did notice more empty appups in R14. In R15, Crypto and Compiler are two applications that we use that have empty appups. Thanks, Ulf -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmercer@REDACTED Tue Aug 28 16:40:01 2012 From: dmercer@REDACTED (David Mercer) Date: Tue, 28 Aug 2012 09:40:01 -0500 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <571C776A-1FD9-40F2-B352-06CA77A106F2@gmail.com> References: <503BD9DE.2080206@ninenines.eu> <503CC27E.5070404@berlin.ccc.de> <503CCEF7.4080405@ninenines.eu> <001201cd8529$69874b00$3c95e100$@com> <571C776A-1FD9-40F2-B352-06CA77A106F2@gmail.com> Message-ID: <005801cd852b$01e31680$05a94380$@com> On Tuesday, August 28, 2012, Zabrane Mickael wrote: > > Rebar doesn?t work on all the platforms Erlang works on, so it is not > a > > general solution. > > But a correctly written Makefile does (even on Windows). Yep, that's what I use, for exactly that reason. Except that Makefile's have different syntax due to different directory delimiters, unless you compile under Cygwin. I forget why I can't use Erlang's emake. Anyone use that successfully? Anyone know why I don't? Cheers, DBM From zabrane3@REDACTED Tue Aug 28 16:40:12 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 28 Aug 2012 16:40:12 +0200 Subject: [erlang-questions] [ANN] erserve Erlang/R (Rserve) connector - 3.0.0 In-Reply-To: References: Message-ID: <659E42BD-9812-461D-990D-E3665F8398A3@gmail.com> Hi Daniel, Extremely useful. Thanks for sharing ... and thanks to Klarna too! Regards, Zabrane On Aug 28, 2012, at 1:15 PM, Daniel Eliasson wrote: > Dear all, > > I announce to you the existence of erserve 3.0.0. It is available at > https://github.com/del/erserve and a connection pool library for it at > https://github.com/del/erserve_pool > > What is it? > === > erserve is an Erlang application that lets you easily connect to an R > instance via Rserve ( http://rforge.net/Rserve/ ) over a binary > protocol. > You can send strings to R to be evaluated, and also upload data to > variables in R via binary transport. > > Who is it for? > === > erserve should come in handy if you want to perform some kind of > calculations that are tedious/difficult in Erlang, for instance > running data through machine learning models. It can also shorten your > deployment times by allowing you to model things in R, and then just > use your results from Erlang, instead of porting them there. > > Is it fast? > === > It's not bad. With 10 workers and a pool of 10 erserve connections, my > workstation (Core i7 2.93 Ghz, 16 GB RAM) could run 10 000 individual > classifications through a random forest in 8 seconds. That's about > 1250 calls per second in throughput, with each invidivual call taking > about 3 ms round-trip. > > === > If you have any questions, please let me know. If anyone has use for > this, I would greatly appreciate feedback on improvements. > > I'd also like to thank my employer, Klarna AB, for letting me do some > of the work on this open source software during work hours. > > Best, > Daniel Eliasson > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Tue Aug 28 16:41:55 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 28 Aug 2012 16:41:55 +0200 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <005801cd852b$01e31680$05a94380$@com> References: <503BD9DE.2080206@ninenines.eu> <503CC27E.5070404@berlin.ccc.de> <503CCEF7.4080405@ninenines.eu> <001201cd8529$69874b00$3c95e100$@com> <571C776A-1FD9-40F2-B352-06CA77A106F2@gmail.com> <005801cd852b$01e31680$05a94380$@com> Message-ID: >> But a correctly written Makefile does (even on Windows). > > Yep, that's what I use, for exactly that reason. Except that Makefile's > have different syntax due to different directory delimiters, unless you > compile under Cygwin. I forget why I can't use Erlang's emake. Anyone use > that successfully? Anyone know why I don't? We're using Makefile + Emakefile with "mingw" without any problem. Regards, Zabrane From essen@REDACTED Tue Aug 28 17:08:19 2012 From: essen@REDACTED (=?windows-1252?Q?Lo=EFc_Hoguin?=) Date: Tue, 28 Aug 2012 17:08:19 +0200 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <001201cd8529$69874b00$3c95e100$@com> References: <503BD9DE.2080206@ninenines.eu> <503CC27E.5070404@berlin.ccc.de> <503CCEF7.4080405@ninenines.eu> <001201cd8529$69874b00$3c95e100$@com> Message-ID: <503CDEE3.5050902@ninenines.eu> On 08/28/2012 04:28 PM, David Mercer wrote: > On Tuesday, August 28, 2012, Lo?c Hoguin wrote: > >> rebar makes them really easy. They're much easier to build and deploy >> than custom solutions. > > Rebar doesn?t work on all the platforms Erlang works on, so it is not a > general solution. You can still use rebar to generate all the files and then tweak as needed. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From daniel.goertzen@REDACTED Tue Aug 28 17:23:28 2012 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Tue, 28 Aug 2012 10:23:28 -0500 Subject: [erlang-questions] ssh synchronous send_eof()? Message-ID: I am trying to send a large binary (a couple MB) followed by an eof through an ssh channel using the following: ssh_connection:send(Conn, Chan, BigBinary), ssh_connection:send_eof(Conn, Chan), send() returns immediately and the ssh processes will keep transmitting ssh packets until done. Great. send_eof() causes the eof message to be sent *asynchronously* without waiting for the above data to finish.. only about 87kB gets through before the remote system gets the eof and stops listening. Not great. Is there any way to do a synchronous send_eof() or send()? Thanks, Dan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Tue Aug 28 18:01:54 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Tue, 28 Aug 2012 18:01:54 +0200 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503BD9DE.2080206@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> Message-ID: <503CEB72.7060701@gmail.com> http://musings-of-an-erlang-priest.blogspot.com/2012/06/why-love-erlang.html Cheers, ___ /orben On 2012-08-27 22:34, Lo?c Hoguin wrote: > Hello, > > I am wondering what you guys like the most about Erlang/OTP, > especially newcomers, maybe it changed your life, allowed you to climb > the Everest (or at least sleep at night). > > I'll be synthesizing that into a few key points that I am hopeful will > be reusable by anyone wanting to sell Erlang to their > friends/work/clients. > > So help me by telling us what's so good about it! > > Thanks. > -- http://www.linkedin.com/in/torbenhoffmann From g9414002.pccu.edu.tw@REDACTED Tue Aug 28 18:50:02 2012 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Wed, 29 Aug 2012 00:50:02 +0800 Subject: [erlang-questions] Why does Erlang have control structures? In-Reply-To: References: Message-ID: On Tue, Aug 28, 2012 at 4:56 AM, Jayson Barley wrote: > I am not sure I understand why we have them. For instance I can take the > following code > > is_greater_than(X, Y) -> > if > X>Y -> > true; > true -> % works as an 'else' branch > false > end. > > And make it > > is_true(true) -> > true; > is_true(false) -> > false. > > is_greater_than(X, Y) -> > is_true(X>Y). > > Your is_true/1 means that it may meet either true, false, or other values. However, 'if' statement means that it would meet two cases definitely. is_true/1 is an identity function (fun(X) -> X end) working on Boolean values. Then it may be is_true(X>Y) or is_true(is_true(X>Y)) or is_true(is_true(is_true(is_true(is_true(is_true(X>Y)))))). Eventually, It is (X > Y). > I can do the same thing with case statements > > is_valid_signal(Signal) -> > case Signal of > {signal, _What, _From, _To} -> > true; > {signal, _What, _To} -> > true; > _Else -> > false > end. > > Becomes > > switch_signal({signal, _What, _From, _To}) -> > true; > switch_signal({signal, _What, _To}) -> > true; > switch_signal(_Else) -> > false. > > is_valid_signal(Signal) -> > switch_signal(Signal). > > is_valid_signal(Signal) -> case Signal of {signal, _What, _From, _To} -> true; {signal, _What, _To} -> true; _Else -> false end. The above is an alternative version of is_valid_signal({signal, _What, _From, _To}) -> true; is_valid_signal({signal, _What, _To}) -> true; s_valid_signal(_Else) -> false. Don't repeat it. > I know that the control structures are a little bit faster, not much, but > I find that the function form is more readable. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > In my opinion, 'if' block limits the domain, and 'case' block provides a way more writable than to write many function heads to pick its matching patterns. -- Best Regards. --- Y-H. H. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kilgoretrout62@REDACTED Tue Aug 28 22:12:18 2012 From: kilgoretrout62@REDACTED (kilgore trout) Date: Tue, 28 Aug 2012 21:12:18 +0100 Subject: [erlang-questions] Graphing tools In-Reply-To: <253B92DD-70DE-4EA8-B7B2-E36811966BF2@cs.otago.ac.nz> References: <253B92DD-70DE-4EA8-B7B2-E36811966BF2@cs.otago.ac.nz> Message-ID: Hello All, Thanks for all the replies, it seems there's a lot more options than I first thought. And there's an Erlang/R bridge! How come I didn't think of looking it up earlier: https://r-forge.r-project.org/projects/rserlang/ This presents the perfect excuse to polish up my basic R skills! Richard, that's a "no" on sharing the raw data unfortunately, the powers that be aren't into the sharing / open-source philosophy, at least not yet anyway :-( //KT. On Tue, Aug 28, 2012 at 1:26 AM, Richard O'Keefe wrote: > > On 28/08/2012, at 11:42 AM, Stu Bailey wrote: > > > There are several nice javascript data visualization tools here: > > The reason that I recommended R is that there is a *vast* amount > of stuff already available free for/in R to do all sorts of data > mining and analysis stuff. There is also an R mailing list that > has some very smart, informed, and helpful people on it, and it > is even more active than the Erlang mailing list. If it is just > about pretty pictures, javascript is cool, but so is rrd. > If it's anything more than that, you need at least something like > Matlab or R or Octave or ... > > Just out of curiosity, could the raw data be made available? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladimir.kirichenko@REDACTED Wed Aug 29 01:04:36 2012 From: vladimir.kirichenko@REDACTED (Volodymyr Kyrychenko) Date: Wed, 29 Aug 2012 02:04:36 +0300 Subject: [erlang-questions] Why does Erlang have control structures? In-Reply-To: References: Message-ID: <503D4E84.4080508@gmail.com> Jayson Barley wrote: > I am not sure I understand why we have them. For instance I can take the > following code > > is_greater_than(X, Y) -> > if > X>Y -> > true; > true -> % works as an 'else' branch > false > end. > > And make it > > is_true(true) -> > true; > is_true(false) -> > false. > > is_greater_than(X, Y) -> > is_true(X>Y). Because erlang has no call-by-name/need. What you're proposing exists in Smalltalk. x ifTrue: [ code ] In erlang for this to work it should be like: if_(X>Y, fun() -> do something end, fun() -> this_is_else_for_something_to_return_if_not end). For this to work without having to wrap everything into funs there should be lazy evaluation order in the language. -- Volodymyr Kyrychenko -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From toby@REDACTED Wed Aug 29 01:36:14 2012 From: toby@REDACTED (Toby Thain) Date: Tue, 28 Aug 2012 19:36:14 -0400 Subject: [erlang-questions] Why does Erlang have control structures? In-Reply-To: <503D4E84.4080508@gmail.com> References: <503D4E84.4080508@gmail.com> Message-ID: <503D55EE.3090207@telegraphics.com.au> On 28/08/12 7:04 PM, Volodymyr Kyrychenko wrote: > Jayson Barley wrote: >> I am not sure I understand why we have them. For instance I can take the >> following code >> >> is_greater_than(X, Y) -> >> if >> X>Y -> >> true; >> true -> % works as an 'else' branch >> false >> end. >> >> And make it >> >> is_true(true) -> >> true; >> is_true(false) -> >> false. >> >> is_greater_than(X, Y) -> >> is_true(X>Y). > > Because erlang has no call-by-name/need. > > What you're proposing exists in Smalltalk. > > x ifTrue: [ code ] This lazy block passing also exists in Scala, which enables various forms of ad-hoc control structures like: spawn { // code block to run in another thread } or val f = future { /* some computation wanted later */ } Other Scala features of interest to Erlangers are immutable bindings, pattern matching, and of course actors. --Toby > > In erlang for this to work it should be like: > > if_(X>Y, fun() -> do something end, fun() -> > this_is_else_for_something_to_return_if_not end). > > For this to work without having to wrap everything into funs there > should be lazy evaluation order in the language. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From steven.charles.davis@REDACTED Wed Aug 29 01:53:01 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 28 Aug 2012 16:53:01 -0700 (PDT) Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503BD9DE.2080206@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> Message-ID: <48c3a01a-abfe-4253-ac25-6a3e2f3e12e0@googlegroups.com> Cheap process spawning (which is transformative) Pattern matching (especially with binary pattern matching) On Monday, August 27, 2012 3:35:00 PM UTC-5, Lo?c Hoguin wrote: > > Hello, > > I am wondering what you guys like the most about Erlang/OTP, especially > newcomers, maybe it changed your life, allowed you to climb the Everest > (or at least sleep at night). > > I'll be synthesizing that into a few key points that I am hopeful will > be reusable by anyone wanting to sell Erlang to their > friends/work/clients. > > So help me by telling us what's so good about it! > > Thanks. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed Aug 29 03:55:07 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 28 Aug 2012 21:55:07 -0400 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503BD9DE.2080206@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> Message-ID: <503D767B.4010107@ferd.ca> That's a tough one, because I want to avoid the general "processes! they good!" 1. Understandability I don't mean 'readability'. I strongly believed the two concepts are somewhat exclusive. Poetry can be beautiful and fun to read, while not being able to understand the underlying meaning. Erlang, through OTP, allows to break down application in quickly understandable parts. I can pick many OTP projects (given they're broken in OTP apps), check at the dependencies, and figure out what is the top level, what is lower-level, and break all the fragments individually into something manageable, that I can hold in my head. This does mean easier maintainability, because pretty much all aspects of maintenance have 'understanding the code base' as a prerequisite. The easier things are to figure out, the better I am. 2. Growing a System Through Fault tolerance and OTP apps allows me to keep a system functional even when imperfect or buggy. This is great because none of the software I write will be bug free, nor satisfactory to the customer in the first few iterations. Being able to do hot code loading, shut down or restart only part of a node (due to OTP apps) during a deployment, inspect it while it runs, connect to various nodes, etc. is all awesome. And everything works with or without GUIs so I can check anything even from my smartphones, away from home, if things go wrong. Overall I see it as a very good environment from a devops perspective. On 12-08-27 4:34 PM, Lo?c Hoguin wrote: > Hello, > > I am wondering what you guys like the most about Erlang/OTP, > especially newcomers, maybe it changed your life, allowed you to climb > the Everest (or at least sleep at night). > > I'll be synthesizing that into a few key points that I am hopeful will > be reusable by anyone wanting to sell Erlang to their > friends/work/clients. > > So help me by telling us what's so good about it! > > Thanks. > From dmkolesnikov@REDACTED Wed Aug 29 05:40:14 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Wed, 29 Aug 2012 06:40:14 +0300 Subject: [erlang-questions] Why does Erlang have control structures? In-Reply-To: <503D55EE.3090207@telegraphics.com.au> References: <503D4E84.4080508@gmail.com> <503D55EE.3090207@telegraphics.com.au> Message-ID: <4584148248032837026@unknownmsgid> Hello, Did I missed something but given Scala constructions has equivalents in Erlang? See below Regards, Dmitry On 29.8.2012, at 2.36, Toby Thain wrote: > On 28/08/12 7:04 PM, Volodymyr Kyrychenko wrote: >> Jayson Barley wrote: >>> I am not sure I understand why we have them. For instance I can take the >>> following code >>> >>> is_greater_than(X, Y) -> >>> if >>> X>Y -> >>> true; >>> true -> % works as an 'else' branch >>> false >>> end. >>> >>> And make it >>> >>> is_true(true) -> >>> true; >>> is_true(false) -> >>> false. >>> >>> is_greater_than(X, Y) -> >>> is_true(X>Y). >> >> Because erlang has no call-by-name/need. >> >> What you're proposing exists in Smalltalk. >> >> x ifTrue: [ code ] > > This lazy block passing also exists in Scala, which enables various forms of ad-hoc control structures like: > > spawn { > // code block to run in another thread > } > In Erlang spawn( fun()-> // code block to run end ) > or > > val f = future { /* some computation wanted later */ } > F = fun() -> // some computation end > Other Scala features of interest to Erlangers are immutable bindings, pattern matching, and of course actors. > > --Toby > >> >> In erlang for this to work it should be like: >> >> if_(X>Y, fun() -> do something end, fun() -> >> this_is_else_for_something_to_return_if_not end). >> >> For this to work without having to wrap everything into funs there >> should be lazy evaluation order in the language. >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Wed Aug 29 05:56:16 2012 From: g@REDACTED (Garrett Smith) Date: Tue, 28 Aug 2012 22:56:16 -0500 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503BD9DE.2080206@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> Message-ID: On Mon, Aug 27, 2012 at 3:34 PM, Lo?c Hoguin wrote: > Hello, > > I am wondering what you guys like the most about Erlang/OTP, especially > newcomers, maybe it changed your life, allowed you to climb the Everest (or > at least sleep at night). > > I'll be synthesizing that into a few key points that I am hopeful will be > reusable by anyone wanting to sell Erlang to their friends/work/clients. You know of course this is a topic of great interest for me :) I believe that Erlang's key differentiator, which people should look closely at is this: - Fine grained process isolation The motivation for the design was to build systems that can survive faults. As a pragmatic functional language, Erlang lets you build such systems cheaply. So you get systems that run effectively with less code, less testing, and less maintenance. I'd be curious how many people, based on their experience building production Erlang systems, would agree or disagree with this. Garrett From g@REDACTED Wed Aug 29 06:16:53 2012 From: g@REDACTED (Garrett Smith) Date: Tue, 28 Aug 2012 23:16:53 -0500 Subject: [erlang-questions] How to use dbg and fprof to profile a running server ? In-Reply-To: References: Message-ID: On Tue, Aug 28, 2012 at 4:17 AM, Bo Chen wrote: > Hello, every one, > I am a rookie in erlang. Recently I am trying to use the dbg and > fprof module to profile a running server to observe the performance. Since > the server is based on mochiweb, a quite popular web server framework, and I > only wish to see the performance of my own code, I tried the dbg module. I > have tried in the following two ways: > > 1: use dbg to write trace to file and use fprof to analyse the result > dbg:stop_clear(), > dbg:tracer(port,dbg:trace_port(file,"fprof.trace")), > dbg:tpl(util,[]), > dbg:tpl(...), > dbg:p(all,[call,return_to, procs, timestamp,running,garbage_collection]), > dbg:stop_clear(), > fprof:profile(), > Fname = lists:append("profile/",Name), > fprof:analyse({dest, Fname}). > > 2: like fprof, but use dbg:tpl to confine the module > fprof:trace([start, {procs, all}]), > dbg:tpl(util,[]), > dbg:tpl(...), > fprof:trace([stop]), > fprof:profile(), > Fname = lists:append("profile/",Name), > fprof:analyse({dest, Fname}). > > Sadly, neither way seems to work. > > So any idea about how to combine the dbg and fprof module? > Or maybe someone would like to point out the mistake I have made? > > Thanks in advance for your reply~ e2 has a simple tracing helper that you might take a look at: https://github.com/gar1t/e2 In a shell, you can run e2_debug:trace_module(some_module) or e2_debug:trace_function(some_module, some_function) and you'll get simple output printed to the console. Here's the module: https://github.com/gar1t/e2/blob/master/src/e2_debug.erl It's by no means as flexible as using dbg directly, but it's very easy to use. Garrett From vances@REDACTED Wed Aug 29 06:26:16 2012 From: vances@REDACTED (Vance Shipley) Date: Wed, 29 Aug 2012 09:56:16 +0530 Subject: [erlang-questions] Mnesia: stopped db nodes Message-ID: <20120829042616.GA334@aluminum.wavenet.lk> Is there really no way to coax mnesia into reconnecting other than to restart the application? I get that the decisions can only be made at the application level but if I have a read only node which was momentarily disconnected do I really have to stop and start mnesia to get back online? I'd really like to just tell it it's OK, everythings fine. -- -Vance From ok@REDACTED Wed Aug 29 06:56:20 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 29 Aug 2012 16:56:20 +1200 Subject: [erlang-questions] Why does Erlang have control structures? In-Reply-To: <4584148248032837026@unknownmsgid> References: <503D4E84.4080508@gmail.com> <503D55EE.3090207@telegraphics.com.au> <4584148248032837026@unknownmsgid> Message-ID: <34A9F05C-CE58-440D-B82C-EED775D5CA8F@cs.otago.ac.nz> On 29/08/2012, at 3:40 PM, dmitry kolesnikov wrote: > Did I missed something but given Scala constructions has equivalents > in Erlang? See below You missed two things: (a) brevity and (b) a future is not the same thing as a closure -- it gets evaluated just once. One of the core strengths of Erlang is the things it *cannot* do. From moxford@REDACTED Wed Aug 29 07:13:46 2012 From: moxford@REDACTED (Mike Oxford) Date: Tue, 28 Aug 2012 22:13:46 -0700 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: References: <503BD9DE.2080206@ninenines.eu> Message-ID: On Tue, Aug 28, 2012 at 8:56 PM, Garrett Smith wrote: > So you get systems that run effectively with less code, less testing, > and less maintenance. > > I'd be curious how many people, based on their experience building > production Erlang systems, would agree or disagree with this. > I believe you're treading dangerous territory here. :-) There is less custom code...but such is the case with any framework. There is less testing ONLY due to less volume of code, which is the case with any framework. There is less maintenance due to ... you guessed it, less volume of code. So, really, you're looking at "Framework vs Non-Framework." What "sold me" on erlang originally was: 1) Distributed was "easy." 2) Shared-state was built-in 3) No downtime code deployments 4) Supervisors The reality: 1) Distributed is easy, kinda. Easier. Getting a message to another node is trivial, which is what Erlang does and from then on it's application-level code (eg, you still have to write it and do work. :) 2) Shared state is done via mnesia. House of cards, IMO, having run it. It's just too fragile as-is. One node burps and your whole database goes into a corner, sucking its thumb until you fix it. Ulf's unsplit is a good start, but given current-state I'd have to say it's too risky to run in production for anything without a back-plane network. Split-brain is hard -- may be that wedging Riak in would work here (Ulf's unsplit uses vector-clocks, inspired by Riak.) It would be nice if mnesia at least said "node dead is dead, let the rest run even if the first comes back." We had a 3-node cluster and one node would burp and all three nodes would fail to run. Setting master-node works but then you start going down the "not fully distributed" route. Majority helps ... until a node comes back. If you have 30 nodes and one goes down and comes back...both have node-down states and your whole set of 29 runs to the corner.... Majority + Unsplit + Master is about the best you can get, until your partition falls along Master-lines and when you come back 20+ nodes are all doing copies off the master to sync up. 8-) I really like mnesia...really really, and I really really tried to shoehorn it in. Maybe I'm doing something wrong, but I see similar issues with RabbitMQ's use of mnesia so I don't think I'm too far off. I would love to be shown incorrect here....please? Anyone? 3) These are hard. Doable, but you end up with a LOT of testing of the upgrade/downgrade paths. You start questioning the value vs "rolling restarts." It's a golden-hammer when you need it, though you have to really look at the value and "cost" of using such a weapon regularly (testing time vs just rolling restarts.) 4) #winning =:= #winning Love 'em. That said, Erlang has become my personal "gold standard" for other languages, and by which I judge all others. :) -mox -------------- next part -------------- An HTML attachment was scrubbed... URL: From moxford@REDACTED Wed Aug 29 07:19:22 2012 From: moxford@REDACTED (Mike Oxford) Date: Tue, 28 Aug 2012 22:19:22 -0700 Subject: [erlang-questions] Mnesia: stopped db nodes In-Reply-To: <20120829042616.GA334@aluminum.wavenet.lk> References: <20120829042616.GA334@aluminum.wavenet.lk> Message-ID: Look into Ulf's unsplit https://github.com/uwiger/unsplit Best you can do is run a Master + Majority + unsplit, I believe. Or use something like Riak instead (note: have to write sibling resolvers if you allow_mult=true) but at least your whole setup isn't frozen when a node comes back. :) G'luck! -mox On Tue, Aug 28, 2012 at 9:26 PM, Vance Shipley wrote: > Is there really no way to coax mnesia into reconnecting other than to > restart the application? I get that the decisions can only be made at > the application level but if I have a read only node which was momentarily > disconnected do I really have to stop and start mnesia to get back online? > I'd really like to just tell it it's OK, everythings fine. > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zerthurd@REDACTED Wed Aug 29 07:42:59 2012 From: zerthurd@REDACTED (Maxim Treskin) Date: Wed, 29 Aug 2012 12:42:59 +0700 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: References: <503BD9DE.2080206@ninenines.eu> Message-ID: 1. Hot code update 2. WYSIWYG code (? Lev Walkin) ? you get exactly desired behaviour of your code without invasions from dark side 3. Dialyzer ? first you don't care about types, then you do; it really helpful 4. Code coverage integrated with test frameworks (eunit, common test) 5. Rich processes tracing: messages, function calls with templates, process related events 6. Fprof, Eprof 7. Supervisors tree ? the best way to organize your code parts in project and runtime system On 29 August 2012 12:13, Mike Oxford wrote: > On Tue, Aug 28, 2012 at 8:56 PM, Garrett Smith wrote: > >> So you get systems that run effectively with less code, less testing, >> and less maintenance. >> >> I'd be curious how many people, based on their experience building >> production Erlang systems, would agree or disagree with this. >> > > I believe you're treading dangerous territory here. :-) > > There is less custom code...but such is the case with any framework. > There is less testing ONLY due to less volume of code, which is the case > with any framework. > There is less maintenance due to ... you guessed it, less volume of code. > > So, really, you're looking at "Framework vs Non-Framework." > > What "sold me" on erlang originally was: > 1) Distributed was "easy." > 2) Shared-state was built-in > 3) No downtime code deployments > 4) Supervisors > > The reality: > 1) Distributed is easy, kinda. Easier. Getting a message to another > node is trivial, which is what Erlang does and from then on it's > application-level code (eg, you still have to write it and do work. :) > > 2) Shared state is done via mnesia. House of cards, IMO, having run it. > It's just too fragile as-is. One node burps and your whole database goes > into a corner, sucking its thumb until you fix it. Ulf's unsplit is a good > start, but given current-state I'd have to say it's too risky to run in > production for anything without a back-plane network. Split-brain is hard > -- may be that wedging Riak in would work here (Ulf's unsplit uses > vector-clocks, inspired by Riak.) It would be nice if mnesia at least said > "node dead is dead, let the rest run even if the first comes back." We had > a 3-node cluster and one node would burp and all three nodes would fail to > run. Setting master-node works but then you start going down the "not > fully distributed" route. Majority helps ... until a node comes back. If > you have 30 nodes and one goes down and comes back...both have node-down > states and your whole set of 29 runs to the corner.... Majority + Unsplit > + Master is about the best you can get, until your partition falls along > Master-lines and when you come back 20+ nodes are all doing copies off the > master to sync up. 8-) I really like mnesia...really really, and I really > really tried to shoehorn it in. Maybe I'm doing something wrong, but I see > similar issues with RabbitMQ's use of mnesia so I don't think I'm too far > off. I would love to be shown incorrect here....please? Anyone? > > 3) These are hard. Doable, but you end up with a LOT of testing of the > upgrade/downgrade paths. You start questioning the value vs "rolling > restarts." It's a golden-hammer when you need it, though you have to > really look at the value and "cost" of using such a weapon regularly > (testing time vs just rolling restarts.) > > 4) #winning =:= #winning Love 'em. > > That said, Erlang has become my personal "gold standard" for other > languages, and by which I judge all others. :) > > -mox > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Max Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Aug 29 09:36:43 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 29 Aug 2012 09:36:43 +0200 Subject: [erlang-questions] Fail politely, fail noisily Message-ID: As I was revising my Erlang book I came up with a new slogan (like the old "let it crash") ... I like slogans they are easy to remember - here's a new one. Fail politely, fail noisily means the user should get a polite error message and the programmer should get an informative error message when an application crashes. There should be TWO error messages not one. I want to collect examples of polite and noisy error messages for a future lecture. Today I got this error message - which I consider not just noisy but bordering upon downright rude - all I did was click on a button - I didn't want to unleash the wrath of the java monster - I was required to tell the system how many hours I had worked but the java monster spake thus, and there was fear and trembling in the land and all that read what was writ were sore afraid: --- com.sap.mw.jco.JCO$Exception: (103) RFC_ERROR_LOGON_FAILURE: System received an expired SSO ticket at com.sap.mw.jco.MiddlewareJRfc.generateJCoException(MiddlewareJRfc.java:555) at com.sap.mw.jco.MiddlewareJRfc$Client.connect(MiddlewareJRfc.java:1099) at com.sap.mw.jco.JCO$Client.connect(JCO.java:3644) at com.sap.mw.jco.JCO$Pool.getClient(JCO.java:5904) at com.sap.mw.jco.JCO$PoolManager.getClient(JCO.java:7081) at com.sap.mw.jco.JCO$PoolManager.getClient(JCO.java:7030) at com.sap.tc.webdynpro.serverimpl.core.sl.AbstractJCOClientConnection.getClient(AbstractJCOClientConnection.java:444) at com.sap.pcuigp.xssfpm.wd.BackendConnections.connectModelInternal(BackendConnections.java:326) at com.sap.pcuigp.xssfpm.wd.BackendConnections.initBackend(BackendConnections.java:266) at com.sap.pcuigp.xssfpm.wd.BackendConnections.connectModel(BackendConnections.java:167) at com.sap.pcuigp.xssfpm.wd.wdp.InternalBackendConnections.connectModel(InternalBackendConnections.java:229) at com.sap.pcuigp.xssfpm.wd.FPMComponent$FPM.connectModel(FPMComponent.java:864) at com.sap.pcuigp.xssfpm.wd.FPMComponent$FPMProxy.connectModel(FPMComponent.java:1094) at com.sap.pcuigp.xssfpm.wd.BackendConnections.init(BackendConnections.java:155) at com.sap.pcuigp.xssfpm.wd.wdp.InternalBackendConnections.init(InternalBackendConnections.java:225) at com.sap.pcuigp.xssfpm.wd.FPMComponent.wdDoInit(FPMComponent.java:204) at com.sap.pcuigp.xssfpm.wd.wdp.InternalFPMComponent.wdDoInit(InternalFPMComponent.java:105) at com.sap.tc.webdynpro.progmodel.generation.DelegatingComponent.doInit(DelegatingComponent.java:161) at com.sap.tc.webdynpro.progmodel.controller.Controller.initController(Controller.java:227) at com.sap.tc.webdynpro.progmodel.components.Component.initController(Component.java:258) at com.sap.tc.webdynpro.progmodel.controller.Controller.init(Controller.java:206) at com.sap.tc.webdynpro.clientserver.cal.ClientApplication.init(ClientApplication.java:590) at com.sap.tc.webdynpro.clientserver.cal.ClientApplication.doPreprocessing(ClientApplication.java:1457) at com.sap.tc.webdynpro.clientserver.session.ApplicationSession.doPreprocessing(ApplicationSession.java:660) at com.sap.tc.webdynpro.clientserver.session.ApplicationSession.doProcessing(ApplicationSession.java:349) at com.sap.tc.webdynpro.clientserver.session.RequestManager.doProcessing(RequestManager.java:326) at com.sap.tc.webdynpro.serverimpl.core.sessionctx.AbstractExecutionContextDispatcher.delegateToRequestManager(AbstractExecutionContextDispatcher.java:62) at com.sap.tc.webdynpro.serverimpl.wdc.sessionctx.DispatchHandlerForRequestManager.service(DispatchHandlerForRequestManager.java:39) at com.sap.tc.webdynpro.serverimpl.wdc.sessionctx.DispatchHandlerForRequestManager.service(DispatchHandlerForRequestManager.java:46) at com.sap.engine.services.servlets_jsp.server.deploy.impl.module.IRequestDispatcherImpl.dispatch(IRequestDispatcherImpl.java:270) at com.sap.tc.webdynpro.serverimpl.wdc.sessionctx.ExecutionContextDispatcher.dispatchToAppContext(ExecutionContextDispatcher.java:68) at com.sap.tc.webdynpro.serverimpl.core.sessionctx.AbstractExecutionContextDispatcher.dispatch(AbstractExecutionContextDispatcher.java:53) at com.sap.tc.webdynpro.clientserver.session.RequestManager.doProcessing(RequestManager.java:245) at com.sap.tc.webdynpro.clientserver.embedding.JavaApplicationProxy$Runner.callRequestManager(JavaApplicationProxy.java:1244) at com.sap.tc.webdynpro.clientserver.embedding.JavaApplicationProxy$Runner.callEmbeddedApplication(JavaApplicationProxy.java:1122) at com.sap.tc.webdynpro.clientserver.embedding.JavaApplicationProxy$StartCommand.doExecute(JavaApplicationProxy.java:1575) at com.sap.tc.webdynpro.clientserver.embedding.JavaApplicationProxy$AbstractCommand.execute(JavaApplicationProxy.java:1488) at com.sap.tc.webdynpro.clientserver.embedding.JavaApplicationProxy$Runner.execute(JavaApplicationProxy.java:1028) at com.sap.tc.webdynpro.clientserver.embedding.JavaApplicationProxy.execute(JavaApplicationProxy.java:859) at com.sap.tc.webdynpro.clientserver.embedding.JavaApplicationProxy.start1(JavaApplicationProxy.java:637) at com.sap.tc.webdynpro.portal.pb.impl.JavaApplicationProxyAdapter.create(JavaApplicationProxyAdapter.java:166) at com.sap.portal.pb.PageBuilder.updateApplications(PageBuilder.java:1691) at com.sap.portal.pb.PageBuilder.createPage(PageBuilder.java:411) at com.sap.portal.pb.PageBuilder.init(PageBuilder.java:655) at com.sap.portal.pb.PageBuilder.wdDoInit(PageBuilder.java:227) at com.sap.portal.pb.wdp.InternalPageBuilder.wdDoInit(InternalPageBuilder.java:137) at com.sap.tc.webdynpro.progmodel.generation.DelegatingComponent.doInit(DelegatingComponent.java:161) at com.sap.tc.webdynpro.progmodel.controller.Controller.initController(Controller.java:227) at com.sap.tc.webdynpro.progmodel.components.Component.initController(Component.java:258) at com.sap.tc.webdynpro.progmodel.controller.Controller.init(Controller.java:206) at com.sap.tc.webdynpro.clientserver.cal.ClientApplication.init(ClientApplication.java:590) at com.sap.tc.webdynpro.clientserver.cal.ClientApplication.doPreprocessing(ClientApplication.java:1457) at com.sap.tc.webdynpro.clientserver.session.ApplicationSession.doPreprocessing(ApplicationSession.java:660) at com.sap.tc.webdynpro.clientserver.session.ApplicationSession.doProcessing(ApplicationSession.java:349) at com.sap.tc.webdynpro.clientserver.session.RequestManager.doProcessing(RequestManager.java:326) at com.sap.tc.webdynpro.serverimpl.core.AbstractDispatcherServlet.doContent(AbstractDispatcherServlet.java:87) at com.sap.tc.webdynpro.serverimpl.wdc.DispatcherServlet.doContent(DispatcherServlet.java:89) at com.sap.tc.webdynpro.serverimpl.core.AbstractDispatcherServlet.doPost(AbstractDispatcherServlet.java:62) at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) at com.sap.engine.services.servlets_jsp.server.Invokable.invoke(Invokable.java:152) at com.sap.engine.services.servlets_jsp.server.Invokable.invoke(Invokable.java:38) at com.sap.engine.services.servlets_jsp.server.HttpHandlerImpl.runServlet(HttpHandlerImpl.java:457) at com.sap.engine.services.servlets_jsp.server.HttpHandlerImpl.handleRequest(HttpHandlerImpl.java:210) at com.sap.engine.services.httpserver.server.RequestAnalizer.startServlet(RequestAnalizer.java:441) at com.sap.engine.services.httpserver.server.RequestAnalizer.startServlet(RequestAnalizer.java:430) at com.sap.engine.services.servlets_jsp.filters.DSRWebContainerFilter.process(DSRWebContainerFilter.java:38) at com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) at com.sap.engine.services.servlets_jsp.filters.ServletSelector.process(ServletSelector.java:81) at com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) at com.sap.engine.services.servlets_jsp.filters.ApplicationSelector.process(ApplicationSelector.java:276) at com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) at com.sap.engine.services.httpserver.filters.WebContainerInvoker.process(WebContainerInvoker.java:81) at com.sap.engine.services.httpserver.chain.HostFilter.process(HostFilter.java:9) at com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) at com.sap.engine.services.httpserver.filters.ResponseLogWriter.process(ResponseLogWriter.java:60) at com.sap.engine.services.httpserver.chain.HostFilter.process(HostFilter.java:9) at com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) at com.sap.engine.services.httpserver.filters.DefineHostFilter.process(DefineHostFilter.java:27) at com.sap.engine.services.httpserver.chain.ServerFilter.process(ServerFilter.java:12) at com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) at com.sap.engine.services.httpserver.filters.MonitoringFilter.process(MonitoringFilter.java:29) at com.sap.engine.services.httpserver.chain.ServerFilter.process(ServerFilter.java:12) at com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) at com.sap.engine.services.httpserver.filters.SessionSizeFilter.process(SessionSizeFilter.java:26) at com.sap.engine.services.httpserver.chain.ServerFilter.process(ServerFilter.java:12) at com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) at com.sap.engine.services.httpserver.filters.MemoryStatisticFilter.process(MemoryStatisticFilter.java:57) at com.sap.engine.services.httpserver.chain.ServerFilter.process(ServerFilter.java:12) at com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) at com.sap.engine.services.httpserver.filters.DSRHttpFilter.process(DSRHttpFilter.java:43) at com.sap.engine.services.httpserver.chain.ServerFilter.process(ServerFilter.java:12) at com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) at com.sap.engine.services.httpserver.server.Processor.chainedRequest(Processor.java:475) at com.sap.engine.services.httpserver.server.Processor$FCAProcessorThread.process(Processor.java:269) at com.sap.engine.services.httpserver.server.rcm.RequestProcessorThread.run(RequestProcessorThread.java:56) at com.sap.engine.core.thread.execution.Executable.run(Executable.java:122) at com.sap.engine.core.thread.execution.Executable.run(Executable.java:101) at com.sap.engine.core.thread.execution.CentralExecutor$SingleThread.run(CentralExecutor.java:328) Caused by: RfcException: message: System received an expired SSO ticket Return code: RFC_SYS_EXCEPTION(3) error group: 103 key: RFC_ERROR_LOGON_FAILURE at com.sap.mw.rfc.api.RfcApi.RfcOpen(RfcApi.java:831) at com.sap.mw.jco.MiddlewareJRfc$Client.connect(MiddlewareJRfc.java:1092) ... 97 more -- there was also a one liner System received an Expired SSO ticket (I guess this is the meaning of the longer message) - (I think the old BSD crashed with the epic words "pointers bungled, sorry" and then just immediately halted - but that was from memory - can anybody remember this) /Joe From thomas.elsgaard@REDACTED Wed Aug 29 09:47:01 2012 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Wed, 29 Aug 2012 09:47:01 +0200 Subject: [erlang-questions] Fail politely, fail noisily In-Reply-To: References: Message-ID: Hi Joe There is also PHK's concept of "guru meditation" in the varnish http accelerator.. http://www.version2.dk/blog/guru-meditation-9738 Basically the idea is: the user cannot do anything about the error, so let it fail, but provide the transaction Id (xid) so the admin can can troubleshoot... Thomas onsdag den 29. august 2012 skrev Joe Armstrong : > As I was revising my Erlang book I came up with a new slogan (like the > old "let it crash") ... I like slogans they are > easy to remember - here's a new one. > > Fail politely, fail noisily means the user should get a polite error > message > and the programmer should get an informative error message when an > application crashes. > > There should be TWO error messages not one. > > I want to collect examples of polite and noisy error messages for a > future lecture. > > Today I got this error message - which I consider not just noisy but > bordering upon downright rude - all > I did was click on a button - I didn't want to unleash the wrath of > the java monster - I was required to tell the system > how many hours I had worked but the java monster spake thus, and there > was fear and trembling in the land > and all that read what was writ were sore afraid: > > --- > > com.sap.mw.jco.JCO$Exception: (103) RFC_ERROR_LOGON_FAILURE: System > received an expired SSO ticket > at > com.sap.mw.jco.MiddlewareJRfc.generateJCoException(MiddlewareJRfc.java:555) > at > com.sap.mw.jco.MiddlewareJRfc$Client.connect(MiddlewareJRfc.java:1099) > at com.sap.mw.jco.JCO$Client.connect(JCO.java:3644) > at com.sap.mw.jco.JCO$Pool.getClient(JCO.java:5904) > at com.sap.mw.jco.JCO$PoolManager.getClient(JCO.java:7081) > at com.sap.mw.jco.JCO$PoolManager.getClient(JCO.java:7030) > at > com.sap.tc.webdynpro.serverimpl.core.sl.AbstractJCOClientConnection.getClient(AbstractJCOClientConnection.java:444) > at > com.sap.pcuigp.xssfpm.wd.BackendConnections.connectModelInternal(BackendConnections.java:326) > at > com.sap.pcuigp.xssfpm.wd.BackendConnections.initBackend(BackendConnections.java:266) > at > com.sap.pcuigp.xssfpm.wd.BackendConnections.connectModel(BackendConnections.java:167) > at > com.sap.pcuigp.xssfpm.wd.wdp.InternalBackendConnections.connectModel(InternalBackendConnections.java:229) > at > com.sap.pcuigp.xssfpm.wd.FPMComponent$FPM.connectModel(FPMComponent.java:864) > at > com.sap.pcuigp.xssfpm.wd.FPMComponent$FPMProxy.connectModel(FPMComponent.java:1094) > at > com.sap.pcuigp.xssfpm.wd.BackendConnections.init(BackendConnections.java:155) > at > com.sap.pcuigp.xssfpm.wd.wdp.InternalBackendConnections.init(InternalBackendConnections.java:225) > at > com.sap.pcuigp.xssfpm.wd.FPMComponent.wdDoInit(FPMComponent.java:204) > at > com.sap.pcuigp.xssfpm.wd.wdp.InternalFPMComponent.wdDoInit(InternalFPMComponent.java:105) > at > com.sap.tc.webdynpro.progmodel.generation.DelegatingComponent.doInit(DelegatingComponent.java:161) > at > com.sap.tc.webdynpro.progmodel.controller.Controller.initController(Controller.java:227) > at > com.sap.tc.webdynpro.progmodel.components.Component.initController(Component.java:258) > at > com.sap.tc.webdynpro.progmodel.controller.Controller.init(Controller.java:206) > at > com.sap.tc.webdynpro.clientserver.cal.ClientApplication.init(ClientApplication.java:590) > at > com.sap.tc.webdynpro.clientserver.cal.ClientApplication.doPreprocessing(ClientApplication.java:1457) > at > com.sap.tc.webdynpro.clientserver.session.ApplicationSession.doPreprocessing(ApplicationSession.java:660) > at > com.sap.tc.webdynpro.clientserver.session.ApplicationSession.doProcessing(ApplicationSession.java:349) > at > com.sap.tc.webdynpro.clientserver.session.RequestManager.doProcessing(RequestManager.java:326) > at > com.sap.tc.webdynpro.serverimpl.core.sessionctx.AbstractExecutionContextDispatcher.delegateToRequestManager(AbstractExecutionContextDispatcher.java:62) > at > com.sap.tc.webdynpro.serverimpl.wdc.sessionctx.DispatchHandlerForRequestManager.service(DispatchHandlerForRequestManager.java:39) > at > com.sap.tc.webdynpro.serverimpl.wdc.sessionctx.DispatchHandlerForRequestManager.service(DispatchHandlerForRequestManager.java:46) > at > com.sap.engine.services.servlets_jsp.server.deploy.impl.module.IRequestDispatcherImpl.dispatch(IRequestDispatcherImpl.java:270) > at > com.sap.tc.webdynpro.serverimpl.wdc.sessionctx.ExecutionContextDispatcher.dispatchToAppContext(ExecutionContextDispatcher.java:68) > at > com.sap.tc.webdynpro.serverimpl.core.sessionctx.AbstractExecutionContextDispatcher.dispatch(AbstractExecutionContextDispatcher.java:53) > at > com.sap.tc.webdynpro.clientserver.session.RequestManager.doProcessing(RequestManager.java:245) > at > com.sap.tc.webdynpro.clientserver.embedding.JavaApplicationProxy$Runner.callRequestManager(JavaApplicationProxy.java:1244) > at > com.sap.tc.webdynpro.clientserver.embedding.JavaApplicationProxy$Runner.callEmbeddedApplication(JavaApplicationProxy.java:1122) > at > com.sap.tc.webdynpro.clientserver.embedding.JavaApplicationProxy$StartCommand.doExecute(JavaApplicationProxy.java:1575) > at > com.sap.tc.webdynpro.clientserver.embedding.JavaApplicationProxy$AbstractCommand.execute(JavaApplicationProxy.java:1488) > at > com.sap.tc.webdynpro.clientserver.embedding.JavaApplicationProxy$Runner.execute(JavaApplicationProxy.java:1028) > at > com.sap.tc.webdynpro.clientserver.embedding.JavaApplicationProxy.execute(JavaApplicationProxy.java:859) > at > com.sap.tc.webdynpro.clientserver.embedding.JavaApplicationProxy.start1(JavaApplicationProxy.java:637) > at > com.sap.tc.webdynpro.portal.pb.impl.JavaApplicationProxyAdapter.create(JavaApplicationProxyAdapter.java:166) > at > com.sap.portal.pb.PageBuilder.updateApplications(PageBuilder.java:1691) > at com.sap.portal.pb.PageBuilder.createPage(PageBuilder.java:411) > at com.sap.portal.pb.PageBuilder.init(PageBuilder.java:655) > at com.sap.portal.pb.PageBuilder.wdDoInit(PageBuilder.java:227) > at > com.sap.portal.pb.wdp.InternalPageBuilder.wdDoInit(InternalPageBuilder.java:137) > at > com.sap.tc.webdynpro.progmodel.generation.DelegatingComponent.doInit(DelegatingComponent.java:161) > at > com.sap.tc.webdynpro.progmodel.controller.Controller.initController(Controller.java:227) > at > com.sap.tc.webdynpro.progmodel.components.Component.initController(Component.java:258) > at > com.sap.tc.webdynpro.progmodel.controller.Controller.init(Controller.java:206) > at > com.sap.tc.webdynpro.clientserver.cal.ClientApplication.init(ClientApplication.java:590) > at > com.sap.tc.webdynpro.clientserver.cal.ClientApplication.doPreprocessing(ClientApplication.java:1457) > at > com.sap.tc.webdynpro.clientserver.session.ApplicationSession.doPreprocessing(ApplicationSession.java:660) > at > com.sap.tc.webdynpro.clientserver.session.ApplicationSession.doProcessing(ApplicationSession.java:349) > at > com.sap.tc.webdynpro.clientserver.session.RequestManager.doProcessing(RequestManager.java:326) > at > com.sap.tc.webdynpro.serverimpl.core.AbstractDispatcherServlet.doContent(AbstractDispatcherServlet.java:87) > at > com.sap.tc.webdynpro.serverimpl.wdc.DispatcherServlet.doContent(DispatcherServlet.java:89) > at > com.sap.tc.webdynpro.serverimpl.core.AbstractDispatcherServlet.doPost(AbstractDispatcherServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at > com.sap.engine.services.servlets_jsp.server.Invokable.invoke(Invokable.java:152) > at > com.sap.engine.services.servlets_jsp.server.Invokable.invoke(Invokable.java:38) > at > com.sap.engine.services.servlets_jsp.server.HttpHandlerImpl.runServlet(HttpHandlerImpl.java:457) > at > com.sap.engine.services.servlets_jsp.server.HttpHandlerImpl.handleRequest(HttpHandlerImpl.java:210) > at > com.sap.engine.services.httpserver.server.RequestAnalizer.startServlet(RequestAnalizer.java:441) > at > com.sap.engine.services.httpserver.server.RequestAnalizer.startServlet(RequestAnalizer.java:430) > at > com.sap.engine.services.servlets_jsp.filters.DSRWebContainerFilter.process(DSRWebContainerFilter.java:38) > at > com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) > at > com.sap.engine.services.servlets_jsp.filters.ServletSelector.process(ServletSelector.java:81) > at > com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) > at > com.sap.engine.services.servlets_jsp.filters.ApplicationSelector.process(ApplicationSelector.java:276) > at > com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) > at > com.sap.engine.services.httpserver.filters.WebContainerInvoker.process(WebContainerInvoker.java:81) > at > com.sap.engine.services.httpserver.chain.HostFilter.process(HostFilter.java:9) > at > com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) > at > com.sap.engine.services.httpserver.filters.ResponseLogWriter.process(ResponseLogWriter.java:60) > at > com.sap.engine.services.httpserver.chain.HostFilter.process(HostFilter.java:9) > at > com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) > at > com.sap.engine.services.httpserver.filters.DefineHostFilter.process(DefineHostFilter.java:27) > at > com.sap.engine.services.httpserver.chain.ServerFilter.process(ServerFilter.java:12) > at > com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) > at > com.sap.engine.services.httpserver.filters.MonitoringFilter.process(MonitoringFilter.java:29) > at > com.sap.engine.services.httpserver.chain.ServerFilter.process(ServerFilter.java:12) > at > com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) > at > com.sap.engine.services.httpserver.filters.SessionSizeFilter.process(SessionSizeFilter.java:26) > at > com.sap.engine.services.httpserver.chain.ServerFilter.process(ServerFilter.java:12) > at > com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) > at > com.sap.engine.services.httpserver.filters.MemoryStatisticFilter.process(MemoryStatisticFilter.java:57) > at > com.sap.engine.services.httpserver.chain.ServerFilter.process(ServerFilter.java:12) > at > com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) > at > com.sap.engine.services.httpserver.filters.DSRHttpFilter.process(DSRHttpFilter.java:43) > at > com.sap.engine.services.httpserver.chain.ServerFilter.process(ServerFilter.java:12) > at > com.sap.engine.services.httpserver.chain.AbstractChain.process(AbstractChain.java:78) > at > com.sap.engine.services.httpserver.server.Processor.chainedRequest(Processor.java:475) > at > com.sap.engine.services.httpserver.server.Processor$FCAProcessorThread.process(Processor.java:269) > at > com.sap.engine.services.httpserver.server.rcm.RequestProcessorThread.run(RequestProcessorThread.java:56) > at > com.sap.engine.core.thread.execution.Executable.run(Executable.java:122) > at > com.sap.engine.core.thread.execution.Executable.run(Executable.java:101) > at > com.sap.engine.core.thread.execution.CentralExecutor$SingleThread.run(CentralExecutor.java:328) > Caused by: > RfcException: > message: System received an expired SSO ticket > Return code: RFC_SYS_EXCEPTION(3) > error group: 103 > key: RFC_ERROR_LOGON_FAILURE > at com.sap.mw.rfc.api.RfcApi.RfcOpen(RfcApi.java:831) > at > com.sap.mw.jco.MiddlewareJRfc$Client.connect(MiddlewareJRfc.java:1092) > ... 97 more > > -- there was also a one liner > > System received an Expired SSO ticket > > (I guess this is the meaning of the longer message) > > - > > (I think the old BSD crashed with the epic words "pointers bungled, > sorry" and then just immediately halted - but that was from > memory - can anybody remember this) > > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel@REDACTED Wed Aug 29 10:22:42 2012 From: daniel@REDACTED (Daniel Eliasson) Date: Wed, 29 Aug 2012 10:22:42 +0200 Subject: [erlang-questions] Graphing tools In-Reply-To: References: <253B92DD-70DE-4EA8-B7B2-E36811966BF2@cs.otago.ac.nz> Message-ID: Hi Kilgore, You might also be interested in https://github.com/del/erserve which is a library I wrote to let Erlang call to R. It's being developed actively, and I'd love to get some users and feedback. /Daniel On 28 August 2012 22:12, kilgore trout wrote: > Hello All, > Thanks for all the replies, it seems there's a lot more options than I first > thought. > And there's an Erlang/R bridge! How come I didn't think of looking it up > earlier: > https://r-forge.r-project.org/projects/rserlang/ > This presents the perfect excuse to polish up my basic R skills! > Richard, that's a "no" on sharing the raw data unfortunately, the powers > that be aren't into the sharing / open-source philosophy, at least not yet > anyway :-( > //KT. > > > > On Tue, Aug 28, 2012 at 1:26 AM, Richard O'Keefe wrote: >> >> >> On 28/08/2012, at 11:42 AM, Stu Bailey wrote: >> >> > There are several nice javascript data visualization tools here: >> >> The reason that I recommended R is that there is a *vast* amount >> of stuff already available free for/in R to do all sorts of data >> mining and analysis stuff. There is also an R mailing list that >> has some very smart, informed, and helpful people on it, and it >> is even more active than the Erlang mailing list. If it is just >> about pretty pictures, javascript is cool, but so is rrd. >> If it's anything more than that, you need at least something like >> Matlab or R or Octave or ... >> >> Just out of curiosity, could the raw data be made available? >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From eriksoe@REDACTED Wed Aug 29 11:33:14 2012 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Wed, 29 Aug 2012 11:33:14 +0200 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503BD9DE.2080206@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> Message-ID: One thing I've found useful is the ease of having small embedded DSLs -- data-driven code, also known as. Writing small interpreters can do wonders sometimes, and is easy in Erlang. Here @work we used that for a project were we needed a set of heuristics-based data transformation - "if the data has this form, apply that transformation". And I've also written data-driven unit tests. Erlang's literals are fairly good for this. The generators of QuickCheck and its kin are another example of this. (Erlang's far from unique wrt this, but it's a thing to like nonetheless. Scala and Haskell are stronger at DSLs in some respects, I believe -- but I don't think they have something like file:consult/1.) And, as others have mentioned: good, strong encapsulation, in the form of immutable values and process-level isolation - encourages separation of concerns, increasing the chance of understandable code. 2012/8/27 Lo?c Hoguin > Hello, > > I am wondering what you guys like the most about Erlang/OTP, especially > newcomers, maybe it changed your life, allowed you to climb the Everest (or > at least sleep at night). > > I'll be synthesizing that into a few key points that I am hopeful will be > reusable by anyone wanting to sell Erlang to their friends/work/clients. > > So help me by telling us what's so good about it! > > Thanks. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Aug 29 12:45:55 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 29 Aug 2012 12:45:55 +0200 Subject: [erlang-questions] Mike Williams: Small is beautiful slides? Message-ID: <503DF2E3.6060305@ninenines.eu> Greetings, Looking for the slides to this talk, anyone seen them? http://www.erlang-factory.com/conference/SFBay2012/speakers/MikeWilliams Thanks. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From essen@REDACTED Wed Aug 29 13:24:32 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 29 Aug 2012 13:24:32 +0200 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503BD9DE.2080206@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> Message-ID: <503DFBF0.6040600@ninenines.eu> Thanks to everyone for helping. So far I've been able to put all these things into 6 key categories. This is obviously not fully done but I'd like to hear your thoughts on it. If you have anything to add you're welcome too! Categorization might not be the best either. Write less, better code * Pattern matching * Bit syntax * Simple base language * DSLs * Only worry about success cases (links to crash and fault tolerance) Fault tolerant concurrency by design * Erlang models the real world * Lightweight processes * Fine-grained process isolation * Shared nothing messaging, messages processed in the order we want * Crash early, crash often (catch errors where they happen) A better OS for your applications * OTP * A system as a collection of small separate components * Understandability and maintainability * Upgrade without the need for restarting * Numerous ways to inspect, debug and trace running nodes * Combine many stacks without interference Distributed * Easy to setup, easy to manage * Same code for talking to a local or a remote process Community * Nice, welcoming people * Modest people, no drama queen * Experts willing to share their knowledge Mature * Code unlikely to break in future versions * Sound design principles for building robust systems (OTP) * Numerous testing tools * Property-based testing -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From max.lapshin@REDACTED Wed Aug 29 13:35:21 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 29 Aug 2012 15:35:21 +0400 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503DFBF0.6040600@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> <503DFBF0.6040600@ninenines.eu> Message-ID: What are people talking about, when they say, that erlang have DSL? Erlang lacks any metaprogramming (parse_transform is not an example of good metaprogramming =). From erlang@REDACTED Wed Aug 29 14:03:18 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 29 Aug 2012 14:03:18 +0200 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503BD9DE.2080206@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> Message-ID: term_to_binary and binary_to_term It's incredibly useful to just flip backwards and forwards between internal and external representation of terms. Databases which can store *anything* sending *anything* over a socket encrypting *anything* become one-liners receive F -> F() end /Joe On Mon, Aug 27, 2012 at 10:34 PM, Lo?c Hoguin wrote: > Hello, > > I am wondering what you guys like the most about Erlang/OTP, especially > newcomers, maybe it changed your life, allowed you to climb the Everest (or > at least sleep at night). > > I'll be synthesizing that into a few key points that I am hopeful will be > reusable by anyone wanting to sell Erlang to their friends/work/clients. > > So help me by telling us what's so good about it! > > Thanks. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From paul.james.barry@REDACTED Wed Aug 29 14:06:58 2012 From: paul.james.barry@REDACTED (Paul Barry) Date: Wed, 29 Aug 2012 13:06:58 +0100 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: References: <503BD9DE.2080206@ninenines.eu> Message-ID: For me, it's the way Erlang makes me think about what I'm trying to do with my code. I find I'm thinking more, understanding more, and coding better. On 29 August 2012 13:03, Joe Armstrong wrote: > term_to_binary and binary_to_term > > It's incredibly useful to just flip backwards and forwards between > internal and external > representation of terms. Databases which can store *anything* sending > *anything* over a socket > encrypting *anything* become one-liners > > receive > F -> F() > end > > /Joe > > On Mon, Aug 27, 2012 at 10:34 PM, Lo?c Hoguin wrote: >> Hello, >> >> I am wondering what you guys like the most about Erlang/OTP, especially >> newcomers, maybe it changed your life, allowed you to climb the Everest (or >> at least sleep at night). >> >> I'll be synthesizing that into a few key points that I am hopeful will be >> reusable by anyone wanting to sell Erlang to their friends/work/clients. >> >> So help me by telling us what's so good about it! >> >> Thanks. >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From emmiller@REDACTED Wed Aug 29 18:09:11 2012 From: emmiller@REDACTED (Evan Miller) Date: Wed, 29 Aug 2012 11:09:11 -0500 Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503DFBF0.6040600@ninenines.eu> References: <503BD9DE.2080206@ninenines.eu> <503DFBF0.6040600@ninenines.eu> Message-ID: One thing that hasn't been mentioned is "hackability": everything is a term, so it's easy to inspect and alter other people's "opaque" data structures even without a documented API. On Wed, Aug 29, 2012 at 6:24 AM, Lo?c Hoguin wrote: > Thanks to everyone for helping. > > So far I've been able to put all these things into 6 key categories. This is > obviously not fully done but I'd like to hear your thoughts on it. If you > have anything to add you're welcome too! Categorization might not be the > best either. > > Write less, better code > * Pattern matching > * Bit syntax > * Simple base language > * DSLs > * Only worry about success cases (links to crash and fault tolerance) > > Fault tolerant concurrency by design > * Erlang models the real world > * Lightweight processes > * Fine-grained process isolation > * Shared nothing messaging, messages processed in the order we want > * Crash early, crash often (catch errors where they happen) > > A better OS for your applications > * OTP > * A system as a collection of small separate components > * Understandability and maintainability > * Upgrade without the need for restarting > * Numerous ways to inspect, debug and trace running nodes > * Combine many stacks without interference > > Distributed > * Easy to setup, easy to manage > * Same code for talking to a local or a remote process > > Community > * Nice, welcoming people > * Modest people, no drama queen > * Experts willing to share their knowledge > > Mature > * Code unlikely to break in future versions > * Sound design principles for building robust systems (OTP) > * Numerous testing tools > * Property-based testing > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Evan Miller http://www.evanmiller.org/ From emmiller@REDACTED Wed Aug 29 22:07:28 2012 From: emmiller@REDACTED (Evan Miller) Date: Wed, 29 Aug 2012 15:07:28 -0500 Subject: [erlang-questions] [ANN] TinyMQ Message-ID: I've split Chicago Boss's message queue system into its own project. Meet TinyMQ: https://github.com/evanmiller/tinymq The message queue is channel-based. Erlang processes can check a channel for messages posted after a certain timestamp, or subscribe to the channel and receive an Erlang message as soon as a new message is posted to the channel. Channels can have an unlimited number of subscribers (and processes can be subscribed to more than one channel). Channels are created and destroyed automatically, so the API is extremely simple -- 5 functions total. Each channel is managed by a gen_server so you get all the benefits of an OTP design. The code base is quite small (~250 lines) so newbies who want to see a functioning application with multiple interacting gen_servers might want to check it out and learn from it, although the documentation is somewhat sparse. The main drawback of TinyMQ is that it is not distributed and does not have any failover logic. It's best for developers who want a simple message queue that runs on one machine. The code has been used in Chicago Boss for some time now and has proven to be a popular feature. Hopefully others can benefit from it now. Best regards Evan From thomas.elsgaard@REDACTED Wed Aug 29 22:28:39 2012 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Wed, 29 Aug 2012 22:28:39 +0200 Subject: [erlang-questions] [ANN] TinyMQ In-Reply-To: References: Message-ID: > > The main drawback of TinyMQ is that it is not distributed and does not > have any failover logic. It's best for developers who want a simple > message queue that runs on one machine. The code has been used in > Chicago Boss for some time now and has proven to be a popular feature. > Hopefully others can benefit from it now. > > Best regards > > Evan > Fantastic, looking forward to look at it... Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Wed Aug 29 23:42:35 2012 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 29 Aug 2012 22:42:35 +0100 (BST) Subject: [erlang-questions] What do you like the most about Erlang/OTP? In-Reply-To: <503CC27E.5070404@berlin.ccc.de> Message-ID: <0c3f1e9b-dfaa-43e6-9de3-b273080bfdbd@knuth> ----- Original Message ----- > From: "John-Paul Bader" > To: "erlang-questions" > > orrdict:fetch(foo, []) throws and error. This was done on purpose, there is a 'find' which returns whether it was there and a 'fetch' which assumes that it is there. > There are plenty of examples where similar operations in > different > modules either throw or just return undefined but never in a > predictable way > My favorite: calling split on a one element list throws an error But you are right here. Robert From yousuke.hara@REDACTED Thu Aug 30 07:44:40 2012 From: yousuke.hara@REDACTED (Yousuke Hara) Date: Thu, 30 Aug 2012 14:44:40 +0900 Subject: [erlang-questions] [ANN] LeoFS 0.10.0 released (S3-Compatible Cloud Storage System) Message-ID: Hello, We have released LeoFS v0.10.0. LeoFS is S3-compatible cloud storage system with Erlang. We released first version on github on July 4th. - LeoFS site - - LeoFS repository - - LeoProject repository - LeoFS v0.10.0 was improved and fixed as follows: * Improve S3-API's compatibility * Add S3-authentication * Add S3-bucket * Add S3-related command in LeoFS's manager * "s3-gen-key" : Generate a S3 key pair(AccessKeyID and SecretAccessKey) * "s3-set-endpoint" : Register a new S3 Endpoint * "s3-delete-endpoint" : Delete a S3 Endpoint * "s3-get-endpoints" : Retrieve all of S3 Endpoints registered * "s3-get-buckets" : Retrieve all of Buckets registered * Improve order of system launch * Remove "attach command" in manager - After Storage launched, the node's state is automatically changed to "attached" * Improve rebalance-function's performance which is about 5 times compare with v0.9.1 * Improve compact-function. Restrain storage's load when compact objects. * Fix bugs * Deletion of Zero bytes in Storage * Behavior after the restart of Manager * Re-register procs into the Manager's monitor LeoFS's document: and LeoFS's milestones: if you are interested in my project, Please feel free to contact me (or us) anytime. Thank you, LeoFS lead developer Yosuke Hara (Tokyo, Japan) From zabrane3@REDACTED Thu Aug 30 08:00:06 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 30 Aug 2012 08:00:06 +0200 Subject: [erlang-questions] [ANN] LeoFS 0.10.0 released (S3-Compatible Cloud Storage System) In-Reply-To: References: Message-ID: Simply "awesome" job Yousuke. Congrats for the new release!!! Regards, Zabrane On Aug 30, 2012, at 7:44 AM, Yousuke Hara wrote: > Hello, > > We have released LeoFS v0.10.0. LeoFS is S3-compatible cloud storage > system with Erlang. > We released first version on github on July 4th. > > - LeoFS site - > - LeoFS repository - > - LeoProject repository - > > > LeoFS v0.10.0 was improved and fixed as follows: > > * Improve S3-API's compatibility > * Add S3-authentication > * Add S3-bucket > * Add S3-related command in LeoFS's manager > * "s3-gen-key" : Generate a S3 key pair(AccessKeyID and SecretAccessKey) > * "s3-set-endpoint" : Register a new S3 Endpoint > * "s3-delete-endpoint" : Delete a S3 Endpoint > * "s3-get-endpoints" : Retrieve all of S3 Endpoints registered > * "s3-get-buckets" : Retrieve all of Buckets registered > * Improve order of system launch > * Remove "attach command" in manager - After Storage launched, the > node's state is automatically changed to "attached" > * Improve rebalance-function's performance which is about 5 times > compare with v0.9.1 > * Improve compact-function. Restrain storage's load when compact objects. > * Fix bugs > * Deletion of Zero bytes in Storage > * Behavior after the restart of Manager > * Re-register procs into the Manager's monitor > > > LeoFS's document: > and LeoFS's milestones: > > > if you are interested in my project, Please feel free to contact me > (or us) anytime. > Thank you, > > > LeoFS lead developer > Yosuke Hara (Tokyo, Japan) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Thu Aug 30 08:00:36 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 30 Aug 2012 08:00:36 +0200 Subject: [erlang-questions] [ANN] TinyMQ In-Reply-To: References: Message-ID: Congrats Evan!!! Regards, Zabrane On Aug 29, 2012, at 10:07 PM, Evan Miller wrote: > I've split Chicago Boss's message queue system into its own project. > Meet TinyMQ: > > https://github.com/evanmiller/tinymq > > The message queue is channel-based. Erlang processes can check a > channel for messages posted after a certain timestamp, or subscribe to > the channel and receive an Erlang message as soon as a new message is > posted to the channel. Channels can have an unlimited number of > subscribers (and processes can be subscribed to more than one > channel). > > Channels are created and destroyed automatically, so the API is > extremely simple -- 5 functions total. Each channel is managed by a > gen_server so you get all the benefits of an OTP design. The code base > is quite small (~250 lines) so newbies who want to see a functioning > application with multiple interacting gen_servers might want to check > it out and learn from it, although the documentation is somewhat > sparse. > > The main drawback of TinyMQ is that it is not distributed and does not > have any failover logic. It's best for developers who want a simple > message queue that runs on one machine. The code has been used in > Chicago Boss for some time now and has proven to be a popular feature. > Hopefully others can benefit from it now. > > Best regards > > Evan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Thu Aug 30 10:10:50 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 30 Aug 2012 10:10:50 +0200 Subject: [erlang-questions] [ANN] LeoFS 0.10.0 released (S3-Compatible Cloud Storage System) In-Reply-To: References: Message-ID: <503F200A.2090105@ninenines.eu> Great job! On 08/30/2012 07:44 AM, Yousuke Hara wrote: > Hello, > > We have released LeoFS v0.10.0. LeoFS is S3-compatible cloud storage > system with Erlang. > We released first version on github on July 4th. > > - LeoFS site - > - LeoFS repository - > - LeoProject repository - > > > LeoFS v0.10.0 was improved and fixed as follows: > > * Improve S3-API's compatibility > * Add S3-authentication > * Add S3-bucket > * Add S3-related command in LeoFS's manager > * "s3-gen-key" : Generate a S3 key pair(AccessKeyID and SecretAccessKey) > * "s3-set-endpoint" : Register a new S3 Endpoint > * "s3-delete-endpoint" : Delete a S3 Endpoint > * "s3-get-endpoints" : Retrieve all of S3 Endpoints registered > * "s3-get-buckets" : Retrieve all of Buckets registered > * Improve order of system launch > * Remove "attach command" in manager - After Storage launched, the > node's state is automatically changed to "attached" > * Improve rebalance-function's performance which is about 5 times > compare with v0.9.1 > * Improve compact-function. Restrain storage's load when compact objects. > * Fix bugs > * Deletion of Zero bytes in Storage > * Behavior after the restart of Manager > * Re-register procs into the Manager's monitor > > > LeoFS's document: > and LeoFS's milestones: > > > if you are interested in my project, Please feel free to contact me > (or us) anytime. > Thank you, > > > LeoFS lead developer > Yosuke Hara (Tokyo, Japan) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From zabrane3@REDACTED Thu Aug 30 10:11:47 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 30 Aug 2012 10:11:47 +0200 Subject: [erlang-questions] Validate UTF-8 binary Message-ID: Hi guys, I'm looking after a code which can tell if a binary() is UTF-8 valid or not: is_utf8_valid(Bin) when is_binary(Bin) -> true; is_utf8_valid(Bin) when is_binary(Bin) -> {false, Reason}. Something to share? Regards, Zabrane From vladdu55@REDACTED Thu Aug 30 10:16:36 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 30 Aug 2012 10:16:36 +0200 Subject: [erlang-questions] [ANN] TinyMQ In-Reply-To: References: Message-ID: On Wed, Aug 29, 2012 at 10:07 PM, Evan Miller wrote: > I've split Chicago Boss's message queue system into its own project. > Meet TinyMQ: > > https://github.com/evanmiller/tinymq That's great! Thanks and congratulations! regards, Vlad From marc@REDACTED Thu Aug 30 10:17:28 2012 From: marc@REDACTED (Marc Worrell) Date: Thu, 30 Aug 2012 10:17:28 +0200 Subject: [erlang-questions] Validate UTF-8 binary In-Reply-To: References: Message-ID: <94C31B84-C197-4305-9A49-CE5FC7DAD177@worrell.nl> (and now to the list as well) Hi Zabrane, Check https://github.com/zotonic/zotonic/blob/master/src/support/z_string.erl Function: sanitize_utf8/1 - Marc On 30 aug. 2012, at 10:11, Zabrane Mickael wrote: > Hi guys, > > I'm looking after a code which can tell if a binary() is UTF-8 valid or not: > > is_utf8_valid(Bin) when is_binary(Bin) -> true; > is_utf8_valid(Bin) when is_binary(Bin) -> {false, Reason}. > > Something to share? > > Regards, > Zabrane > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From freeakk@REDACTED Thu Aug 30 10:21:14 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Thu, 30 Aug 2012 12:21:14 +0400 Subject: [erlang-questions] Validate UTF-8 binary In-Reply-To: <94C31B84-C197-4305-9A49-CE5FC7DAD177@worrell.nl> References: <94C31B84-C197-4305-9A49-CE5FC7DAD177@worrell.nl> Message-ID: is_unicode_valid(Str) -> Bin = unicode:characters_to_binary(Str), if is_binary(Bin) -> true; true -> {false, Bin} end. Str can be binary or characters. -- Best regards, Uvarov Michael From bengt.kleberg@REDACTED Thu Aug 30 10:21:42 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 30 Aug 2012 10:21:42 +0200 Subject: [erlang-questions] Validate UTF-8 binary In-Reply-To: References: Message-ID: <1346314902.5319.25.camel@seasc0057> Greetings, No code, just the comment that is_*() functions should return true/false. Not {}. Drop is_ from the name and you are fine. bengt On Thu, 2012-08-30 at 10:11 +0200, Zabrane Mickael wrote: > Hi guys, > > I'm looking after a code which can tell if a binary() is UTF-8 valid or not: > > is_utf8_valid(Bin) when is_binary(Bin) -> true; > is_utf8_valid(Bin) when is_binary(Bin) -> {false, Reason}. > > Something to share? > > Regards, > Zabrane > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Thu Aug 30 10:24:47 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 30 Aug 2012 10:24:47 +0200 Subject: [erlang-questions] Validate UTF-8 binary In-Reply-To: <1346314902.5319.25.camel@seasc0057> References: <1346314902.5319.25.camel@seasc0057> Message-ID: ;-) Regards, Zabrane On Aug 30, 2012, at 10:21 AM, Bengt Kleberg wrote: > Greetings, > > No code, just the comment that is_*() functions should return > true/false. Not {}. Drop is_ from the name and you are fine. > > > bengt > > On Thu, 2012-08-30 at 10:11 +0200, Zabrane Mickael wrote: >> Hi guys, >> >> I'm looking after a code which can tell if a binary() is UTF-8 valid or not: >> >> is_utf8_valid(Bin) when is_binary(Bin) -> true; >> is_utf8_valid(Bin) when is_binary(Bin) -> {false, Reason}. >> >> Something to share? >> >> Regards, >> Zabrane >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Thu Aug 30 10:25:52 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 30 Aug 2012 10:25:52 +0200 Subject: [erlang-questions] Validate UTF-8 binary In-Reply-To: <1346314902.5319.25.camel@seasc0057> References: <1346314902.5319.25.camel@seasc0057> Message-ID: <66A9A4CB-822E-40DD-BA96-FD390FA9BCEB@gmail.com> Thanks guys. You saved my day. Regards Zab On Aug 30, 2012, at 10:21 AM, Michael Uvarov wrote: > is_unicode_valid(Str) -> > Bin = unicode:characters_to_binary(Str), > if is_binary(Bin) -> true; true -> {false, Bin} end. > > Str can be binary or characters. > > -- > Best regards, > Uvarov Michael From marc@REDACTED Thu Aug 30 10:47:27 2012 From: marc@REDACTED (Marc Worrell) Date: Thu, 30 Aug 2012 10:47:27 +0200 Subject: [erlang-questions] Validate UTF-8 binary In-Reply-To: References: <94C31B84-C197-4305-9A49-CE5FC7DAD177@worrell.nl> Message-ID: <84B60350-ACF3-4B0C-B4DC-EDD2E7F23BC1@worrell.nl> Good pointer Michael. The Zotonic routines are from the R12 days, now that we are dropping support for R13( and earlier) we can replace some code with the unicode module. - Marc On 30 aug. 2012, at 10:21, Michael Uvarov wrote: > is_unicode_valid(Str) -> > Bin = unicode:characters_to_binary(Str), > if is_binary(Bin) -> true; true -> {false, Bin} end. > > Str can be binary or characters. > > -- > Best regards, > Uvarov Michael From desired.mta@REDACTED Thu Aug 30 11:38:10 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Thu, 30 Aug 2012 11:38:10 +0200 Subject: [erlang-questions] incomplete union checks in dialyzer Message-ID: Hi, -module(gg). -export([oho/0]). -type(error_enum() :: f01 | f02 | f03 | f04 | f05 | f06 | f07 | f08 | f09 | f10 | f11 | f12 | f13 | f14 ). -spec oho() -> error_enum(). oho() -> oh_no. $ erlc +debug_info gg.erl && dialyzer --no_check_plt gg.beam Proceeding with analysis... done in 0m0.48s done (passed successfully) Which is wrong ('oh_no' is not part of error_enum()). However, if I remove "| f14`": $ erlc +debug_info gg.erl && dialyzer --no_check_plt gg.beam Proceeding with analysis... gg.erl:9: Invalid type specification for function gg:oho/0. The success typing is () -> 'oh_no' done in 0m0.51s done (warnings were emitted) Which is right. How come? -- Motiejus Jak?tys From gleber.p@REDACTED Thu Aug 30 11:41:11 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 30 Aug 2012 11:41:11 +0200 Subject: [erlang-questions] Segmentation fault with lcnt compiled beam and +A Message-ID: Hello folks I have encountered the following segmentation fault. It happens when I combine lock-counter-compiled binary and "+A N" where N >= 64. But it works with up to 63 async threads. Erlang compiled without lcnt works on the same machine without a problem. The stacktrace is the following: $ gdb /opt/erlang/r15b01_lcnt/erts-5.9.1/bin/beam.smp core Copyright stuff ... snip... This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /opt/erlang/r15b01_lcnt/erts-5.9.1/bin/beam.smp...done. [New LWP 25517] [New LWP 25483] ...snip... [New LWP 25463] [New LWP 25466] warning: Can't read pathname for load map: Input/output error. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/opt/erlang/r15b01_lcnt/erts-5.9.1/bin/beam.smp -A 64 -- -root /opt/erlang/r15b'. Program terminated with signal 11, Segmentation fault. #0 erts_lcnt_lock (lock=) at beam/erl_lock_count.c:445 445 eltd->lock_in_conflict = 0; (gdb) bt #0 erts_lcnt_lock (lock=) at beam/erl_lock_count.c:445 #1 0x0000000000454096 in erts_mtx_lock_x (line=3480, mtx=0x87b9d8, file=) at beam/erl_threads.h:1342 #2 erts_alcu_alloc_thr_pref (type=92, extra=, size=1664) at beam/erl_alloc_util.c:3480 #3 0x00000000005c81f4 in ts_event_pool (size=25, endpp=0x7f00217f9e88) at common/ethr_aux.c:351 #4 0x00000000005c8306 in ts_event_alloc () at common/ethr_aux.c:394 #5 0x00000000005c86a5 in ethr_make_ts_event__ (tsepp=0x7f00217f9ec0) at common/ethr_aux.c:421 #6 0x00000000005cd9ce in thr_wrapper (vtwd=) at pthread/ethread.c:91 #7 0x00007f01f69c7e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #8 0x00007f01f64ed4bd in clone () from /lib/x86_64-linux-gnu/libc.so.6 #9 0x0000000000000000 in ?? () (gdb) $ uname -a Linux lb4 3.2.0-24-generic #37-Ubuntu SMP Wed Apr 25 08:43:22 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 12.04 LTS Release: 12.04 Codename: precise Erlang was configured via Kerl using the following configure options: "--enable-hipe --enable-smp-support --enable-threads --enable-kernel-poll --enable-lock-counter". I can reproduce this bug on 2-core a 32 bit machine with the same Ubuntu, but on it it fails if there are more than 93 async threads: gleber@REDACTED:~$ erl -lcnt +A 94 Segmentation fault gleber@REDACTED:~$ erl -lcnt +A 93 Erlang R15B01 (erts-5.9.1) [source] [smp:2:2] [async-threads:93] [hipe] [kernel-poll:false] [lock-counting] Eshell V5.9.1 (abort with ^G) 1> Have anyone encountered similar problem? Can I provide any more information helpful for pinpointing the issue? Best regards, Gleb Peregud From kostis@REDACTED Thu Aug 30 11:51:58 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 30 Aug 2012 11:51:58 +0200 Subject: [erlang-questions] incomplete union checks in dialyzer In-Reply-To: References: Message-ID: <503F37BE.4030002@cs.ntua.gr> On 08/30/2012 11:38 AM, Motiejus Jak?tys wrote: > Hi, > > -module(gg). > -export([oho/0]). > > -type(error_enum() :: > f01 | f02 | f03 | f04 | f05 | f06 | f07 > | f08 | f09 | f10 | f11 | f12 | f13 | f14 > ). > > -spec oho() -> error_enum(). > oho() -> oh_no. > > > $ erlc +debug_info gg.erl&& dialyzer --no_check_plt gg.beam > Proceeding with analysis... done in 0m0.48s > done (passed successfully) > > Which is wrong ('oh_no' is not part of error_enum()). However, if I > remove "| f14`": > > $ erlc +debug_info gg.erl&& dialyzer --no_check_plt gg.beam > Proceeding with analysis... > gg.erl:9: Invalid type specification for function gg:oho/0. The > success typing is () -> 'oh_no' > done in 0m0.51s > done (warnings were emitted) > > Which is right. How come? > Both are right. Dialyzer never promised that it will find all discrepancies in your code. It only promised that /when/ it produces some warning, then that warning is correct. Now, this particular behaviour happens because there is the following define in lib/hipe/cerl/erl_types.erl: -define(SET_LIMIT, 13). Make this bigger if you want and you will get more precise analysis results at the risk of increased analysis time. Kostis From desired.mta@REDACTED Thu Aug 30 12:02:48 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Thu, 30 Aug 2012 12:02:48 +0200 Subject: [erlang-questions] incomplete union checks in dialyzer In-Reply-To: <503F3A77.4010004@cs.ntua.gr> References: <503F37BE.4030002@cs.ntua.gr> <503F3A77.4010004@cs.ntua.gr> Message-ID: On Thu, Aug 30, 2012 at 12:03 PM, Kostis Sagonas wrote: > On 08/30/2012 11:53 AM, Motiejus Jak?tys wrote: >> >> I wonder how hard would this be to encapsulate and add to dialyzer as >> a run-time option? > > > Very simple, I guess. Thanks again. I will look into that. (posting back to mailing list) -- Motiejus Jak?tys From tobias.lindahl@REDACTED Thu Aug 30 14:41:19 2012 From: tobias.lindahl@REDACTED (Tobias Lindahl) Date: Thu, 30 Aug 2012 14:41:19 +0200 Subject: [erlang-questions] incomplete union checks in dialyzer In-Reply-To: References: <503F37BE.4030002@cs.ntua.gr> <503F3A77.4010004@cs.ntua.gr> Message-ID: 2012/8/30 Motiejus Jak?tys > On Thu, Aug 30, 2012 at 12:03 PM, Kostis Sagonas > wrote: > > On 08/30/2012 11:53 AM, Motiejus Jak?tys wrote: > >> > >> I wonder how hard would this be to encapsulate and add to dialyzer as > >> a run-time option? > > > > > > Very simple, I guess. > > Thanks again. I will look into that. > > I'm not sure that this is a good idea. This set limit is controlling the size of the inferred types, basically telling Dialyzer when to abstract some types (i.e., 15 separate atoms will be collapsed into atom()). This is of course affecting the fix point behavior of the analysis, as Kostis says, which is fine. However, if the plt is built with one setting and the later analysis uses another, I can imagine that you might end up in a situation where no fix point can be found, yielding an infinite loop. Making this into a run time option indicates that it is something you can play around with in order to get better results. To some extent it is, but since it can have pretty serious implications it might not be a good idea. Tobias -------------- next part -------------- An HTML attachment was scrubbed... URL: From tmr@REDACTED Thu Aug 30 15:18:29 2012 From: tmr@REDACTED (Tomas Morstein) Date: Thu, 30 Aug 2012 15:18:29 +0200 (CEST) Subject: [erlang-questions] [ANN] EGTM: embedded ACID compliant NoSQL engine for Erlang In-Reply-To: Message-ID: <5d12fc35-5c74-43b2-b991-ba18bddcd878@zimbra> For those who were interested in benchmarking, take a look here: http://ksbhaskar.blogspot.com/2011/02/from-44-seconds-to-27-seconds-simple.html From kanslozegast@REDACTED Thu Aug 30 16:08:56 2012 From: kanslozegast@REDACTED (Michael van Slingerland) Date: Thu, 30 Aug 2012 16:08:56 +0200 Subject: [erlang-questions] how to bind to divert socket in erlang Message-ID: Hi, I am working on a transparent proxy on openbsd that adds headers to http get requests. everything passing through the machine destined for port 80 will be redirect through a divert-socket to a userland daemon listening on port 8080. In python it is quite simple: socket.IPPROTO_DIVERT = 258 self.sock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_DIVERT) self.sock.bind(("0.0.0.0", port)) This makes the program listen on a divert socket on port 8080. My question is how I can set this in erlang? I am no erlang expert, I have been browsing through the gen_tcp an inet docs, but no success. Any help appreciated! thanks, Mike From gleber.p@REDACTED Thu Aug 30 16:25:37 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 30 Aug 2012 16:25:37 +0200 Subject: [erlang-questions] how to bind to divert socket in erlang In-Reply-To: References: Message-ID: I'm not sure if it helps, but read inet:setops/2 (especially last few paragraphs) and it it doesn't help, take a look at procket: https://github.com/msantos/procket On Thu, Aug 30, 2012 at 4:08 PM, Michael van Slingerland wrote: > Hi, > > I am working on a transparent proxy on openbsd that adds headers to > http get requests. > everything passing through the machine destined for port 80 will be > redirect through a divert-socket to a userland daemon listening on > port 8080. > > In python it is quite simple: > > socket.IPPROTO_DIVERT = 258 > self.sock = socket.socket(socket.AF_INET, socket.SOCK_RAW, > socket.IPPROTO_DIVERT) > self.sock.bind(("0.0.0.0", port)) > > This makes the program listen on a divert socket on port 8080. > > My question is how I can set this in erlang? > I am no erlang expert, I have been browsing through the gen_tcp an > inet docs, but no success. > > Any help appreciated! > > thanks, > Mike > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From egil@REDACTED Thu Aug 30 16:42:37 2012 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Thu, 30 Aug 2012 16:42:37 +0200 Subject: [erlang-questions] Segmentation fault with lcnt compiled beam and +A In-Reply-To: References: Message-ID: <503F7BDD.2080802@erlang.org> Hi! Thanks for reporting this! I'll try to look into it. // Bj?rn-Egil On 2012-08-30 11:41, Gleb Peregud wrote: > Hello folks > > I have encountered the following segmentation fault. It happens when I > combine lock-counter-compiled binary and "+A N" where N >= 64. But it > works with up to 63 async threads. Erlang compiled without lcnt works > on the same machine without a problem. The stacktrace is the > following: > > $ gdb /opt/erlang/r15b01_lcnt/erts-5.9.1/bin/beam.smp core > Copyright stuff ... snip... > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /opt/erlang/r15b01_lcnt/erts-5.9.1/bin/beam.smp...done. > [New LWP 25517] > [New LWP 25483] > ...snip... > [New LWP 25463] > [New LWP 25466] > > warning: Can't read pathname for load map: Input/output error. > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". > Core was generated by `/opt/erlang/r15b01_lcnt/erts-5.9.1/bin/beam.smp > -A 64 -- -root /opt/erlang/r15b'. > Program terminated with signal 11, Segmentation fault. > #0 erts_lcnt_lock (lock=) at beam/erl_lock_count.c:445 > 445 eltd->lock_in_conflict = 0; > (gdb) bt > #0 erts_lcnt_lock (lock=) at beam/erl_lock_count.c:445 > #1 0x0000000000454096 in erts_mtx_lock_x (line=3480, mtx=0x87b9d8, > file=) > at beam/erl_threads.h:1342 > #2 erts_alcu_alloc_thr_pref (type=92, extra=, > size=1664) at beam/erl_alloc_util.c:3480 > #3 0x00000000005c81f4 in ts_event_pool (size=25, > endpp=0x7f00217f9e88) at common/ethr_aux.c:351 > #4 0x00000000005c8306 in ts_event_alloc () at common/ethr_aux.c:394 > #5 0x00000000005c86a5 in ethr_make_ts_event__ (tsepp=0x7f00217f9ec0) > at common/ethr_aux.c:421 > #6 0x00000000005cd9ce in thr_wrapper (vtwd=) at > pthread/ethread.c:91 > #7 0x00007f01f69c7e9a in start_thread () from > /lib/x86_64-linux-gnu/libpthread.so.0 > #8 0x00007f01f64ed4bd in clone () from /lib/x86_64-linux-gnu/libc.so.6 > #9 0x0000000000000000 in ?? () > (gdb) > > $ uname -a > Linux lb4 3.2.0-24-generic #37-Ubuntu SMP Wed Apr 25 08:43:22 UTC 2012 > x86_64 x86_64 x86_64 GNU/Linux > $ lsb_release -a > No LSB modules are available. > Distributor ID: Ubuntu > Description: Ubuntu 12.04 LTS > Release: 12.04 > Codename: precise > > Erlang was configured via Kerl using the following configure options: > "--enable-hipe --enable-smp-support --enable-threads > --enable-kernel-poll --enable-lock-counter". > > I can reproduce this bug on 2-core a 32 bit machine with the same > Ubuntu, but on it it fails if there are more than 93 async threads: > > gleber@REDACTED:~$ erl -lcnt +A 94 > Segmentation fault > gleber@REDACTED:~$ erl -lcnt +A 93 > Erlang R15B01 (erts-5.9.1) [source] [smp:2:2] [async-threads:93] > [hipe] [kernel-poll:false] [lock-counting] > > Eshell V5.9.1 (abort with ^G) > 1> > > Have anyone encountered similar problem? Can I provide any more > information helpful for pinpointing the issue? > > Best regards, > Gleb Peregud > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From egil@REDACTED Thu Aug 30 16:46:40 2012 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Thu, 30 Aug 2012 16:46:40 +0200 Subject: [erlang-questions] Segmentation fault with lcnt compiled beam and +A In-Reply-To: <503F7BDD.2080802@erlang.org> References: <503F7BDD.2080802@erlang.org> Message-ID: <503F7CD0.6050701@erlang.org> On 2012-08-30 16:42, Bj?rn-Egil Dahlberg wrote: > Hi! > > Thanks for reporting this! I'll try to look into it. I couldn't reproduce the error on current OTP_R15B02-maint branch but not I could reproduce the error in OTP_R15B01. Hopefully the problem is already fixed =) Give me a shout if you see it in current maint branch as well. // Bj?rn-Egil > > // Bj?rn-Egil > > On 2012-08-30 11:41, Gleb Peregud wrote: >> Hello folks >> >> I have encountered the following segmentation fault. It happens when I >> combine lock-counter-compiled binary and "+A N" where N >= 64. But it >> works with up to 63 async threads. Erlang compiled without lcnt works >> on the same machine without a problem. The stacktrace is the >> following: >> >> $ gdb /opt/erlang/r15b01_lcnt/erts-5.9.1/bin/beam.smp core >> Copyright stuff ... snip... >> This GDB was configured as "x86_64-linux-gnu". >> For bug reporting instructions, please see: >> ... >> Reading symbols from >> /opt/erlang/r15b01_lcnt/erts-5.9.1/bin/beam.smp...done. >> [New LWP 25517] >> [New LWP 25483] >> ...snip... >> [New LWP 25463] >> [New LWP 25466] >> >> warning: Can't read pathname for load map: Input/output error. >> [Thread debugging using libthread_db enabled] >> Using host libthread_db library >> "/lib/x86_64-linux-gnu/libthread_db.so.1". >> Core was generated by `/opt/erlang/r15b01_lcnt/erts-5.9.1/bin/beam.smp >> -A 64 -- -root /opt/erlang/r15b'. >> Program terminated with signal 11, Segmentation fault. >> #0 erts_lcnt_lock (lock=) at beam/erl_lock_count.c:445 >> 445 eltd->lock_in_conflict = 0; >> (gdb) bt >> #0 erts_lcnt_lock (lock=) at beam/erl_lock_count.c:445 >> #1 0x0000000000454096 in erts_mtx_lock_x (line=3480, mtx=0x87b9d8, >> file=) >> at beam/erl_threads.h:1342 >> #2 erts_alcu_alloc_thr_pref (type=92, extra=, >> size=1664) at beam/erl_alloc_util.c:3480 >> #3 0x00000000005c81f4 in ts_event_pool (size=25, >> endpp=0x7f00217f9e88) at common/ethr_aux.c:351 >> #4 0x00000000005c8306 in ts_event_alloc () at common/ethr_aux.c:394 >> #5 0x00000000005c86a5 in ethr_make_ts_event__ (tsepp=0x7f00217f9ec0) >> at common/ethr_aux.c:421 >> #6 0x00000000005cd9ce in thr_wrapper (vtwd=) at >> pthread/ethread.c:91 >> #7 0x00007f01f69c7e9a in start_thread () from >> /lib/x86_64-linux-gnu/libpthread.so.0 >> #8 0x00007f01f64ed4bd in clone () from /lib/x86_64-linux-gnu/libc.so.6 >> #9 0x0000000000000000 in ?? () >> (gdb) >> >> $ uname -a >> Linux lb4 3.2.0-24-generic #37-Ubuntu SMP Wed Apr 25 08:43:22 UTC 2012 >> x86_64 x86_64 x86_64 GNU/Linux >> $ lsb_release -a >> No LSB modules are available. >> Distributor ID: Ubuntu >> Description: Ubuntu 12.04 LTS >> Release: 12.04 >> Codename: precise >> >> Erlang was configured via Kerl using the following configure options: >> "--enable-hipe --enable-smp-support --enable-threads >> --enable-kernel-poll --enable-lock-counter". >> >> I can reproduce this bug on 2-core a 32 bit machine with the same >> Ubuntu, but on it it fails if there are more than 93 async threads: >> >> gleber@REDACTED:~$ erl -lcnt +A 94 >> Segmentation fault >> gleber@REDACTED:~$ erl -lcnt +A 93 >> Erlang R15B01 (erts-5.9.1) [source] [smp:2:2] [async-threads:93] >> [hipe] [kernel-poll:false] [lock-counting] >> >> Eshell V5.9.1 (abort with ^G) >> 1> >> >> Have anyone encountered similar problem? Can I provide any more >> information helpful for pinpointing the issue? >> >> Best regards, >> Gleb Peregud >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > From jesper.louis.andersen@REDACTED Thu Aug 30 16:52:53 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 30 Aug 2012 16:52:53 +0200 Subject: [erlang-questions] Fail politely, fail noisily In-Reply-To: References: Message-ID: On Aug 29, 2012, at 9:47 AM, Thomas Elsgaard wrote: > Hi Joe > > There is also PHK's concept of "guru meditation" in the varnish http accelerator.. > > http://www.version2.dk/blog/guru-meditation-9738 > > Basically the idea is: the user cannot do anything about the error, so let it fail, but provide the transaction Id (xid) so the admin can can troubleshoot... I did this in erlang. It is very easy. When the web server crashes on the message, you grab Trace = erlang:get_stacktrace(), and Now = erlang:now(). Then you generate a unique token: crypto:sha(term_to_binary({Trace, Now})) and you put that token to the user as well as in the crash log of the server. Now any bug reported with a proper token just requires a grep on the logs to find the exact error the user had. I think it took me 1-2 hours to write and test. From jesper.louis.andersen@REDACTED Thu Aug 30 17:25:41 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 30 Aug 2012 17:25:41 +0200 Subject: [erlang-questions] How to use dbg and fprof to profile a running server ? In-Reply-To: References: Message-ID: On Aug 28, 2012, at 11:17 AM, Bo Chen wrote: > Hello, every one, > I am a rookie in erlang. Recently I am trying to use the dbg and fprof module to profile a running server to observe the performance. Since the server is based on mochiweb, a quite popular web server framework, and I only wish to see the performance of my own code, I tried the dbg module. I have tried in the following two ways: > I would look into eprof first. It will give you profile outputs in the "large" and when you then know where the time is spent I would target that area with fprof. Also, if you are running on multicore, don't skip the thought that you may have to do lock count profiling as well. For your own code, I would set up a typical test case and then use that as the profile basis for fprof. From egil@REDACTED Thu Aug 30 17:54:19 2012 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Thu, 30 Aug 2012 17:54:19 +0200 Subject: [erlang-questions] How to use dbg and fprof to profile a running server ? In-Reply-To: References: Message-ID: <503F8CAB.5080700@erlang.org> On 2012-08-30 17:25, Jesper Louis Andersen wrote: > On Aug 28, 2012, at 11:17 AM, Bo Chen wrote: > >> Hello, every one, >> I am a rookie in erlang. Recently I am trying to use the dbg and fprof module to profile a running server to observe the performance. Since the server is based on mochiweb, a quite popular web server framework, and I only wish to see the performance of my own code, I tried the dbg module. I have tried in the following two ways: >> > I would look into eprof first. It will give you profile outputs in the "large" and when you then know where the time is spent I would target that area with fprof. Also, if you are running on multicore, don't skip the thought that you may have to do lock count profiling as well. > > For your own code, I would set up a typical test case and then use that as the profile basis for fprof. I agree with Jesper. Start with eprof to get a feeling for what might be troublesome. Also eprof can handle a much higher load than fprof can, i.e. eprof is a magnitude or two faster. The lock counter, lcnt, can also handle relative high loads. It can also dismiss some results thus saving some memory and performance. lcnt can also be used to see msg-queue locks on erlang processes, thus useful to profile erlang application and not just to tweak the runtime. // Bj?rn-Egil Erlang/OTP From desired.mta@REDACTED Thu Aug 30 22:01:58 2012 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Thu, 30 Aug 2012 22:01:58 +0200 Subject: [erlang-questions] incomplete union checks in dialyzer In-Reply-To: References: <503F37BE.4030002@cs.ntua.gr> <503F3A77.4010004@cs.ntua.gr> Message-ID: <20120830200158.GA4846@precise> On Thu, Aug 30, 2012 at 02:41:19PM +0200, Tobias Lindahl wrote: > 2012/8/30 Motiejus Jak?tys > > > On Thu, Aug 30, 2012 at 12:03 PM, Kostis Sagonas > > wrote: > > > On 08/30/2012 11:53 AM, Motiejus Jak?tys wrote: > > >> > > >> I wonder how hard would this be to encapsulate and add to dialyzer as > > >> a run-time option? > > > > > > > > > Very simple, I guess. > > > > Thanks again. I will look into that. > > > > > I'm not sure that this is a good idea. This set limit is controlling the > size of the inferred types, basically telling Dialyzer when to abstract > some types (i.e., 15 separate atoms will be collapsed into atom()). This is > of course affecting the fix point behavior of the analysis, as Kostis says, > which is fine. > > However, if the plt is built with one setting and the later analysis uses > another, I can imagine that you might end up in a situation where no fix > point can be found, yielding an infinite loop. > > Making this into a run time option indicates that it is something you can > play around with in order to get better results. To some extent it is, but > since it can have pretty serious implications it might not be a good idea. Hi, thanks for your comments. This is a good point. PLT and dialyzer runtime should be compatible. How about making this an option for --build_plt? And always getting the value from PLT for analysis? Motiejus -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From ok@REDACTED Fri Aug 31 08:20:22 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 31 Aug 2012 18:20:22 +1200 Subject: [erlang-questions] What I dislike about Erlang Message-ID: We've just had a thread about what people like about Erlang. We also had the announcement of TinyMQ. So I'm going to use this as an example of what's *really* wrong with Erlang. Don't get me wrong. I endorse everything everyone else has said in favour of Erlang. Erlang is like democracy: the worst thing in its class except for all the others, and something that is increasingly imitated by people who just don't get some of the fundamental things about it. I also endorse what people have said in praise of TinyMQ. There are lots of things that it does right: - there is a README - there are EDoc comments with @specs for the public interface - the functions and variables are named well enough that I was never in doubt about what any part of the code was up to, at least not for longer than a second or two - the hard work of process management is delegated to OTP behaviours At this point, it's looking better than anything I've written. Make no mistake: I am not saying that Erlang or TinyMQ are *bad*. They are good things; I'm just ranting somewhat vaguely about why they should be better. LUMPS OF INDISTINGUISHABLE CODE. Up to a certain level of hand-waving, TinyMQ can be roughly understood thus: The TinyMQ *system* is a monitor guarding a dictionary mapping strings to channnels, where a channel is a monitor guarding a bag of subscribers and a sliding window of {Message, Timestamp} pairs. YOU CANNOT SEE THIS AT A GLANCE. This is not Evan Miller's fault. *Anything* you write in Erlang is going to end up as lumps of indistinguishable code, because there is nothing else for it to be. This is also true in C, C++, Java, C#, Javascript, Go, Eiffel, Smalltalk, Prolog, Haskell, Clean, SML, ..., not to mention Visual Basic and Fortran. Almost the only languages I know where it doesn't *have* to be true are Lisp, Scheme, and Lisp-Flavoured Erlang. Arguably Prolog *could* be in this group, but in practice it usually is in the other camp. Thanks to the preprocessor, C *can* be made rather more scrutable, but for some reason this is frowned on. There's the e2 project (http://e2project.org) which is a step in a good direction, but it doesn't do much about this problem. A version of TinyMQ using e2_service instead of gen_server would in fact exacerbate the problem by mushing handle_call/3, handle_cast/2, and handle_info/2 into one function, turning three lumps into one bigger lump. LUMPS OF DATA. Take tinymq_channel_controller as an example. Using an OTP behaviour means that all six dimensions of the state are mushed together in one data structure. This goes a long way towards hiding the fact that supervisor, channel, and max_age are never changed messages, subscribers, and last_pull *are* changed. One teeny tiny step here would be to offer an alternative set of callbacks for some behaviours where the "state" is separated into immutable "context" and mutable "state", so that it is obvious *by construction* that the context information *can't* be changed. Another option would be to have some way of annotation in a -record declaration that a field cannot be updated. I prefer the segregation approach on the grounds of no language change being needed and the improved efficiency of not copying fields that can't have changed. Others might prefer the revise -record approach on the grounds of not having to change or duplicate the OTP behaviours. I had to reach each file in detail - to find that certain fields *happened* not to be changed - to understand the design well enough to tell that this was almost certainly deliberate. WE DOCUMENT THE WRONG THINGS. It's well known that there are two kinds of documentation, "external" documentation for people writing clients of a module, and "internal" documentation for people maintaining the module itself. It's also well known that the division is simplistic; if the external documentation is silent about material points you have to read the internal documentation. In languages like Prolog and Erlang and Scheme where you build data structures out of existing "universal" types and have no data structure declarations, we tend to document procedures but not data. This is backwards. If you understand the data, and especially its invariants, the code is often pretty obvious. There are two examples of this in TinyMQ. One is specific to TinyMQ. The other other is nearly universal in Erlang practice. Erlang systems are made of lots of processes sending messages to each other. Joe Armstrong has often said THINK ABOUT THE PROTOCOLS. But Erlang programmers very seldom *write* about the protocols. Using the OTP behaviours, a "concurrent object" is implemented as a module with a bunch of interface functions that forward messages through the OTP layer to the callback code managed by whatever behaviour it is. This protocol is unique to each kind of concurrent object. It's often generated in one module (the one with the interface functions) and consumed in another (the one with the callback code), as it is in TinyMQ. And it's not documented. It is possible to reconstruct this protocol by reading the code in detail and noting down what you see. It is troublesome when, as in TinyMQ, the two modules disagree about the protocol. It's clear that _something_ is wrong, but what, exactly? For example, tinymq_controller has a case handle_cast({set_max_age, newMaxAge}, State) -> but this is the only occurrence of set_max_age anywhere in TinyMQ. Is its presence in tinymq_controller an example of dead code, or is its absence from the rest of the application an example of missing code? The same question can be asked about 'expire' (which would forget a channel without making it actually go away, if it could ever be invoked, which it can't.) Almost as soon as I started reading Erlang code many years ago it seemed obvious to me that documenting (and if possible, type checking) these internal protocols was a very important part of Erlang internal documentation. There must be something wrong with my brain, because other people don't seem to feel this lack anywhere nearly as strongly as I do. I think Joe Armstrong sort of sees this at the next level up or he would never have invented UBF. But Occam, Go, and Sing# have typed channels, so they *are* addressing the issue, and *do* have a natural central point to document what the alternatives of an internal protocol signify. Another documentation failure is that we fail to document what is not there. In TinyMQ, a channel automatically comes into existence when you try to use it. Perhaps as a consequence of this, there is no way to shut a channel down. In TinyMQ, old messages are not removed from a channel when they expire, but the next time someone does a 'subscribe' (waves hands) or a 'poll' or a 'push' *after* they expire. So if processes stop sending and requesting messages to some channel, the last few messages, no matter how large, may hang around forever. I'm sure there is a reason, but because it's a reason for something *not* being there, there's no obvious place to hang the comment, and there isn't one. (Except for the dead 'expire' clause mentioned above.) IT'S HARD TO SPOT SALIENT DETAIL IN A SEA OF GLUE CODE. The central fact about TinyMQ is that it holds the messages of a channel in a simple list of {Message, Timestamp} pairs. As a result, every operation on the data takes time linear in the current size. This is not stated anywhere in any comments nor in the README. You have to read the code in detail to discover this. And it is a rather nasty surprise. If a channel holds N messages, the operations *can* be done in O(log(N)) time. (I believe it is possible to do even better.) Some sliding window applications have a bound on the number of elements in the window. This one has a bound on the age of elements, but they could arrive at a very high rate, so N *could* get large. It is very easy to implement the necessary operations using lists, so much so that they are present in several copies. Revising the TinyMQ implementation to work better with long queues would be harder than necessary because of this. And this goes un-noticed because there is so much glue code for the guts to get lost in. Given that Evan Miller took the trouble to use library components for structuring this application, why didn't he take the next step, and use the existing 'sliding window' library data structure? Because there is none! Yet sliding windows of one sort or another have come up before in this mailing list. Perhaps we should have a Wiki page on trapexit to gather requirements for one or more sliding window libraries. Or perhaps not. "true religion jeans for women" -- what has that or "Cheap Nike Shoes" to do with Erlang/OTP (http://www.trapexit.org/forum/viewforum.php?f=20)? From max.lapshin@REDACTED Fri Aug 31 08:42:57 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 31 Aug 2012 10:42:57 +0400 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: Message-ID: I really don't understand, what makes you sad. Is it that people don't write internal documentation? Perhaps it is because code is a documenation. From f@REDACTED Fri Aug 31 09:36:10 2012 From: f@REDACTED (Francesco Mazzoli) Date: Fri, 31 Aug 2012 08:36:10 +0100 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: Message-ID: <874nnjy0lx.wl%f@mazzo.li> At Fri, 31 Aug 2012 10:42:57 +0400, Max Lapshin wrote: > I really don't understand, what makes you sad. > > Is it that people don't write internal documentation? Perhaps it is because > code is a documenation. Did you read his post? His main point about documentation is that people tend to document functions but not data structures, which is something that I fully agree with. This problem derives from Erlang being unityped - In languages like Haskell or SML with proper ADTs documenting them is a natural thing to do, and that's usually the main part of the documentation in general. -- Francesco * Often in error, never in doubt From rapsey@REDACTED Fri Aug 31 09:43:41 2012 From: rapsey@REDACTED (Rapsey) Date: Fri, 31 Aug 2012 09:43:41 +0200 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: Message-ID: My biggest gripe with erlang are the limitations of records. Anyone know when frames will make an appearance? Sergej On Fri, Aug 31, 2012 at 8:20 AM, Richard O'Keefe wrote: > We've just had a thread about what people like about Erlang. > We also had the announcement of TinyMQ. > So I'm going to use this as an example of what's *really* > wrong with Erlang. > > Don't get me wrong. I endorse everything everyone else has > said in favour of Erlang. Erlang is like democracy: the worst > thing in its class except for all the others, and something > that is increasingly imitated by people who just don't get > some of the fundamental things about it. > > I also endorse what people have said in praise of TinyMQ. > There are lots of things that it does right: > - there is a README > - there are EDoc comments with @specs for the public > interface > - the functions and variables are named well enough that > I was never in doubt about what any part of the code was > up to, at least not for longer than a second or two > - the hard work of process management is delegated to OTP > behaviours > At this point, it's looking better than anything I've written. > > Make no mistake: I am not saying that Erlang or TinyMQ are *bad*. > They are good things; I'm just ranting somewhat vaguely about > why they should be better. > > > LUMPS OF INDISTINGUISHABLE CODE. > > Up to a certain level of hand-waving, TinyMQ can be roughly > understood thus: > The TinyMQ *system* is a monitor > guarding a dictionary mapping strings to channnels, > where > a channel is a monitor > guarding a bag of subscribers and > a sliding window of {Message, Timestamp} pairs. > > YOU CANNOT SEE THIS AT A GLANCE. > > This is not Evan Miller's fault. *Anything* you write in > Erlang is going to end up as lumps of indistinguishable code, > because there is nothing else for it to be. > > This is also true in C, C++, Java, C#, Javascript, Go, > Eiffel, Smalltalk, Prolog, Haskell, Clean, SML, ..., > not to mention Visual Basic and Fortran. > > Almost the only languages I know where it doesn't *have* to > be true are Lisp, Scheme, and Lisp-Flavoured Erlang. Arguably > Prolog *could* be in this group, but in practice it usually is > in the other camp. Thanks to the preprocessor, C *can* be > made rather more scrutable, but for some reason this is frowned on. > > There's the e2 project (http://e2project.org) which is a step > in a good direction, but it doesn't do much about this problem. > A version of TinyMQ using e2_service instead of gen_server > would in fact exacerbate the problem by mushing > handle_call/3, handle_cast/2, and handle_info/2 into one > function, turning three lumps into one bigger lump. > > LUMPS OF DATA. > > Take tinymq_channel_controller as an example. > Using an OTP behaviour means that all six dimensions of the state > are mushed together in one data structure. This goes a long way > towards hiding the fact that > > supervisor, channel, and max_age are never changed > messages, subscribers, and last_pull *are* changed. > > One teeny tiny step here would be to offer an alternative set of > callbacks for some behaviours where the "state" is separated into > immutable "context" and mutable "state", so that it is obvious > *by construction* that the context information *can't* be changed. > > Another option would be to have some way of annotation in a > -record declaration that a field cannot be updated. > > I prefer the segregation approach on the grounds of no language > change being needed and the improved efficiency of not copying > fields that can't have changed. Others might prefer the revise > -record approach on the grounds of not having to change or > duplicate the OTP behaviours. > > I had to reach each file in detail > - to find that certain fields *happened* not to be changed > - to understand the design well enough to tell that this was > almost certainly deliberate. > > WE DOCUMENT THE WRONG THINGS. > > It's well known that there are two kinds of documentation, > "external" documentation for people writing clients of a module, > and "internal" documentation for people maintaining the module > itself. It's also well known that the division is simplistic; > if the external documentation is silent about material points > you have to read the internal documentation. > > In languages like Prolog and Erlang and Scheme where you build > data structures out of existing "universal" types and have no > data structure declarations, we tend to document procedures > but not data. This is backwards. If you understand the data, > and especially its invariants, the code is often pretty obvious. > > There are two examples of this in TinyMQ. One is specific to > TinyMQ. The other other is nearly universal in Erlang practice. > > Erlang systems are made of lots of processes sending messages > to each other. Joe Armstrong has often said THINK ABOUT THE > PROTOCOLS. But Erlang programmers very seldom *write* about > the protocols. > > Using the OTP behaviours, a "concurrent object" is implemented > as a module with a bunch of interface functions that forward > messages through the OTP layer to the callback code managed by > whatever behaviour it is. This protocol is unique to each kind > of concurrent object. It's often generated in one module (the > one with the interface functions) and consumed in another (the > one with the callback code), as it is in TinyMQ. And it's not > documented. > > It is possible to reconstruct this protocol by reading the code > in detail and noting down what you see. It is troublesome when, > as in TinyMQ, the two modules disagree about the protocol. It's > clear that _something_ is wrong, but what, exactly? > > For example, tinymq_controller has a case > handle_cast({set_max_age, newMaxAge}, State) -> > but this is the only occurrence of set_max_age anywhere in TinyMQ. > Is its presence in tinymq_controller an example of dead code, > or is its absence from the rest of the application an example > of missing code? The same question can be asked about 'expire' > (which would forget a channel without making it actually go away, > if it could ever be invoked, which it can't.) > > Almost as soon as I started reading Erlang code many years ago > it seemed obvious to me that documenting (and if possible, type > checking) these internal protocols was a very important part of > Erlang internal documentation. There must be something wrong > with my brain, because other people don't seem to feel this lack > anywhere nearly as strongly as I do. I think Joe Armstrong sort > of sees this at the next level up or he would never have invented > UBF. > > But Occam, Go, and Sing# have typed channels, so they *are* > addressing the issue, and *do* have a natural central point to > document what the alternatives of an internal protocol signify. > > Another documentation failure is that we fail to document what > is not there. In TinyMQ, a channel automatically comes into > existence when you try to use it. Perhaps as a consequence of > this, there is no way to shut a channel down. In TinyMQ, old > messages are not removed from a channel when they expire, but > the next time someone does a 'subscribe' (waves hands) or a 'poll' > or a 'push' *after* they expire. So if processes stop sending > and requesting messages to some channel, the last few messages, > no matter how large, may hang around forever. I'm sure there > is a reason, but because it's a reason for something *not* being > there, there's no obvious place to hang the comment, and there > isn't one. (Except for the dead 'expire' clause mentioned above.) > > IT'S HARD TO SPOT SALIENT DETAIL IN A SEA OF GLUE CODE. > > The central fact about TinyMQ is that it holds the messages of > a channel in a simple list of {Message, Timestamp} pairs. As > a result, every operation on the data takes time linear in the > current size. > > This is not stated anywhere in any comments nor in the README. > You have to read the code in detail to discover this. And it > is a rather nasty surprise. If a channel holds N messages, > the operations *can* be done in O(log(N)) time. (I believe it > is possible to do even better.) Some sliding window applications > have a bound on the number of elements in the window. This one > has a bound on the age of elements, but they could arrive at a > very high rate, so N *could* get large. > > It is very easy to implement the necessary operations using lists, > so much so that they are present in several copies. Revising the > TinyMQ implementation to work better with long queues would be > harder than necessary because of this. And this goes un-noticed > because there is so much glue code for the guts to get lost in. > > Given that Evan Miller took the trouble to use library components > for structuring this application, why didn't he take the next step, > and use the existing 'sliding window' library data structure? > > Because there is none! > > Yet sliding windows of one sort or another have come up before in > this mailing list. Perhaps we should have a Wiki page on > trapexit to gather requirements for one or more sliding window > libraries. Or perhaps not. "true religion jeans for women" -- > what has that or "Cheap Nike Shoes" to do with Erlang/OTP > (http://www.trapexit.org/forum/viewforum.php?f=20)? > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Aug 31 09:57:53 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 31 Aug 2012 11:57:53 +0400 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: <874nnjy0lx.wl%f@mazzo.li> References: <874nnjy0lx.wl%f@mazzo.li> Message-ID: On Fri, Aug 31, 2012 at 11:36 AM, Francesco Mazzoli wrote: > > Did you read his post? His main point about documentation is that people tend > to document functions but not data structures, which is something that I fully > agree with. > This post has nothing to do with erlang. It is just an ancient question: waste your time on documenting inner code or not. I never document such things, because it is silly: you spend more time on rewriting documentation. And I perfectly read ffmpeg sources without any documentation inside them. So, about erlang. There are two BIG problems in erlang: error_logger and lack of frames. error_logger is a problem, because 99% times, when erlang VM crashes, it was error_logger, that allocated 40 GB of RAM and died. If erlang is eating 100% CPU, it is error_logger. Error_logger is the heel of Achilles of erlang. And it is a pity, because it makes impossible to create gen_server with states larger than 100 KBytes. Lack of frames makes almost impossible hot code upgrades with changing of data structures. It makes impossible distributing plugins. From gordon@REDACTED Fri Aug 31 10:41:58 2012 From: gordon@REDACTED (Gordon Guthrie) Date: Fri, 31 Aug 2012 09:41:58 +0100 Subject: [erlang-questions] TrapExit was Re: What I dislike about Erlang Message-ID: In his latest post Richard O'Keefe makes a good point about TrapExit: Or perhaps not. "true religion jeans for women" -- what has that or "Cheap Nike Shoes" to do with Erlang/OTP http://www.trapexit.org/forum/viewforum.php?f=20)? Spammers overrunning the site for two months seems to me to indicate that the site is dead - or certainly abandoned by an administrator. Doesn't look so good for the community. Gordon From essen@REDACTED Fri Aug 31 10:42:34 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Fri, 31 Aug 2012 10:42:34 +0200 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: Message-ID: <504078FA.5040909@ninenines.eu> On 08/31/2012 08:42 AM, Max Lapshin wrote: > I really don't understand, what makes you sad. > > Is it that people don't write internal documentation? Perhaps it is > because code is a documenation. Only to an extent. If your gen_server is only meant to be accessed through the API functions, then sure, that's fine. If it can receive messages from other sources, then these should probably be documented. For gen_server the module description would be a good place to put explanations, and handle_info should probably have a detailed spec, clause by clause, on what it can receive. I think the difficulty in documenting the protocol is that we write modules while the protocol is about processes. What do we do about processes that use more than one module that can send messages? I go about it with an "Internals" documentation but that's separate from the code. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From thomasl_erlang@REDACTED Fri Aug 31 10:47:53 2012 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Fri, 31 Aug 2012 01:47:53 -0700 (PDT) Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: <874nnjy0lx.wl%f@mazzo.li> Message-ID: <1346402873.7673.YahooMailNeo@web111409.mail.gq1.yahoo.com> ----- Original Message ----- > From: Max Lapshin > > Lack of frames makes almost impossible hot code upgrades with changing > of data structures. It makes impossible distributing plugins. The whole "upgrade your record/state" thing is pretty awkward (because at this point you usually have two definitions of the same record, for those who haven't thought about it). One approach is to write a shim module and do a more elaborate upgrade, which is a pain. Another approach is to grunt that "records are just syntactic sugar for tuples" and hack away. But I think this problem?could also be fixed by changing the gen_* interface. Add two callbacks, invoked at the appropriate points: * convert internal state to key-value list (when exiting the old version) * convert key-value list to internal state (when entering the new version) No more clashing record definitions. It would also be nice if the preprocessor could generate the from/to conversions.? Usual disclaimers apply. Best, Thomas From michael.eugene.turner@REDACTED Fri Aug 31 11:18:23 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Fri, 31 Aug 2012 18:18:23 +0900 Subject: [erlang-questions] Which EEP would you pick as the most important one to implement next? Message-ID: Since it seems to A Time for Big Questions on this mailing list. -michael turner From essen@REDACTED Fri Aug 31 11:30:20 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Fri, 31 Aug 2012 11:30:20 +0200 Subject: [erlang-questions] Which EEP would you pick as the most important one to implement next? In-Reply-To: References: Message-ID: <5040842C.6010702@ninenines.eu> On 08/31/2012 11:18 AM, Michael Turner wrote: > Since it seems to A Time for Big Questions on this mailing list. Sounds useful: http://www.erlang.org/eeps/eep-0035.html -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From jesper.louis.andersen@REDACTED Fri Aug 31 11:56:22 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 31 Aug 2012 11:56:22 +0200 Subject: [erlang-questions] Which EEP would you pick as the most important one to implement next? In-Reply-To: <5040842C.6010702@ninenines.eu> References: <5040842C.6010702@ninenines.eu> Message-ID: <06DFFE45-7A40-46B9-85F9-F58FD4A592AA@erlang-solutions.com> I'd definitely like EEP 16, the is_between/3 function (and usable in guards) I'd also like a clamp(X, Lo, Hi) with implementation: clamp(X, Lo, Hi) -> min(max(X, Lo), Hi). (Brace for mistakes - I might have gotten it wrong). The idea is to provide a limit on X. X will never go below Lo and will never go beyond Hi. It turns out that my code is using this quite a lot, and often in inner loops, so getting a fast variant could be useful. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen On Aug 31, 2012, at 11:30 AM, Lo?c Hoguin wrote: > On 08/31/2012 11:18 AM, Michael Turner wrote: >> Since it seems to A Time for Big Questions on this mailing list. > > Sounds useful: http://www.erlang.org/eeps/eep-0035.html > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ivan@REDACTED Fri Aug 31 11:59:18 2012 From: ivan@REDACTED (Ivan Uemlianin) Date: Fri, 31 Aug 2012 10:59:18 +0100 Subject: [erlang-questions] Which EEP would you pick as the most important one to implement next? In-Reply-To: <06DFFE45-7A40-46B9-85F9-F58FD4A592AA@erlang-solutions.com> References: <5040842C.6010702@ninenines.eu> <06DFFE45-7A40-46B9-85F9-F58FD4A592AA@erlang-solutions.com> Message-ID: <50408AF6.5070005@llaisdy.com> On 31/08/2012 10:56, Jesper Louis Andersen wrote: > clamp(X, Lo, Hi) -> > min(max(X, Lo), Hi). > > (Brace for mistakes - I might have gotten it wrong). The idea is to provide a limit on X. X will never go below Lo and will never go beyond Hi. It turns out that my code is using this quite a lot, and often in inner loops, so getting a fast variant could be useful. Looks interesting. Are the values integers or floats? What about a NIF? Ivan -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin "hilaritas excessum habere nequit" (Spinoza, Ethica, IV, XLII) ============================================================ From f@REDACTED Fri Aug 31 12:45:03 2012 From: f@REDACTED (Francesco Mazzoli) Date: Fri, 31 Aug 2012 11:45:03 +0100 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: <874nnjy0lx.wl%f@mazzo.li> Message-ID: <87vcfzgx1s.wl%f@mazzo.li> At Fri, 31 Aug 2012 11:57:53 +0400, Max Lapshin wrote: > This post has nothing to do with erlang. It is just an ancient question: waste > your time on documenting inner code or not. Again, did you read the post? It's well known that there are two kinds of documentation, "external" documentation for people writing clients of a module, and "internal" documentation for people maintaining the module itself. It's also well known that the division is simplistic; if the external documentation is silent about material points you have to read the internal documentation. The "internal" documentation, as you call it, is the only one that really matter. It's no waste of time. It rarely happens that after using a library for more than I few hours I don't need to look at the code to understand what the functions I'm using do. Moreover, as ROK says, I really don't see the distinction: the "internal" and "external" documentation should be one. > I never document such things, because it is silly: you spend more time on > rewriting documentation. I'd much rater have something that explains the internal design that some edoc that tells me more or less what functions will do. > And I perfectly read ffmpeg sources without any documentation inside them. Well, some code is pretty self explanatory. But most code isn't. -- Francesco * Often in error, never in doubt From schintke@REDACTED Fri Aug 31 13:17:04 2012 From: schintke@REDACTED (Florian Schintke) Date: Fri, 31 Aug 2012 13:17:04 +0200 Subject: [erlang-questions] Which EEP would you pick as the most important one to implement next? In-Reply-To: References: Message-ID: <20120831111704.GE21549@csr-pc9.zib.de> I like EEP19 as there seem to be a wide spectrum of nice use cases simplifying written code and thereby supporting the spirit of Erlang. http://www.erlang.org/eeps/eep-0019.html Florian From zabrane3@REDACTED Fri Aug 31 13:32:42 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 31 Aug 2012 13:32:42 +0200 Subject: [erlang-questions] Which EEP would you pick as the most important one to implement next? In-Reply-To: <20120831111704.GE21549@csr-pc9.zib.de> References: <20120831111704.GE21549@csr-pc9.zib.de> Message-ID: +1 for EEP19 Regards, Zabrane On Aug 31, 2012, at 1:17 PM, Florian Schintke wrote: > I like EEP19 as there seem to be a wide spectrum of nice use cases > simplifying written code and thereby supporting the spirit of Erlang. > > http://www.erlang.org/eeps/eep-0019.html > > Florian > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From demeshchuk@REDACTED Fri Aug 31 13:44:12 2012 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Fri, 31 Aug 2012 15:44:12 +0400 Subject: [erlang-questions] Which EEP would you pick as the most important one to implement next? In-Reply-To: <5040842C.6010702@ninenines.eu> References: <5040842C.6010702@ninenines.eu> Message-ID: Totally agree with EEP 35, looks like a great addition. On Fri, Aug 31, 2012 at 1:30 PM, Lo?c Hoguin wrote: > On 08/31/2012 11:18 AM, Michael Turner wrote: > >> Since it seems to A Time for Big Questions on this mailing list. >> > > Sounds useful: http://www.erlang.org/eeps/**eep-0035.html > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -- Best regards, Dmitry Demeshchuk -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Aug 31 14:19:32 2012 From: ok@REDACTED (ok@REDACTED) Date: Sat, 1 Sep 2012 00:19:32 +1200 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: <874nnjy0lx.wl%f@mazzo.li> Message-ID: <484a06b4ebed1da1e5bc07e83a5b24ba.squirrel@chasm.otago.ac.nz> > On Fri, Aug 31, 2012 at 11:36 AM, Francesco Mazzoli wrote: >> >> Did you read his post? His main point about documentation is that >> people tend >> to document functions but not data structures, which is something that I >> fully >> agree with. >> > > This post has nothing to do with erlang. It is just an ancient > question: waste your time on documenting inner code or not. Waste? In this case, code that is basically *good*, written by a competent Erlang programmer, has a serious performance limitation --- which may well have been a complete non-issue in its original context, but now that the code is available for use in other contexts, may be very important. And this is not documented *anywhere*, and it is hard to see in the code because it is swamped by other things. > I never document such things, because it is silly: you spend more time > on rewriting documentation. Nobody said documentation had to be *bulky*. In my Smalltalk system, I've found that the code changes like dreams as I refactor and extend it, but that the data structure invariants are far more stable. It is precisely the act of documenting those invariants that makes it *possible* to work so easily on the code. > And I perfectly read ffmpeg sources without any documentation inside them. Perhaps that is why the ffmpeg site says the software "work really well 99% of the time". That kind of failure rate is not tolerated in the Erlang world. Hmm. Why am I believing this? Ad fontes! Calculemus! Download ffmpeg sources. Count SLOC: 460,014 lines Count comments: 77,320 lines (excluding copyright notices & GNU licences) That's roughly one comment line for every 6 SLOC. This is *NOT* "sources without any documentation inside them". Returning to Erlang, I thought I had made the point that I wanted the structure of the code to reveal what is going on, but it doesn't. And for the specific example considered, 10 lines of commentary would have rendered roughly 200 SLOC of code pretty near superfluous to understanding. From rapsey@REDACTED Fri Aug 31 14:28:58 2012 From: rapsey@REDACTED (Rapsey) Date: Fri, 31 Aug 2012 14:28:58 +0200 Subject: [erlang-questions] Which EEP would you pick as the most important one to implement next? In-Reply-To: References: <5040842C.6010702@ninenines.eu> Message-ID: I'm surprised implementing frames is not an EEP. This is definitely the most lacking feature of erlang by far. Sergej On Fri, Aug 31, 2012 at 1:44 PM, Dmitry Demeshchuk wrote: > Totally agree with EEP 35, looks like a great addition. > > > On Fri, Aug 31, 2012 at 1:30 PM, Lo?c Hoguin wrote: > >> On 08/31/2012 11:18 AM, Michael Turner wrote: >> >>> Since it seems to A Time for Big Questions on this mailing list. >>> >> >> Sounds useful: http://www.erlang.org/eeps/**eep-0035.html >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > > > -- > Best regards, > Dmitry Demeshchuk > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Aug 31 14:42:05 2012 From: ok@REDACTED (ok@REDACTED) Date: Sat, 1 Sep 2012 00:42:05 +1200 Subject: [erlang-questions] Which EEP would you pick as the most important one to implement next? In-Reply-To: References: <5040842C.6010702@ninenines.eu> Message-ID: <87e026c04bdafbec879831e969e632a4.squirrel@chasm.otago.ac.nz> > I'm surprised implementing frames is not an EEP. This is definitely the > most lacking feature of erlang by far. It would have been, but by the time I was ready to submit it as one, there was a hard requirement that EEPS be marked up in a basically undefined formalism, and a number of things I needed to mark up, I could not figure out how to mark up. That requirement terminated my contribution to EEPs. I learned RUNOFF. I learned SCRIBE. I learned Troff. I learned TeX. I learned LaTeX. I learned Lout. I learned SGML, HTML, XML, XHTML, ... I'm willing to learn any markup notation that has a tolerably complete tolerably accurate manual. Markdown, however, does not. From vladdu55@REDACTED Fri Aug 31 14:55:51 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 31 Aug 2012 14:55:51 +0200 Subject: [erlang-questions] Which EEP would you pick as the most important one to implement next? In-Reply-To: <87e026c04bdafbec879831e969e632a4.squirrel@chasm.otago.ac.nz> References: <5040842C.6010702@ninenines.eu> <87e026c04bdafbec879831e969e632a4.squirrel@chasm.otago.ac.nz> Message-ID: Hi Richard, On Fri, Aug 31, 2012 at 2:42 PM, wrote: >> I'm surprised implementing frames is not an EEP. This is definitely the >> most lacking feature of erlang by far. > > It would have been, but by the time I was ready to submit it > as one, there was a hard requirement that EEPS be marked up > in a basically undefined formalism, and a number of things I > needed to mark up, I could not figure out how to mark up. > That requirement terminated my contribution to EEPs. > > I learned RUNOFF. I learned SCRIBE. I learned Troff. I > learned TeX. I learned LaTeX. I learned Lout. I learned > SGML, HTML, XML, XHTML, ... I'm willing to learn any markup > notation that has a tolerably complete tolerably accurate > manual. Markdown, however, does not. This is certainly not what the EEP maintainers would really like to see, but any valid HTML that can be found inside a tag is also valid Markdown. The weird formalism are just shortcuts for commonly used structures. So technically, you could write contents of the EEP in plain HTML and it will follow the letter of the EEP guidelines. You only have to add the EEP headers and some delimiters. Please don't let this detail delay a most needed proposal! best regards, Vlad From thomas.jarvstrand@REDACTED Fri Aug 31 15:59:45 2012 From: thomas.jarvstrand@REDACTED (=?ISO-8859-1?Q?Thomas_J=E4rvstrand?=) Date: Fri, 31 Aug 2012 15:59:45 +0200 Subject: [erlang-questions] Licensing an Erlang/Emacs Lisp project Message-ID: Hi, I'm currently working on a project that is a combination of Erlang and Emacs Lisp. The project is currently open sourced under the LGPL but I recently realized that some of the submodules I use are under GPL (the elisp ones) and so I believe I have to change my licensing. My problem is that I have no idea what license to choose. Part of my code will be running in Erlang and part of the code will be running in Elisp inside an Emacs instance. The submodules are not actually shipped *with*the project if that makes any difference, but are fetched upon building the project. Specifically, in Erlang I currently have one submodule licensed under the MIT license, and one under Apache v2. In Elisp I'm using 3 submodules, all of which are licensed under GPLv3. Is there any way that I can cook this soup? I hear that Erlang doesn't really work that well with GPL so should I pull out the Erlang parts of the project and license them separately? Can I then use these as part of my project? The Elisp code will be communicating with the Erlang parts through a rest-interface if that makes any difference. All these licenses confuse me to no end, so it'd be great to get some advice. Basically my project a bunch of porcelain code that binds together and configures a bunch of useful tools for erlang development. The whole point of it is to be an easy-to-setup solution, so I'd really not like to have to resort to "download this from here, configure this way, rinse, repeat". I have no issue with licensing my project under GPL, but I would like a model that is compatible with what I currently use and gives me freedom to add new parts to the project in the future. I would be thankful for any help Sincerely Thomas J?rvstrand -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Fri Aug 31 16:20:31 2012 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 31 Aug 2012 16:20:31 +0200 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: Message-ID: On Fri, Aug 31, 2012 at 8:20 AM, Richard O'Keefe wrote: > We've just had a thread about what people like about Erlang. > We also had the announcement of TinyMQ. > So I'm going to use this as an example of what's *really* > wrong with Erlang. > > Don't get me wrong. I endorse everything everyone else has > said in favour of Erlang. Erlang is like democracy: the worst > thing in its class except for all the others, and something > that is increasingly imitated by people who just don't get > some of the fundamental things about it. > > I also endorse what people have said in praise of TinyMQ. > There are lots of things that it does right: > - there is a README > - there are EDoc comments with @specs for the public > interface > - the functions and variables are named well enough that > I was never in doubt about what any part of the code was > up to, at least not for longer than a second or two > - the hard work of process management is delegated to OTP > behaviours > At this point, it's looking better than anything I've written. > > Make no mistake: I am not saying that Erlang or TinyMQ are *bad*. > They are good things; I'm just ranting somewhat vaguely about > why they should be better. > > > LUMPS OF INDISTINGUISHABLE CODE. > > Up to a certain level of hand-waving, TinyMQ can be roughly > understood thus: > The TinyMQ *system* is a monitor > guarding a dictionary mapping strings to channnels, > where > a channel is a monitor > guarding a bag of subscribers and > a sliding window of {Message, Timestamp} pairs. > > YOU CANNOT SEE THIS AT A GLANCE. > > This is not Evan Miller's fault. *Anything* you write in > Erlang is going to end up as lumps of indistinguishable code, > because there is nothing else for it to be. > > This is also true in C, C++, Java, C#, Javascript, Go, > Eiffel, Smalltalk, Prolog, Haskell, Clean, SML, ..., > not to mention Visual Basic and Fortran. > > Almost the only languages I know where it doesn't *have* to > be true are Lisp, Scheme, and Lisp-Flavoured Erlang. Arguably > Prolog *could* be in this group, but in practice it usually is > in the other camp. Thanks to the preprocessor, C *can* be > made rather more scrutable, but for some reason this is frowned on. > > There's the e2 project (http://e2project.org) which is a step > in a good direction, but it doesn't do much about this problem. > A version of TinyMQ using e2_service instead of gen_server > would in fact exacerbate the problem by mushing > handle_call/3, handle_cast/2, and handle_info/2 into one > function, turning three lumps into one bigger lump. > > LUMPS OF DATA. > > Take tinymq_channel_controller as an example. > Using an OTP behaviour means that all six dimensions of the state > are mushed together in one data structure. This goes a long way > towards hiding the fact that > > supervisor, channel, and max_age are never changed > messages, subscribers, and last_pull *are* changed. > > One teeny tiny step here would be to offer an alternative set of > callbacks for some behaviours where the "state" is separated into > immutable "context" and mutable "state", so that it is obvious > *by construction* that the context information *can't* be changed. > > Another option would be to have some way of annotation in a > -record declaration that a field cannot be updated. > > I prefer the segregation approach on the grounds of no language > change being needed and the improved efficiency of not copying > fields that can't have changed. Others might prefer the revise > -record approach on the grounds of not having to change or > duplicate the OTP behaviours. > > I had to reach each file in detail > - to find that certain fields *happened* not to be changed > - to understand the design well enough to tell that this was > almost certainly deliberate. > > WE DOCUMENT THE WRONG THINGS. > > It's well known that there are two kinds of documentation, > "external" documentation for people writing clients of a module, > and "internal" documentation for people maintaining the module > itself. It's also well known that the division is simplistic; > if the external documentation is silent about material points > you have to read the internal documentation. Thank you. I was wondering - perhaps it is wrong to publish source code - If you have to read the internal documentation to understand the external behavior of a program then the external documentation is not good enough. Dare you publish only binary code and documentation of the interfaces? I find that large numbers of programs cannot be understood by reading the documentation of the interfaces - you have to read the internal documentation and (horrors) the code. Reading code is no fun - since you always wonder *why* they wrote it that way, and not some other way and you get tempted to change it. I think you should only publish binary code and external documentation. If a user wants to know how to use the code and have to ask then you have failed to document your code. If a user wants to see the code, not because they wish to use it, but because they wish to see how you solved the problem - then you can let them see the code. Programs and code are supposed to be black-boxes. If you have to open the black box and peep inside then they are not black boxes any more. The practice of reading code to figure out how to use the code is crazy and an incredible waste of time. When programming I spend most of my time fixing things that should not be broken and figuring out stuff that should be documented. I have said many times - code is the result of research - It might take me hours of research to write twenty lines of code.If I publish the 20 lines and throw away the research I am doing nobidy a favor. Programs should be released with all the necessary documents needed to understand the code. /Joe > > In languages like Prolog and Erlang and Scheme where you build > data structures out of existing "universal" types and have no > data structure declarations, we tend to document procedures > but not data. This is backwards. If you understand the data, > and especially its invariants, the code is often pretty obvious. > > There are two examples of this in TinyMQ. One is specific to > TinyMQ. The other other is nearly universal in Erlang practice. > > Erlang systems are made of lots of processes sending messages > to each other. Joe Armstrong has often said THINK ABOUT THE > PROTOCOLS. But Erlang programmers very seldom *write* about > the protocols. > > Using the OTP behaviours, a "concurrent object" is implemented > as a module with a bunch of interface functions that forward > messages through the OTP layer to the callback code managed by > whatever behaviour it is. This protocol is unique to each kind > of concurrent object. It's often generated in one module (the > one with the interface functions) and consumed in another (the > one with the callback code), as it is in TinyMQ. And it's not > documented. > > It is possible to reconstruct this protocol by reading the code > in detail and noting down what you see. It is troublesome when, > as in TinyMQ, the two modules disagree about the protocol. It's > clear that _something_ is wrong, but what, exactly? > > For example, tinymq_controller has a case > handle_cast({set_max_age, newMaxAge}, State) -> > but this is the only occurrence of set_max_age anywhere in TinyMQ. > Is its presence in tinymq_controller an example of dead code, > or is its absence from the rest of the application an example > of missing code? The same question can be asked about 'expire' > (which would forget a channel without making it actually go away, > if it could ever be invoked, which it can't.) > > Almost as soon as I started reading Erlang code many years ago > it seemed obvious to me that documenting (and if possible, type > checking) these internal protocols was a very important part of > Erlang internal documentation. There must be something wrong > with my brain, because other people don't seem to feel this lack > anywhere nearly as strongly as I do. I think Joe Armstrong sort > of sees this at the next level up or he would never have invented > UBF. > > But Occam, Go, and Sing# have typed channels, so they *are* > addressing the issue, and *do* have a natural central point to > document what the alternatives of an internal protocol signify. > > Another documentation failure is that we fail to document what > is not there. In TinyMQ, a channel automatically comes into > existence when you try to use it. Perhaps as a consequence of > this, there is no way to shut a channel down. In TinyMQ, old > messages are not removed from a channel when they expire, but > the next time someone does a 'subscribe' (waves hands) or a 'poll' > or a 'push' *after* they expire. So if processes stop sending > and requesting messages to some channel, the last few messages, > no matter how large, may hang around forever. I'm sure there > is a reason, but because it's a reason for something *not* being > there, there's no obvious place to hang the comment, and there > isn't one. (Except for the dead 'expire' clause mentioned above.) > > IT'S HARD TO SPOT SALIENT DETAIL IN A SEA OF GLUE CODE. > > The central fact about TinyMQ is that it holds the messages of > a channel in a simple list of {Message, Timestamp} pairs. As > a result, every operation on the data takes time linear in the > current size. > > This is not stated anywhere in any comments nor in the README. > You have to read the code in detail to discover this. And it > is a rather nasty surprise. If a channel holds N messages, > the operations *can* be done in O(log(N)) time. (I believe it > is possible to do even better.) Some sliding window applications > have a bound on the number of elements in the window. This one > has a bound on the age of elements, but they could arrive at a > very high rate, so N *could* get large. > > It is very easy to implement the necessary operations using lists, > so much so that they are present in several copies. Revising the > TinyMQ implementation to work better with long queues would be > harder than necessary because of this. And this goes un-noticed > because there is so much glue code for the guts to get lost in. > > Given that Evan Miller took the trouble to use library components > for structuring this application, why didn't he take the next step, > and use the existing 'sliding window' library data structure? > > Because there is none! > > Yet sliding windows of one sort or another have come up before in > this mailing list. Perhaps we should have a Wiki page on > trapexit to gather requirements for one or more sliding window > libraries. Or perhaps not. "true religion jeans for women" -- > what has that or "Cheap Nike Shoes" to do with Erlang/OTP > (http://www.trapexit.org/forum/viewforum.php?f=20)? > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Fri Aug 31 16:23:15 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 31 Aug 2012 15:23:15 +0100 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: Message-ID: <1C96D723-BE34-41CB-A550-A269A70AA2AA@gmail.com> On 31 Aug 2012, at 15:20, Joe Armstrong wrote: > Programs and code are supposed to be black-boxes. If you have to open > the black box and > peep inside then they are not black boxes any more. > > The practice of reading code to figure out how to use the code is > crazy and an incredible > waste of time. > +1 > When programming I spend most of my time fixing things that should not be broken > and figuring out stuff that should be documented. > +1, much to my annoyance > I have said many times - code is the result of research - It might > take me hours of research > to write twenty lines of code.If I publish the 20 lines and throw away > the research I am doing nobidy > a favor. I wish this was brainwashed into every programmer at birth. :) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 235 bytes Desc: Message signed with OpenPGP using GPGMail URL: From ivan@REDACTED Fri Aug 31 16:25:31 2012 From: ivan@REDACTED (Ivan Uemlianin) Date: Fri, 31 Aug 2012 15:25:31 +0100 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: <1C96D723-BE34-41CB-A550-A269A70AA2AA@gmail.com> References: <1C96D723-BE34-41CB-A550-A269A70AA2AA@gmail.com> Message-ID: <5040C95B.3050701@llaisdy.com> On 31/08/2012 15:23, Tim Watson wrote: > > On 31 Aug 2012, at 15:20, Joe Armstrong wrote: >>... >> I have said many times - code is the result of research - It might >> take me hours of research >> to write twenty lines of code.If I publish the 20 lines and throw away >> the research I am doing nobidy >> a favor. > > > I wish this was brainwashed into every programmer at birth. :) ... and every manager! -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin "hilaritas excessum habere nequit" (Spinoza, Ethica, IV, XLII) ============================================================ From vinoski@REDACTED Fri Aug 31 16:39:35 2012 From: vinoski@REDACTED (Steve Vinoski) Date: Fri, 31 Aug 2012 10:39:35 -0400 Subject: [erlang-questions] Which EEP would you pick as the most important one to implement next? In-Reply-To: References: <5040842C.6010702@ninenines.eu> <87e026c04bdafbec879831e969e632a4.squirrel@chasm.otago.ac.nz> Message-ID: On Fri, Aug 31, 2012 at 8:55 AM, Vlad Dumitrescu wrote: > Hi Richard, > > On Fri, Aug 31, 2012 at 2:42 PM, wrote: >>> I'm surprised implementing frames is not an EEP. This is definitely the >>> most lacking feature of erlang by far. >> >> It would have been, but by the time I was ready to submit it >> as one, there was a hard requirement that EEPS be marked up >> in a basically undefined formalism, and a number of things I >> needed to mark up, I could not figure out how to mark up. >> That requirement terminated my contribution to EEPs. >> >> I learned RUNOFF. I learned SCRIBE. I learned Troff. I >> learned TeX. I learned LaTeX. I learned Lout. I learned >> SGML, HTML, XML, XHTML, ... I'm willing to learn any markup >> notation that has a tolerably complete tolerably accurate >> manual. Markdown, however, does not. > > This is certainly not what the EEP maintainers would really like to > see, but any valid HTML that can be found inside a tag is also > valid Markdown. The weird formalism are just shortcuts for commonly > used structures. So technically, you could write contents of the EEP > in plain HTML and it will follow the letter of the EEP guidelines. You > only have to add the EEP headers and some delimiters. > > Please don't let this detail delay a most needed proposal! Agreed. Also, Richard, you might try writing your proposal in something other than markdown, something that gives you what you're aiming for, and then using pandoc to convert it to markdown for submission -- see http://johnmacfarlane.net/pandoc/ if you're not familiar with pandoc. --steve From max.lapshin@REDACTED Fri Aug 31 17:18:31 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 31 Aug 2012 19:18:31 +0400 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: <5040C95B.3050701@llaisdy.com> References: <1C96D723-BE34-41CB-A550-A269A70AA2AA@gmail.com> <5040C95B.3050701@llaisdy.com> Message-ID: Guys. It is nice in theory, but I have other practice. For example, there is such a protocol, like MPEG-TS (in fact, problem is deeper, it is a whole set of protocols). There is no single program, that implements it for 100%. Every program lacks something. Without source code it is impossible to find out, what is the problem. So I consider, that "black-box" approach doesn't work, because every code has bugs. If you can't modify this code, don't use it in your program at all. From puzza007@REDACTED Fri Aug 31 17:19:59 2012 From: puzza007@REDACTED (Paul Oliver) Date: Fri, 31 Aug 2012 11:19:59 -0400 Subject: [erlang-questions] Which EEP would you pick as the most important one to implement next? In-Reply-To: References: <5040842C.6010702@ninenines.eu> Message-ID: +1 On Aug 31, 2012 8:29 AM, "Rapsey" wrote: > I'm surprised implementing frames is not an EEP. This is definitely the > most lacking feature of erlang by far. > > > Sergej > > On Fri, Aug 31, 2012 at 1:44 PM, Dmitry Demeshchuk wrote: > >> Totally agree with EEP 35, looks like a great addition. >> >> >> On Fri, Aug 31, 2012 at 1:30 PM, Lo?c Hoguin wrote: >> >>> On 08/31/2012 11:18 AM, Michael Turner wrote: >>> >>>> Since it seems to A Time for Big Questions on this mailing list. >>>> >>> >>> Sounds useful: http://www.erlang.org/eeps/**eep-0035.html >>> >>> -- >>> Lo?c Hoguin >>> Erlang Cowboy >>> Nine Nines >>> http://ninenines.eu >>> >>> ______________________________**_________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/**listinfo/erlang-questions >>> >> >> >> >> -- >> Best regards, >> Dmitry Demeshchuk >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Fri Aug 31 17:24:24 2012 From: ivan@REDACTED (Ivan Uemlianin) Date: Fri, 31 Aug 2012 16:24:24 +0100 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: Message-ID: <5040D728.4010302@llaisdy.com> One point about documentation is the language it's written in. If the documentation was written in Greek but you only understand Turkish, the documentation won't be much good. On 31/08/2012 15:20, Joe Armstrong wrote: > On Fri, Aug 31, 2012 at 8:20 AM, Richard O'Keefe wrote: >> We've just had a thread about what people like about Erlang. >> We also had the announcement of TinyMQ. >> So I'm going to use this as an example of what's *really* >> wrong with Erlang. >> >> Don't get me wrong. I endorse everything everyone else has >> said in favour of Erlang. Erlang is like democracy: the worst >> thing in its class except for all the others, and something >> that is increasingly imitated by people who just don't get >> some of the fundamental things about it. >> >> I also endorse what people have said in praise of TinyMQ. >> There are lots of things that it does right: >> - there is a README >> - there are EDoc comments with @specs for the public >> interface >> - the functions and variables are named well enough that >> I was never in doubt about what any part of the code was >> up to, at least not for longer than a second or two >> - the hard work of process management is delegated to OTP >> behaviours >> At this point, it's looking better than anything I've written. >> >> Make no mistake: I am not saying that Erlang or TinyMQ are *bad*. >> They are good things; I'm just ranting somewhat vaguely about >> why they should be better. >> >> >> LUMPS OF INDISTINGUISHABLE CODE. >> >> Up to a certain level of hand-waving, TinyMQ can be roughly >> understood thus: >> The TinyMQ *system* is a monitor >> guarding a dictionary mapping strings to channnels, >> where >> a channel is a monitor >> guarding a bag of subscribers and >> a sliding window of {Message, Timestamp} pairs. >> >> YOU CANNOT SEE THIS AT A GLANCE. >> >> This is not Evan Miller's fault. *Anything* you write in >> Erlang is going to end up as lumps of indistinguishable code, >> because there is nothing else for it to be. >> >> This is also true in C, C++, Java, C#, Javascript, Go, >> Eiffel, Smalltalk, Prolog, Haskell, Clean, SML, ..., >> not to mention Visual Basic and Fortran. >> >> Almost the only languages I know where it doesn't *have* to >> be true are Lisp, Scheme, and Lisp-Flavoured Erlang. Arguably >> Prolog *could* be in this group, but in practice it usually is >> in the other camp. Thanks to the preprocessor, C *can* be >> made rather more scrutable, but for some reason this is frowned on. >> >> There's the e2 project (http://e2project.org) which is a step >> in a good direction, but it doesn't do much about this problem. >> A version of TinyMQ using e2_service instead of gen_server >> would in fact exacerbate the problem by mushing >> handle_call/3, handle_cast/2, and handle_info/2 into one >> function, turning three lumps into one bigger lump. >> >> LUMPS OF DATA. >> >> Take tinymq_channel_controller as an example. >> Using an OTP behaviour means that all six dimensions of the state >> are mushed together in one data structure. This goes a long way >> towards hiding the fact that >> >> supervisor, channel, and max_age are never changed >> messages, subscribers, and last_pull *are* changed. >> >> One teeny tiny step here would be to offer an alternative set of >> callbacks for some behaviours where the "state" is separated into >> immutable "context" and mutable "state", so that it is obvious >> *by construction* that the context information *can't* be changed. >> >> Another option would be to have some way of annotation in a >> -record declaration that a field cannot be updated. >> >> I prefer the segregation approach on the grounds of no language >> change being needed and the improved efficiency of not copying >> fields that can't have changed. Others might prefer the revise >> -record approach on the grounds of not having to change or >> duplicate the OTP behaviours. >> >> I had to reach each file in detail >> - to find that certain fields *happened* not to be changed >> - to understand the design well enough to tell that this was >> almost certainly deliberate. >> >> WE DOCUMENT THE WRONG THINGS. >> >> It's well known that there are two kinds of documentation, >> "external" documentation for people writing clients of a module, >> and "internal" documentation for people maintaining the module >> itself. It's also well known that the division is simplistic; >> if the external documentation is silent about material points >> you have to read the internal documentation. > > Thank you. > > I was wondering - perhaps it is wrong to publish source code - If you > have to read > the internal documentation to understand the external behavior of a program then > the external documentation is not good enough. > > Dare you publish only binary code and documentation of the interfaces? > > I find that large numbers of programs cannot be understood by reading > the documentation > of the interfaces - you have to read the internal documentation and > (horrors) the code. > > Reading code is no fun - since you always wonder *why* they wrote it > that way, and not some > other way and you get tempted to change it. > > I think you should only publish binary code and external > documentation. If a user wants > to know how to use the code and have to ask then you have failed to > document your code. > > If a user wants to see the code, not because they wish to use it, but > because they wish > to see how you solved the problem - then you can let them see the code. > > Programs and code are supposed to be black-boxes. If you have to open > the black box and > peep inside then they are not black boxes any more. > > The practice of reading code to figure out how to use the code is > crazy and an incredible > waste of time. > > When programming I spend most of my time fixing things that should not be broken > and figuring out stuff that should be documented. > > I have said many times - code is the result of research - It might > take me hours of research > to write twenty lines of code.If I publish the 20 lines and throw away > the research I am doing nobidy > a favor. > > Programs should be released with all the necessary documents needed to > understand the code. > > /Joe > > > > >> >> In languages like Prolog and Erlang and Scheme where you build >> data structures out of existing "universal" types and have no >> data structure declarations, we tend to document procedures >> but not data. This is backwards. If you understand the data, >> and especially its invariants, the code is often pretty obvious. >> >> There are two examples of this in TinyMQ. One is specific to >> TinyMQ. The other other is nearly universal in Erlang practice. >> >> Erlang systems are made of lots of processes sending messages >> to each other. Joe Armstrong has often said THINK ABOUT THE >> PROTOCOLS. But Erlang programmers very seldom *write* about >> the protocols. >> >> Using the OTP behaviours, a "concurrent object" is implemented >> as a module with a bunch of interface functions that forward >> messages through the OTP layer to the callback code managed by >> whatever behaviour it is. This protocol is unique to each kind >> of concurrent object. It's often generated in one module (the >> one with the interface functions) and consumed in another (the >> one with the callback code), as it is in TinyMQ. And it's not >> documented. >> >> It is possible to reconstruct this protocol by reading the code >> in detail and noting down what you see. It is troublesome when, >> as in TinyMQ, the two modules disagree about the protocol. It's >> clear that _something_ is wrong, but what, exactly? >> >> For example, tinymq_controller has a case >> handle_cast({set_max_age, newMaxAge}, State) -> >> but this is the only occurrence of set_max_age anywhere in TinyMQ. >> Is its presence in tinymq_controller an example of dead code, >> or is its absence from the rest of the application an example >> of missing code? The same question can be asked about 'expire' >> (which would forget a channel without making it actually go away, >> if it could ever be invoked, which it can't.) >> >> Almost as soon as I started reading Erlang code many years ago >> it seemed obvious to me that documenting (and if possible, type >> checking) these internal protocols was a very important part of >> Erlang internal documentation. There must be something wrong >> with my brain, because other people don't seem to feel this lack >> anywhere nearly as strongly as I do. I think Joe Armstrong sort >> of sees this at the next level up or he would never have invented >> UBF. >> >> But Occam, Go, and Sing# have typed channels, so they *are* >> addressing the issue, and *do* have a natural central point to >> document what the alternatives of an internal protocol signify. >> >> Another documentation failure is that we fail to document what >> is not there. In TinyMQ, a channel automatically comes into >> existence when you try to use it. Perhaps as a consequence of >> this, there is no way to shut a channel down. In TinyMQ, old >> messages are not removed from a channel when they expire, but >> the next time someone does a 'subscribe' (waves hands) or a 'poll' >> or a 'push' *after* they expire. So if processes stop sending >> and requesting messages to some channel, the last few messages, >> no matter how large, may hang around forever. I'm sure there >> is a reason, but because it's a reason for something *not* being >> there, there's no obvious place to hang the comment, and there >> isn't one. (Except for the dead 'expire' clause mentioned above.) >> >> IT'S HARD TO SPOT SALIENT DETAIL IN A SEA OF GLUE CODE. >> >> The central fact about TinyMQ is that it holds the messages of >> a channel in a simple list of {Message, Timestamp} pairs. As >> a result, every operation on the data takes time linear in the >> current size. >> >> This is not stated anywhere in any comments nor in the README. >> You have to read the code in detail to discover this. And it >> is a rather nasty surprise. If a channel holds N messages, >> the operations *can* be done in O(log(N)) time. (I believe it >> is possible to do even better.) Some sliding window applications >> have a bound on the number of elements in the window. This one >> has a bound on the age of elements, but they could arrive at a >> very high rate, so N *could* get large. >> >> It is very easy to implement the necessary operations using lists, >> so much so that they are present in several copies. Revising the >> TinyMQ implementation to work better with long queues would be >> harder than necessary because of this. And this goes un-noticed >> because there is so much glue code for the guts to get lost in. >> >> Given that Evan Miller took the trouble to use library components >> for structuring this application, why didn't he take the next step, >> and use the existing 'sliding window' library data structure? >> >> Because there is none! >> >> Yet sliding windows of one sort or another have come up before in >> this mailing list. Perhaps we should have a Wiki page on >> trapexit to gather requirements for one or more sliding window >> libraries. Or perhaps not. "true religion jeans for women" -- >> what has that or "Cheap Nike Shoes" to do with Erlang/OTP >> (http://www.trapexit.org/forum/viewforum.php?f=20)? >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin "hilaritas excessum habere nequit" (Spinoza, Ethica, IV, XLII) ============================================================ From ulf@REDACTED Fri Aug 31 18:05:26 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 31 Aug 2012 18:05:26 +0200 Subject: [erlang-questions] Licensing an Erlang/Emacs Lisp project In-Reply-To: References: Message-ID: On 31 Aug 2012, at 15:59, Thomas J?rvstrand wrote: > I'm currently working on a project that is a combination of Erlang and Emacs Lisp. The project is currently open sourced under the LGPL but I recently realized that some of the submodules I use are under GPL (the elisp ones) and so I believe I have to change my licensing. If you include code that is GPL, then you are pretty much stuck with GPL:ing your whole project. You don't need to worry much about Erlang, though. It remains under its own license (nothing you do can alter the license of somebody else's components). Erlang's license is not contageous, and you are allowed to write Erlang code and release it as GPL (popular examples are PropEr and Ejabberd). If you want to get fancy, you can meditate over the Mozilla Public License 2.0, and its wording on compatibility with (L)GPL, but if you're ok with GPL, perhaps you should just go with that. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From emmiller@REDACTED Fri Aug 31 19:42:17 2012 From: emmiller@REDACTED (Evan Miller) Date: Fri, 31 Aug 2012 12:42:17 -0500 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: Message-ID: Richard, Thanks for your comments. To preface, I plead guilty to charges of gross negligence in failing to document TinyMQ's internals. This was laziness on my part. I released TinyMQ only because I felt guilty for sitting on the code for about a year. Like many open-source programmers, I have a lot of demands on my attention, and it is not clear in advance what documentation is actually worth writing. The @spec and @doc strings for the public API seemed like a good start. But if it turned out that no one was interested in using the library in the first place, why should I bother documenting internal protocols and data structures? I've wasted many hours in the past documenting, refactoring, and generally cleaning up application internals for the benefit of nebulous "others", only to receive zero patches and no indication that any of my efforts were of any assistance to anyone. So in the spirit of your capitalized complaints, I will just say: ALL YOU HAVE TO DO IS ASK Want to know about the big-O performance characteristics? Just ask. Want to know how channel creation works? Just ask. As a lazy person, if a few people ask me the same thing I'll usually add a note to the README in order to avert future emails from strangers. We all like a well-documented project, but without feedback and communication it is not clear where one's efforts are best spent on a project that doesn't have an explicit client. If I knew in advance who would be using and reading the code (i.e. if I wrote this code for an employer), I would put more effort into writing documents for that specific audience. But as a rule, if I am just putting some code "out there", I would rather wait and see what people would like to know about, rather than pre-emptively document every thought that has ever occurred to me relating to the code base. Now, I know you were not trying to pick on TinyMQ, and your interest is more in how Erlang tends to result in lumps of code that obscure key characteristics of the application. I agree with the assessment, but I am not quite as hopeless about the situation. I would like to see the development of graphical tools that let you see in an instant how applications are structured and how they behave. I am thinking of something like Pman on steroids, where I can *watch* messages travel between processes, *inspect* gen_server state, and *test* the system by seeing the result of single function calls or many (load-testing). I'd like to be able to do all this with my mouse, and generally get the feeling that I am watching the operation of a machine that *shows* me how messages are passed, processes are created, and state is updated. Did anyone else ever play Marble Drop from Maxis in the late 90s? That is the kind of interface I would like to see for the Erlang run-time. For now, I'll update the README. Evan On Fri, Aug 31, 2012 at 1:20 AM, Richard O'Keefe wrote: > We've just had a thread about what people like about Erlang. > We also had the announcement of TinyMQ. > So I'm going to use this as an example of what's *really* > wrong with Erlang. > > Don't get me wrong. I endorse everything everyone else has > said in favour of Erlang. Erlang is like democracy: the worst > thing in its class except for all the others, and something > that is increasingly imitated by people who just don't get > some of the fundamental things about it. > > I also endorse what people have said in praise of TinyMQ. > There are lots of things that it does right: > - there is a README > - there are EDoc comments with @specs for the public > interface > - the functions and variables are named well enough that > I was never in doubt about what any part of the code was > up to, at least not for longer than a second or two > - the hard work of process management is delegated to OTP > behaviours > At this point, it's looking better than anything I've written. > > Make no mistake: I am not saying that Erlang or TinyMQ are *bad*. > They are good things; I'm just ranting somewhat vaguely about > why they should be better. > > > LUMPS OF INDISTINGUISHABLE CODE. > > Up to a certain level of hand-waving, TinyMQ can be roughly > understood thus: > The TinyMQ *system* is a monitor > guarding a dictionary mapping strings to channnels, > where > a channel is a monitor > guarding a bag of subscribers and > a sliding window of {Message, Timestamp} pairs. > > YOU CANNOT SEE THIS AT A GLANCE. > > This is not Evan Miller's fault. *Anything* you write in > Erlang is going to end up as lumps of indistinguishable code, > because there is nothing else for it to be. > > This is also true in C, C++, Java, C#, Javascript, Go, > Eiffel, Smalltalk, Prolog, Haskell, Clean, SML, ..., > not to mention Visual Basic and Fortran. > > Almost the only languages I know where it doesn't *have* to > be true are Lisp, Scheme, and Lisp-Flavoured Erlang. Arguably > Prolog *could* be in this group, but in practice it usually is > in the other camp. Thanks to the preprocessor, C *can* be > made rather more scrutable, but for some reason this is frowned on. > > There's the e2 project (http://e2project.org) which is a step > in a good direction, but it doesn't do much about this problem. > A version of TinyMQ using e2_service instead of gen_server > would in fact exacerbate the problem by mushing > handle_call/3, handle_cast/2, and handle_info/2 into one > function, turning three lumps into one bigger lump. > > LUMPS OF DATA. > > Take tinymq_channel_controller as an example. > Using an OTP behaviour means that all six dimensions of the state > are mushed together in one data structure. This goes a long way > towards hiding the fact that > > supervisor, channel, and max_age are never changed > messages, subscribers, and last_pull *are* changed. > > One teeny tiny step here would be to offer an alternative set of > callbacks for some behaviours where the "state" is separated into > immutable "context" and mutable "state", so that it is obvious > *by construction* that the context information *can't* be changed. > > Another option would be to have some way of annotation in a > -record declaration that a field cannot be updated. > > I prefer the segregation approach on the grounds of no language > change being needed and the improved efficiency of not copying > fields that can't have changed. Others might prefer the revise > -record approach on the grounds of not having to change or > duplicate the OTP behaviours. > > I had to reach each file in detail > - to find that certain fields *happened* not to be changed > - to understand the design well enough to tell that this was > almost certainly deliberate. > > WE DOCUMENT THE WRONG THINGS. > > It's well known that there are two kinds of documentation, > "external" documentation for people writing clients of a module, > and "internal" documentation for people maintaining the module > itself. It's also well known that the division is simplistic; > if the external documentation is silent about material points > you have to read the internal documentation. > > In languages like Prolog and Erlang and Scheme where you build > data structures out of existing "universal" types and have no > data structure declarations, we tend to document procedures > but not data. This is backwards. If you understand the data, > and especially its invariants, the code is often pretty obvious. > > There are two examples of this in TinyMQ. One is specific to > TinyMQ. The other other is nearly universal in Erlang practice. > > Erlang systems are made of lots of processes sending messages > to each other. Joe Armstrong has often said THINK ABOUT THE > PROTOCOLS. But Erlang programmers very seldom *write* about > the protocols. > > Using the OTP behaviours, a "concurrent object" is implemented > as a module with a bunch of interface functions that forward > messages through the OTP layer to the callback code managed by > whatever behaviour it is. This protocol is unique to each kind > of concurrent object. It's often generated in one module (the > one with the interface functions) and consumed in another (the > one with the callback code), as it is in TinyMQ. And it's not > documented. > > It is possible to reconstruct this protocol by reading the code > in detail and noting down what you see. It is troublesome when, > as in TinyMQ, the two modules disagree about the protocol. It's > clear that _something_ is wrong, but what, exactly? > > For example, tinymq_controller has a case > handle_cast({set_max_age, newMaxAge}, State) -> > but this is the only occurrence of set_max_age anywhere in TinyMQ. > Is its presence in tinymq_controller an example of dead code, > or is its absence from the rest of the application an example > of missing code? The same question can be asked about 'expire' > (which would forget a channel without making it actually go away, > if it could ever be invoked, which it can't.) > > Almost as soon as I started reading Erlang code many years ago > it seemed obvious to me that documenting (and if possible, type > checking) these internal protocols was a very important part of > Erlang internal documentation. There must be something wrong > with my brain, because other people don't seem to feel this lack > anywhere nearly as strongly as I do. I think Joe Armstrong sort > of sees this at the next level up or he would never have invented > UBF. > > But Occam, Go, and Sing# have typed channels, so they *are* > addressing the issue, and *do* have a natural central point to > document what the alternatives of an internal protocol signify. > > Another documentation failure is that we fail to document what > is not there. In TinyMQ, a channel automatically comes into > existence when you try to use it. Perhaps as a consequence of > this, there is no way to shut a channel down. In TinyMQ, old > messages are not removed from a channel when they expire, but > the next time someone does a 'subscribe' (waves hands) or a 'poll' > or a 'push' *after* they expire. So if processes stop sending > and requesting messages to some channel, the last few messages, > no matter how large, may hang around forever. I'm sure there > is a reason, but because it's a reason for something *not* being > there, there's no obvious place to hang the comment, and there > isn't one. (Except for the dead 'expire' clause mentioned above.) > > IT'S HARD TO SPOT SALIENT DETAIL IN A SEA OF GLUE CODE. > > The central fact about TinyMQ is that it holds the messages of > a channel in a simple list of {Message, Timestamp} pairs. As > a result, every operation on the data takes time linear in the > current size. > > This is not stated anywhere in any comments nor in the README. > You have to read the code in detail to discover this. And it > is a rather nasty surprise. If a channel holds N messages, > the operations *can* be done in O(log(N)) time. (I believe it > is possible to do even better.) Some sliding window applications > have a bound on the number of elements in the window. This one > has a bound on the age of elements, but they could arrive at a > very high rate, so N *could* get large. > > It is very easy to implement the necessary operations using lists, > so much so that they are present in several copies. Revising the > TinyMQ implementation to work better with long queues would be > harder than necessary because of this. And this goes un-noticed > because there is so much glue code for the guts to get lost in. > > Given that Evan Miller took the trouble to use library components > for structuring this application, why didn't he take the next step, > and use the existing 'sliding window' library data structure? > > Because there is none! > > Yet sliding windows of one sort or another have come up before in > this mailing list. Perhaps we should have a Wiki page on > trapexit to gather requirements for one or more sliding window > libraries. Or perhaps not. "true religion jeans for women" -- > what has that or "Cheap Nike Shoes" to do with Erlang/OTP > (http://www.trapexit.org/forum/viewforum.php?f=20)? > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Evan Miller http://www.evanmiller.org/ From matti.oinas@REDACTED Fri Aug 31 20:09:09 2012 From: matti.oinas@REDACTED (Matti Oinas) Date: Fri, 31 Aug 2012 21:09:09 +0300 Subject: [erlang-questions] Which EEP would you pick as the most important one to implement next? In-Reply-To: References: <5040842C.6010702@ninenines.eu> <87e026c04bdafbec879831e969e632a4.squirrel@chasm.otago.ac.nz> Message-ID: <5040FDC5.7080507@gmail.com> On 08/31/2012 05:39 PM, Steve Vinoski wrote: > On Fri, Aug 31, 2012 at 8:55 AM, Vlad Dumitrescu wrote: >> Hi Richard, >> >> On Fri, Aug 31, 2012 at 2:42 PM, wrote: >>>> I'm surprised implementing frames is not an EEP. This is definitely the >>>> most lacking feature of erlang by far. >>> It would have been, but by the time I was ready to submit it >>> as one, there was a hard requirement that EEPS be marked up >>> in a basically undefined formalism, and a number of things I >>> needed to mark up, I could not figure out how to mark up. >>> That requirement terminated my contribution to EEPs. >>> >>> I learned RUNOFF. I learned SCRIBE. I learned Troff. I >>> learned TeX. I learned LaTeX. I learned Lout. I learned >>> SGML, HTML, XML, XHTML, ... I'm willing to learn any markup >>> notation that has a tolerably complete tolerably accurate >>> manual. Markdown, however, does not. >> This is certainly not what the EEP maintainers would really like to >> see, but any valid HTML that can be found inside a tag is also >> valid Markdown. The weird formalism are just shortcuts for commonly >> used structures. So technically, you could write contents of the EEP >> in plain HTML and it will follow the letter of the EEP guidelines. You >> only have to add the EEP headers and some delimiters. >> >> Please don't let this detail delay a most needed proposal! > Agreed. Also, Richard, you might try writing your proposal in > something other than markdown, something that gives you what you're > aiming for, and then using pandoc to convert it to markdown for > submission -- see http://johnmacfarlane.net/pandoc/ if you're not > familiar with pandoc. > > --steve > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Richard, if all these methods fail then I volunteer to learn all the needed markdown and do the conversion for you. - Matti From g@REDACTED Fri Aug 31 20:14:46 2012 From: g@REDACTED (Garrett Smith) Date: Fri, 31 Aug 2012 13:14:46 -0500 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: Message-ID: It's fun to watch this process. Back in the days when most source code closed, you'd never get to see the iterations of maturing code. Now we write something and then put it on a publicly accessible server where people can see it in its earliest, rawest form. And to get minds like RoK and Joe to weigh in -- it's very special I think. On Fri, Aug 31, 2012 at 12:42 PM, Evan Miller wrote: > Richard, > > Thanks for your comments. To preface, I plead guilty to charges of > gross negligence in failing to document TinyMQ's internals. This was > laziness on my part. > > I released TinyMQ only because I felt guilty for sitting on the code > for about a year. Like many open-source programmers, I have a lot of > demands on my attention, and it is not clear in advance what > documentation is actually worth writing. The @spec and @doc strings > for the public API seemed like a good start. But if it turned out that > no one was interested in using the library in the first place, why > should I bother documenting internal protocols and data structures? > I've wasted many hours in the past documenting, refactoring, and > generally cleaning up application internals for the benefit of > nebulous "others", only to receive zero patches and no indication that > any of my efforts were of any assistance to anyone. > > So in the spirit of your capitalized complaints, I will just say: > > ALL YOU HAVE TO DO IS ASK > > Want to know about the big-O performance characteristics? Just ask. > Want to know how channel creation works? Just ask. As a lazy person, > if a few people ask me the same thing I'll usually add a note to the > README in order to avert future emails from strangers. We all like a > well-documented project, but without feedback and communication it is > not clear where one's efforts are best spent on a project that doesn't > have an explicit client. If I knew in advance who would be using and > reading the code (i.e. if I wrote this code for an employer), I would > put more effort into writing documents for that specific audience. But > as a rule, if I am just putting some code "out there", I would rather > wait and see what people would like to know about, rather than > pre-emptively document every thought that has ever occurred to me > relating to the code base. > > Now, I know you were not trying to pick on TinyMQ, and your interest > is more in how Erlang tends to result in lumps of code that obscure > key characteristics of the application. I agree with the assessment, > but I am not quite as hopeless about the situation. > > I would like to see the development of graphical tools that let you > see in an instant how applications are structured and how they behave. > I am thinking of something like Pman on steroids, where I can *watch* > messages travel between processes, *inspect* gen_server state, and > *test* the system by seeing the result of single function calls or > many (load-testing). I'd like to be able to do all this with my mouse, > and generally get the feeling that I am watching the operation of a > machine that *shows* me how messages are passed, processes are > created, and state is updated. > > Did anyone else ever play Marble Drop from Maxis in the late 90s? That > is the kind of interface I would like to see for the Erlang run-time. > > For now, I'll update the README. > > Evan > > On Fri, Aug 31, 2012 at 1:20 AM, Richard O'Keefe wrote: >> We've just had a thread about what people like about Erlang. >> We also had the announcement of TinyMQ. >> So I'm going to use this as an example of what's *really* >> wrong with Erlang. >> >> Don't get me wrong. I endorse everything everyone else has >> said in favour of Erlang. Erlang is like democracy: the worst >> thing in its class except for all the others, and something >> that is increasingly imitated by people who just don't get >> some of the fundamental things about it. >> >> I also endorse what people have said in praise of TinyMQ. >> There are lots of things that it does right: >> - there is a README >> - there are EDoc comments with @specs for the public >> interface >> - the functions and variables are named well enough that >> I was never in doubt about what any part of the code was >> up to, at least not for longer than a second or two >> - the hard work of process management is delegated to OTP >> behaviours >> At this point, it's looking better than anything I've written. >> >> Make no mistake: I am not saying that Erlang or TinyMQ are *bad*. >> They are good things; I'm just ranting somewhat vaguely about >> why they should be better. >> >> >> LUMPS OF INDISTINGUISHABLE CODE. >> >> Up to a certain level of hand-waving, TinyMQ can be roughly >> understood thus: >> The TinyMQ *system* is a monitor >> guarding a dictionary mapping strings to channnels, >> where >> a channel is a monitor >> guarding a bag of subscribers and >> a sliding window of {Message, Timestamp} pairs. >> >> YOU CANNOT SEE THIS AT A GLANCE. >> >> This is not Evan Miller's fault. *Anything* you write in >> Erlang is going to end up as lumps of indistinguishable code, >> because there is nothing else for it to be. >> >> This is also true in C, C++, Java, C#, Javascript, Go, >> Eiffel, Smalltalk, Prolog, Haskell, Clean, SML, ..., >> not to mention Visual Basic and Fortran. >> >> Almost the only languages I know where it doesn't *have* to >> be true are Lisp, Scheme, and Lisp-Flavoured Erlang. Arguably >> Prolog *could* be in this group, but in practice it usually is >> in the other camp. Thanks to the preprocessor, C *can* be >> made rather more scrutable, but for some reason this is frowned on. >> >> There's the e2 project (http://e2project.org) which is a step >> in a good direction, but it doesn't do much about this problem. >> A version of TinyMQ using e2_service instead of gen_server >> would in fact exacerbate the problem by mushing >> handle_call/3, handle_cast/2, and handle_info/2 into one >> function, turning three lumps into one bigger lump. >> >> LUMPS OF DATA. >> >> Take tinymq_channel_controller as an example. >> Using an OTP behaviour means that all six dimensions of the state >> are mushed together in one data structure. This goes a long way >> towards hiding the fact that >> >> supervisor, channel, and max_age are never changed >> messages, subscribers, and last_pull *are* changed. >> >> One teeny tiny step here would be to offer an alternative set of >> callbacks for some behaviours where the "state" is separated into >> immutable "context" and mutable "state", so that it is obvious >> *by construction* that the context information *can't* be changed. >> >> Another option would be to have some way of annotation in a >> -record declaration that a field cannot be updated. >> >> I prefer the segregation approach on the grounds of no language >> change being needed and the improved efficiency of not copying >> fields that can't have changed. Others might prefer the revise >> -record approach on the grounds of not having to change or >> duplicate the OTP behaviours. >> >> I had to reach each file in detail >> - to find that certain fields *happened* not to be changed >> - to understand the design well enough to tell that this was >> almost certainly deliberate. >> >> WE DOCUMENT THE WRONG THINGS. >> >> It's well known that there are two kinds of documentation, >> "external" documentation for people writing clients of a module, >> and "internal" documentation for people maintaining the module >> itself. It's also well known that the division is simplistic; >> if the external documentation is silent about material points >> you have to read the internal documentation. >> >> In languages like Prolog and Erlang and Scheme where you build >> data structures out of existing "universal" types and have no >> data structure declarations, we tend to document procedures >> but not data. This is backwards. If you understand the data, >> and especially its invariants, the code is often pretty obvious. >> >> There are two examples of this in TinyMQ. One is specific to >> TinyMQ. The other other is nearly universal in Erlang practice. >> >> Erlang systems are made of lots of processes sending messages >> to each other. Joe Armstrong has often said THINK ABOUT THE >> PROTOCOLS. But Erlang programmers very seldom *write* about >> the protocols. >> >> Using the OTP behaviours, a "concurrent object" is implemented >> as a module with a bunch of interface functions that forward >> messages through the OTP layer to the callback code managed by >> whatever behaviour it is. This protocol is unique to each kind >> of concurrent object. It's often generated in one module (the >> one with the interface functions) and consumed in another (the >> one with the callback code), as it is in TinyMQ. And it's not >> documented. >> >> It is possible to reconstruct this protocol by reading the code >> in detail and noting down what you see. It is troublesome when, >> as in TinyMQ, the two modules disagree about the protocol. It's >> clear that _something_ is wrong, but what, exactly? >> >> For example, tinymq_controller has a case >> handle_cast({set_max_age, newMaxAge}, State) -> >> but this is the only occurrence of set_max_age anywhere in TinyMQ. >> Is its presence in tinymq_controller an example of dead code, >> or is its absence from the rest of the application an example >> of missing code? The same question can be asked about 'expire' >> (which would forget a channel without making it actually go away, >> if it could ever be invoked, which it can't.) >> >> Almost as soon as I started reading Erlang code many years ago >> it seemed obvious to me that documenting (and if possible, type >> checking) these internal protocols was a very important part of >> Erlang internal documentation. There must be something wrong >> with my brain, because other people don't seem to feel this lack >> anywhere nearly as strongly as I do. I think Joe Armstrong sort >> of sees this at the next level up or he would never have invented >> UBF. >> >> But Occam, Go, and Sing# have typed channels, so they *are* >> addressing the issue, and *do* have a natural central point to >> document what the alternatives of an internal protocol signify. >> >> Another documentation failure is that we fail to document what >> is not there. In TinyMQ, a channel automatically comes into >> existence when you try to use it. Perhaps as a consequence of >> this, there is no way to shut a channel down. In TinyMQ, old >> messages are not removed from a channel when they expire, but >> the next time someone does a 'subscribe' (waves hands) or a 'poll' >> or a 'push' *after* they expire. So if processes stop sending >> and requesting messages to some channel, the last few messages, >> no matter how large, may hang around forever. I'm sure there >> is a reason, but because it's a reason for something *not* being >> there, there's no obvious place to hang the comment, and there >> isn't one. (Except for the dead 'expire' clause mentioned above.) >> >> IT'S HARD TO SPOT SALIENT DETAIL IN A SEA OF GLUE CODE. >> >> The central fact about TinyMQ is that it holds the messages of >> a channel in a simple list of {Message, Timestamp} pairs. As >> a result, every operation on the data takes time linear in the >> current size. >> >> This is not stated anywhere in any comments nor in the README. >> You have to read the code in detail to discover this. And it >> is a rather nasty surprise. If a channel holds N messages, >> the operations *can* be done in O(log(N)) time. (I believe it >> is possible to do even better.) Some sliding window applications >> have a bound on the number of elements in the window. This one >> has a bound on the age of elements, but they could arrive at a >> very high rate, so N *could* get large. >> >> It is very easy to implement the necessary operations using lists, >> so much so that they are present in several copies. Revising the >> TinyMQ implementation to work better with long queues would be >> harder than necessary because of this. And this goes un-noticed >> because there is so much glue code for the guts to get lost in. >> >> Given that Evan Miller took the trouble to use library components >> for structuring this application, why didn't he take the next step, >> and use the existing 'sliding window' library data structure? >> >> Because there is none! >> >> Yet sliding windows of one sort or another have come up before in >> this mailing list. Perhaps we should have a Wiki page on >> trapexit to gather requirements for one or more sliding window >> libraries. Or perhaps not. "true religion jeans for women" -- >> what has that or "Cheap Nike Shoes" to do with Erlang/OTP >> (http://www.trapexit.org/forum/viewforum.php?f=20)? >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Evan Miller > http://www.evanmiller.org/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Fri Aug 31 20:16:15 2012 From: g@REDACTED (Garrett Smith) Date: Fri, 31 Aug 2012 13:16:15 -0500 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: Message-ID: This of course wasn't meant to imply that TinyMQ was early or raw! On Fri, Aug 31, 2012 at 1:14 PM, Garrett Smith wrote: > It's fun to watch this process. Back in the days when most source code > closed, you'd never get to see the iterations of maturing code. > > Now we write something and then put it on a publicly accessible server > where people can see it in its earliest, rawest form. > > And to get minds like RoK and Joe to weigh in -- it's very special I think. > > On Fri, Aug 31, 2012 at 12:42 PM, Evan Miller wrote: >> Richard, >> >> Thanks for your comments. To preface, I plead guilty to charges of >> gross negligence in failing to document TinyMQ's internals. This was >> laziness on my part. >> >> I released TinyMQ only because I felt guilty for sitting on the code >> for about a year. Like many open-source programmers, I have a lot of >> demands on my attention, and it is not clear in advance what >> documentation is actually worth writing. The @spec and @doc strings >> for the public API seemed like a good start. But if it turned out that >> no one was interested in using the library in the first place, why >> should I bother documenting internal protocols and data structures? >> I've wasted many hours in the past documenting, refactoring, and >> generally cleaning up application internals for the benefit of >> nebulous "others", only to receive zero patches and no indication that >> any of my efforts were of any assistance to anyone. >> >> So in the spirit of your capitalized complaints, I will just say: >> >> ALL YOU HAVE TO DO IS ASK >> >> Want to know about the big-O performance characteristics? Just ask. >> Want to know how channel creation works? Just ask. As a lazy person, >> if a few people ask me the same thing I'll usually add a note to the >> README in order to avert future emails from strangers. We all like a >> well-documented project, but without feedback and communication it is >> not clear where one's efforts are best spent on a project that doesn't >> have an explicit client. If I knew in advance who would be using and >> reading the code (i.e. if I wrote this code for an employer), I would >> put more effort into writing documents for that specific audience. But >> as a rule, if I am just putting some code "out there", I would rather >> wait and see what people would like to know about, rather than >> pre-emptively document every thought that has ever occurred to me >> relating to the code base. >> >> Now, I know you were not trying to pick on TinyMQ, and your interest >> is more in how Erlang tends to result in lumps of code that obscure >> key characteristics of the application. I agree with the assessment, >> but I am not quite as hopeless about the situation. >> >> I would like to see the development of graphical tools that let you >> see in an instant how applications are structured and how they behave. >> I am thinking of something like Pman on steroids, where I can *watch* >> messages travel between processes, *inspect* gen_server state, and >> *test* the system by seeing the result of single function calls or >> many (load-testing). I'd like to be able to do all this with my mouse, >> and generally get the feeling that I am watching the operation of a >> machine that *shows* me how messages are passed, processes are >> created, and state is updated. >> >> Did anyone else ever play Marble Drop from Maxis in the late 90s? That >> is the kind of interface I would like to see for the Erlang run-time. >> >> For now, I'll update the README. >> >> Evan >> >> On Fri, Aug 31, 2012 at 1:20 AM, Richard O'Keefe wrote: >>> We've just had a thread about what people like about Erlang. >>> We also had the announcement of TinyMQ. >>> So I'm going to use this as an example of what's *really* >>> wrong with Erlang. >>> >>> Don't get me wrong. I endorse everything everyone else has >>> said in favour of Erlang. Erlang is like democracy: the worst >>> thing in its class except for all the others, and something >>> that is increasingly imitated by people who just don't get >>> some of the fundamental things about it. >>> >>> I also endorse what people have said in praise of TinyMQ. >>> There are lots of things that it does right: >>> - there is a README >>> - there are EDoc comments with @specs for the public >>> interface >>> - the functions and variables are named well enough that >>> I was never in doubt about what any part of the code was >>> up to, at least not for longer than a second or two >>> - the hard work of process management is delegated to OTP >>> behaviours >>> At this point, it's looking better than anything I've written. >>> >>> Make no mistake: I am not saying that Erlang or TinyMQ are *bad*. >>> They are good things; I'm just ranting somewhat vaguely about >>> why they should be better. >>> >>> >>> LUMPS OF INDISTINGUISHABLE CODE. >>> >>> Up to a certain level of hand-waving, TinyMQ can be roughly >>> understood thus: >>> The TinyMQ *system* is a monitor >>> guarding a dictionary mapping strings to channnels, >>> where >>> a channel is a monitor >>> guarding a bag of subscribers and >>> a sliding window of {Message, Timestamp} pairs. >>> >>> YOU CANNOT SEE THIS AT A GLANCE. >>> >>> This is not Evan Miller's fault. *Anything* you write in >>> Erlang is going to end up as lumps of indistinguishable code, >>> because there is nothing else for it to be. >>> >>> This is also true in C, C++, Java, C#, Javascript, Go, >>> Eiffel, Smalltalk, Prolog, Haskell, Clean, SML, ..., >>> not to mention Visual Basic and Fortran. >>> >>> Almost the only languages I know where it doesn't *have* to >>> be true are Lisp, Scheme, and Lisp-Flavoured Erlang. Arguably >>> Prolog *could* be in this group, but in practice it usually is >>> in the other camp. Thanks to the preprocessor, C *can* be >>> made rather more scrutable, but for some reason this is frowned on. >>> >>> There's the e2 project (http://e2project.org) which is a step >>> in a good direction, but it doesn't do much about this problem. >>> A version of TinyMQ using e2_service instead of gen_server >>> would in fact exacerbate the problem by mushing >>> handle_call/3, handle_cast/2, and handle_info/2 into one >>> function, turning three lumps into one bigger lump. >>> >>> LUMPS OF DATA. >>> >>> Take tinymq_channel_controller as an example. >>> Using an OTP behaviour means that all six dimensions of the state >>> are mushed together in one data structure. This goes a long way >>> towards hiding the fact that >>> >>> supervisor, channel, and max_age are never changed >>> messages, subscribers, and last_pull *are* changed. >>> >>> One teeny tiny step here would be to offer an alternative set of >>> callbacks for some behaviours where the "state" is separated into >>> immutable "context" and mutable "state", so that it is obvious >>> *by construction* that the context information *can't* be changed. >>> >>> Another option would be to have some way of annotation in a >>> -record declaration that a field cannot be updated. >>> >>> I prefer the segregation approach on the grounds of no language >>> change being needed and the improved efficiency of not copying >>> fields that can't have changed. Others might prefer the revise >>> -record approach on the grounds of not having to change or >>> duplicate the OTP behaviours. >>> >>> I had to reach each file in detail >>> - to find that certain fields *happened* not to be changed >>> - to understand the design well enough to tell that this was >>> almost certainly deliberate. >>> >>> WE DOCUMENT THE WRONG THINGS. >>> >>> It's well known that there are two kinds of documentation, >>> "external" documentation for people writing clients of a module, >>> and "internal" documentation for people maintaining the module >>> itself. It's also well known that the division is simplistic; >>> if the external documentation is silent about material points >>> you have to read the internal documentation. >>> >>> In languages like Prolog and Erlang and Scheme where you build >>> data structures out of existing "universal" types and have no >>> data structure declarations, we tend to document procedures >>> but not data. This is backwards. If you understand the data, >>> and especially its invariants, the code is often pretty obvious. >>> >>> There are two examples of this in TinyMQ. One is specific to >>> TinyMQ. The other other is nearly universal in Erlang practice. >>> >>> Erlang systems are made of lots of processes sending messages >>> to each other. Joe Armstrong has often said THINK ABOUT THE >>> PROTOCOLS. But Erlang programmers very seldom *write* about >>> the protocols. >>> >>> Using the OTP behaviours, a "concurrent object" is implemented >>> as a module with a bunch of interface functions that forward >>> messages through the OTP layer to the callback code managed by >>> whatever behaviour it is. This protocol is unique to each kind >>> of concurrent object. It's often generated in one module (the >>> one with the interface functions) and consumed in another (the >>> one with the callback code), as it is in TinyMQ. And it's not >>> documented. >>> >>> It is possible to reconstruct this protocol by reading the code >>> in detail and noting down what you see. It is troublesome when, >>> as in TinyMQ, the two modules disagree about the protocol. It's >>> clear that _something_ is wrong, but what, exactly? >>> >>> For example, tinymq_controller has a case >>> handle_cast({set_max_age, newMaxAge}, State) -> >>> but this is the only occurrence of set_max_age anywhere in TinyMQ. >>> Is its presence in tinymq_controller an example of dead code, >>> or is its absence from the rest of the application an example >>> of missing code? The same question can be asked about 'expire' >>> (which would forget a channel without making it actually go away, >>> if it could ever be invoked, which it can't.) >>> >>> Almost as soon as I started reading Erlang code many years ago >>> it seemed obvious to me that documenting (and if possible, type >>> checking) these internal protocols was a very important part of >>> Erlang internal documentation. There must be something wrong >>> with my brain, because other people don't seem to feel this lack >>> anywhere nearly as strongly as I do. I think Joe Armstrong sort >>> of sees this at the next level up or he would never have invented >>> UBF. >>> >>> But Occam, Go, and Sing# have typed channels, so they *are* >>> addressing the issue, and *do* have a natural central point to >>> document what the alternatives of an internal protocol signify. >>> >>> Another documentation failure is that we fail to document what >>> is not there. In TinyMQ, a channel automatically comes into >>> existence when you try to use it. Perhaps as a consequence of >>> this, there is no way to shut a channel down. In TinyMQ, old >>> messages are not removed from a channel when they expire, but >>> the next time someone does a 'subscribe' (waves hands) or a 'poll' >>> or a 'push' *after* they expire. So if processes stop sending >>> and requesting messages to some channel, the last few messages, >>> no matter how large, may hang around forever. I'm sure there >>> is a reason, but because it's a reason for something *not* being >>> there, there's no obvious place to hang the comment, and there >>> isn't one. (Except for the dead 'expire' clause mentioned above.) >>> >>> IT'S HARD TO SPOT SALIENT DETAIL IN A SEA OF GLUE CODE. >>> >>> The central fact about TinyMQ is that it holds the messages of >>> a channel in a simple list of {Message, Timestamp} pairs. As >>> a result, every operation on the data takes time linear in the >>> current size. >>> >>> This is not stated anywhere in any comments nor in the README. >>> You have to read the code in detail to discover this. And it >>> is a rather nasty surprise. If a channel holds N messages, >>> the operations *can* be done in O(log(N)) time. (I believe it >>> is possible to do even better.) Some sliding window applications >>> have a bound on the number of elements in the window. This one >>> has a bound on the age of elements, but they could arrive at a >>> very high rate, so N *could* get large. >>> >>> It is very easy to implement the necessary operations using lists, >>> so much so that they are present in several copies. Revising the >>> TinyMQ implementation to work better with long queues would be >>> harder than necessary because of this. And this goes un-noticed >>> because there is so much glue code for the guts to get lost in. >>> >>> Given that Evan Miller took the trouble to use library components >>> for structuring this application, why didn't he take the next step, >>> and use the existing 'sliding window' library data structure? >>> >>> Because there is none! >>> >>> Yet sliding windows of one sort or another have come up before in >>> this mailing list. Perhaps we should have a Wiki page on >>> trapexit to gather requirements for one or more sliding window >>> libraries. Or perhaps not. "true religion jeans for women" -- >>> what has that or "Cheap Nike Shoes" to do with Erlang/OTP >>> (http://www.trapexit.org/forum/viewforum.php?f=20)? >>> >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> -- >> Evan Miller >> http://www.evanmiller.org/ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Fri Aug 31 20:22:59 2012 From: g@REDACTED (Garrett Smith) Date: Fri, 31 Aug 2012 13:22:59 -0500 Subject: [erlang-questions] Which EEP would you pick as the most important one to implement next? In-Reply-To: <5040FDC5.7080507@gmail.com> References: <5040842C.6010702@ninenines.eu> <87e026c04bdafbec879831e969e632a4.squirrel@chasm.otago.ac.nz> <5040FDC5.7080507@gmail.com> Message-ID: On Fri, Aug 31, 2012 at 1:09 PM, Matti Oinas wrote: > On 08/31/2012 05:39 PM, Steve Vinoski wrote: >> >> On Fri, Aug 31, 2012 at 8:55 AM, Vlad Dumitrescu >> wrote: >>> >>> Hi Richard, >>> >>> On Fri, Aug 31, 2012 at 2:42 PM, wrote: >>>>> >>>>> I'm surprised implementing frames is not an EEP. This is definitely the >>>>> most lacking feature of erlang by far. >>>> >>>> It would have been, but by the time I was ready to submit it >>>> as one, there was a hard requirement that EEPS be marked up >>>> in a basically undefined formalism, and a number of things I >>>> needed to mark up, I could not figure out how to mark up. >>>> That requirement terminated my contribution to EEPs. >>>> >>>> I learned RUNOFF. I learned SCRIBE. I learned Troff. I >>>> learned TeX. I learned LaTeX. I learned Lout. I learned >>>> SGML, HTML, XML, XHTML, ... I'm willing to learn any markup >>>> notation that has a tolerably complete tolerably accurate >>>> manual. Markdown, however, does not. >>> >>> This is certainly not what the EEP maintainers would really like to >>> see, but any valid HTML that can be found inside a tag is also >>> valid Markdown. The weird formalism are just shortcuts for commonly >>> used structures. So technically, you could write contents of the EEP >>> in plain HTML and it will follow the letter of the EEP guidelines. You >>> only have to add the EEP headers and some delimiters. >>> >>> Please don't let this detail delay a most needed proposal! >> >> Agreed. Also, Richard, you might try writing your proposal in >> something other than markdown, something that gives you what you're >> aiming for, and then using pandoc to convert it to markdown for >> submission -- see http://johnmacfarlane.net/pandoc/ if you're not >> familiar with pandoc. >> >> --steve >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > Richard, if all these methods fail then I volunteer to learn all the needed > markdown and do the conversion for you. So if there was an EEP #40 for Frames, how would that fall out in the voting? +1 for me - this is easily the most missing feature from the language. I'm not familiar with the EEP process (lazy) but if someone were to just convert Richard's most current proposal, whatever format it's in, to an EEP, would that work? I'd be more than happy to tackle that. Garrett From emmiller@REDACTED Fri Aug 31 20:57:46 2012 From: emmiller@REDACTED (Evan Miller) Date: Fri, 31 Aug 2012 13:57:46 -0500 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: Message-ID: A few more comments specific to TinyMQ: On Fri, Aug 31, 2012 at 1:20 AM, Richard O'Keefe wrote: > Another documentation failure is that we fail to document what > is not there. In TinyMQ, a channel automatically comes into > existence when you try to use it. Perhaps as a consequence of > this, there is no way to shut a channel down. In TinyMQ, old > messages are not removed from a channel when they expire, but > the next time someone does a 'subscribe' (waves hands) or a 'poll' > or a 'push' *after* they expire. So if processes stop sending > and requesting messages to some channel, the last few messages, > no matter how large, may hang around forever. I'm sure there > is a reason, but because it's a reason for something *not* being > there, there's no obvious place to hang the comment, and there > isn't one. (Except for the dead 'expire' clause mentioned above.) > > IT'S HARD TO SPOT SALIENT DETAIL IN A SEA OF GLUE CODE. This probably proves your point, but just to be clear old messages will only "hang around forever" if 1. there are no new messages on a channel and 2. the channel continues to receive requests. If there is no activity on a channel for max_age, the gen_server timeout is invoked here: https://github.com/evanmiller/tinymq/blob/master/src/tinymq_channel_controller.erl#L76 That calls "expire" (erasing the reference to the channel) and then exits the supervisor (eliminating the channel and all of its messages). So in the absence of channel activity, the longest a message will hang around is 2 * max_age, e.g. when there is a push \epsilon seconds before an old message is set to expire. > > The central fact about TinyMQ is that it holds the messages of > a channel in a simple list of {Message, Timestamp} pairs. As > a result, every operation on the data takes time linear in the > current size. > > This is not stated anywhere in any comments nor in the README. > You have to read the code in detail to discover this. And it > is a rather nasty surprise. If a channel holds N messages, > the operations *can* be done in O(log(N)) time. (I believe it > is possible to do even better.) Some sliding window applications > have a bound on the number of elements in the window. This one > has a bound on the age of elements, but they could arrive at a > very high rate, so N *could* get large. I have added some notes about TinyMQ's run-time characteristics to the README: https://github.com/evanmiller/tinymq/commit/568bc7ce94ebaa42e3ce047c375c73323042853a > > It is very easy to implement the necessary operations using lists, > so much so that they are present in several copies. Revising the > TinyMQ implementation to work better with long queues would be > harder than necessary because of this. And this goes un-noticed > because there is so much glue code for the guts to get lost in. > > Given that Evan Miller took the trouble to use library components > for structuring this application, why didn't he take the next step, > and use the existing 'sliding window' library data structure? > > Because there is none! This would be a useful addition! Evan > > Yet sliding windows of one sort or another have come up before in > this mailing list. Perhaps we should have a Wiki page on > trapexit to gather requirements for one or more sliding window > libraries. Or perhaps not. "true religion jeans for women" -- > what has that or "Cheap Nike Shoes" to do with Erlang/OTP > (http://www.trapexit.org/forum/viewforum.php?f=20)? > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Evan Miller http://www.evanmiller.org/ From emmiller@REDACTED Fri Aug 31 21:20:06 2012 From: emmiller@REDACTED (Evan Miller) Date: Fri, 31 Aug 2012 14:20:06 -0500 Subject: [erlang-questions] What I dislike about Erlang In-Reply-To: References: Message-ID: On Fri, Aug 31, 2012 at 12:42 PM, Evan Miller wrote: > Did anyone else ever play Marble Drop from Maxis in the late 90s? That > is the kind of interface I would like to see for the Erlang run-time. For those of you who were not in middle school in 1997, here is a screenshot of Marble Drop: http://www.mobygames.com/images/shots/l/168377-marble-drop-windows-screenshot-one-of-the-many-puzzless.jpg The game was to figure out what order to drop colored balls down into the different funnels to achieve a desired ordering of balls at the bottom, but every drop would (predictably) change the internal state of the level as the marble passed by. I think of the colored balls as being like Erlang messages, the funnels as being processes that receive messages from the outside, the gates and switches as being various bits of internal state, and the trays at the bottom being assertions about the final state of the application. I don't believe in "visual programming" per se but with the right interface I think "visual debugging" would make Erlang programs much more comprehensible (and fun). Evan From thomas.jarvstrand@REDACTED Fri Aug 31 23:42:02 2012 From: thomas.jarvstrand@REDACTED (=?ISO-8859-1?Q?Thomas_J=E4rvstrand?=) Date: Fri, 31 Aug 2012 23:42:02 +0200 Subject: [erlang-questions] Licensing an Erlang/Emacs Lisp project In-Reply-To: References: Message-ID: On Fri, Aug 31, 2012 at 6:05 PM, Ulf Wiger wrote: > > On 31 Aug 2012, at 15:59, Thomas J?rvstrand wrote: > > I'm currently working on a project that is a combination of Erlang and > Emacs Lisp. The project is currently open sourced under the LGPL but I > recently realized that some of the submodules I use are under GPL (the > elisp ones) and so I believe I have to change my licensing. > > > If you include code that is GPL, then you are pretty much stuck with > GPL:ing your whole project. > > You don't need to worry much about Erlang, though. It remains under its > own license > (nothing you do can alter the license of somebody else's components). > Erlang's > license is not contageous, and you are allowed to write Erlang code and > release it as > GPL (popular examples are PropEr and Ejabberd). > > If you want to get fancy, you can meditate over the Mozilla Public License > 2.0, > and its wording on compatibility with (L)GPL, but if you're ok with GPL, > perhaps > you should just go with that. > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > Thank you for your reply. Correct me if I'm wrong but as far as I've understood it, writing a program to be run in the erlang vm is fine since that qualifies in GPL as being system libraries. The problem supposedly is that the moment you call one of the otp standard libraries you're toast since that is the equivalent of dynamic linking, meaning that in practice it's not really feasible to license an Erlang project under GPL? Regards Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: