From n.oxyde@REDACTED Mon Sep 1 00:55:43 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Mon, 1 Sep 2014 00:55:43 +0200 Subject: [erlang-questions] Erlang and Akka In-Reply-To: References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <1409314571.19205.YahooMailNeo@web163602.mail.gq1.yahoo.com> Message-ID: <394C87AB-98FD-4AA2-A589-DAC812D59238@gmail.com> Found this: http://www.erlang.se/workshop/2003/paper/p56-nystrom.pdf -- Anthony Ramine Le 31 ao?t 2014 ? 23:47, Richard A. O'Keefe a ?crit : > Is there a reference for that? From thomasl_erlang@REDACTED Mon Sep 1 10:09:37 2014 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 1 Sep 2014 01:09:37 -0700 Subject: [erlang-questions] Erlang and Akka In-Reply-To: <394C87AB-98FD-4AA2-A589-DAC812D59238@gmail.com> References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <1409314571.19205.YahooMailNeo@web163602.mail.gq1.yahoo.com> <394C87AB-98FD-4AA2-A589-DAC812D59238@gmail.com> Message-ID: <1409558977.1814.YahooMailNeo@web163603.mail.gq1.yahoo.com> Thanks, that's it. Here is Sven-Olof's home page for that system: http://user.it.uu.se/~svenolof/SA/ Best, Thomas On Monday, September 1, 2014 12:55 AM, Anthony Ramine wrote: > > >Found this: > > http://www.erlang.se/workshop/2003/paper/p56-nystrom.pdf > >-- >Anthony Ramine > > >Le 31 ao?t 2014 ? 23:47, Richard A. O'Keefe a ?crit : > >> Is there a reference for that? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wojtek@REDACTED Mon Sep 1 10:28:29 2014 From: wojtek@REDACTED (=?UTF-8?B?V29qdGVrIE5hcmN6ecWEc2tp?=) Date: Mon, 01 Sep 2014 10:28:29 +0200 Subject: [erlang-questions] Erlang and Akka In-Reply-To: References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <54019C80.7070000@power.com.pl> Message-ID: <54042E2D.3080409@power.com.pl> On Sat, Aug 30, 2014 at 5:42 PM, Wojtek Narczy?ski > wrote: > > It is extremely difficult to verify protocol adherence statically. > > > What do you mean by "difficult"? Lacking tooling support? Requiring > too much heavy work on manual annotations? Or something else... Probably all of this and more. It is even difficult do design a correct protocol, for example SIP has deadlocks built-in. CHORD also had it's problems. Type checking is basically automated theorem proving. With usual types the theorems you write are relatively ease to prove. With protocol adherence you'd have to write hard theorems, that might even not be provable automatically. As for the tools, there are quite a few, but they are external to the programming languages, so they check models, not implementations. If you are interested in this (fascinating) subject, this is a good place to start reading: http://www.cs.princeton.edu/courses/archive/spring10/cos598D/FormalMethodsNetworkingOutline.html -- Wojtek -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomasl_erlang@REDACTED Mon Sep 1 10:53:53 2014 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 1 Sep 2014 01:53:53 -0700 Subject: [erlang-questions] Erlang and Akka In-Reply-To: References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <1409314571.19205.YahooMailNeo@web163602.mail.gq1.yahoo.com> Message-ID: <1409561633.43779.YahooMailNeo@web163606.mail.gq1.yahoo.com> Right, an imprecise type analysis that simply merges all receives that can be reached by the process might still give some information. You will also for correctness have to handle PID aliasing, several possible receiving processes for a send. (There are some further complicating features in the full language if you want to go that far.) I'm not too optimistic about the practical power of such an analysis, because there's a lot of overapproximation going on and you will probably get a lot of spurious warnings, but it's "the simplest possible" analysis and so might be worth working out its strengths and weaknesses. I mentioned McErlang previously, here is some more information. I'm not familiar with it, but it seems they made some practical progress while the project lasted. (It doesn't look active anymore, but I'm willing to be corrected!) http://babel.ls.fi.upm.es/~fred/ https://github.com/fredlund/McErlang Best, Thomas On Sunday, August 31, 2014 11:47 PM, Richard A. O'Keefe wrote: > > > >On 30/08/2014, at 12:16 AM, Thomas Lindgren wrote: > >> >> Second, Sven-Olof Nystr?m's old Erlang type analyzer was, at least in theory, capable of inferring types for erlang messages and processes. > >Is there a reference for that? > > >> However, even if we remove a number of language features, an Erlang process can contain many receives connected by arbitrary control flow. Thus, I'd expect that a gen_fsm or suchlike where you essentially implement protocols could potentially yield a very complex type. > >Type systems aren't all-or-nothing. For example, Haskell's type >system seems pretty amazing and as precise as you could wish for, >until you meet dependent type systems like the one in Idris. > >A process type need not be *precise* to be useful. Indeed, when >you think about the fact that the type of a pid is not going to >change, while the precise set of messages it is ready to receive >*will* change from time to time, it's not clear that a precise >type would be useful at all. > >A process type that can tell you "you idiot, you're trying to >send a message that this process will NEVER accept under any >conditions" is enough to be useful. > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Mon Sep 1 11:36:54 2014 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 1 Sep 2014 11:36:54 +0200 Subject: [erlang-questions] Erlang and Akka In-Reply-To: <394C87AB-98FD-4AA2-A589-DAC812D59238@gmail.com> References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <1409314571.19205.YahooMailNeo@web163602.mail.gq1.yahoo.com> <394C87AB-98FD-4AA2-A589-DAC812D59238@gmail.com> Message-ID: Some of the ideas presented in that paper became the basis for a type analysis (without source level declarations) that I wrote with the intent of providing more information to the compiler. Then Kostis Sagonas and Tobias Lindahl took the ball and ran with it, and we ended up with Dialyzer. I remember that Sven-Olof Nystr?m also worked on typing of processes, but the main obstacles were (as Thomas Lindgren also mentioned) the overapproximation of which PIDs will receive which messages, the mixing of messages for different purposes in a single mailbox, and that the type for a process needs to describe what messages are acceptable at what times in order to be really useful. I can't find anything published about this research. He did however write up some of his ideas about how Erlang could be improved, partly in order to get more precise information about communication patterns: http://www.it.uu.se/research/publications/reports/2009-005/ /Richard On Mon, Sep 1, 2014 at 12:55 AM, Anthony Ramine wrote: > Found this: > > http://www.erlang.se/workshop/2003/paper/p56-nystrom.pdf > > -- > Anthony Ramine > > Le 31 ao?t 2014 ? 23:47, Richard A. O'Keefe a ?crit : > > > Is there a reference for that? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fritchie@REDACTED Mon Sep 1 11:45:09 2014 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Mon, 01 Sep 2014 04:45:09 -0500 Subject: [erlang-questions] How to debug dirty NIF? In-Reply-To: Message of "Sat, 30 Aug 2014 09:50:32 +0400." Message-ID: <54822.1409564709@snookles.snookles.com> Hi, Max. Regarding collapse of the regular/non-dirty Erlang schedulers, we've found this daemon'ish thing quite useful. It can detect scheduler collapse, though it must create a bunch of nonsense busy work Erlang processes to measure it. And it can fix the problem temporarily ... until it happens again. https://gist.github.com/slfritchie/d834ec3f8a4321b1c2c2 -Scott From i.vysniauskas@REDACTED Mon Sep 1 11:57:24 2014 From: i.vysniauskas@REDACTED (=?UTF-8?Q?Ignas_Vy=C5=A1niauskas?=) Date: Mon, 1 Sep 2014 12:57:24 +0300 Subject: [erlang-questions] Erlang and Akka In-Reply-To: References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <1409314571.19205.YahooMailNeo@web163602.mail.gq1.yahoo.com> <394C87AB-98FD-4AA2-A589-DAC812D59238@gmail.com> Message-ID: Funnily, I was just thinking about the semantics of Erlang and how ad-hoc receive patterns are major blocker for formal reasoning and also impedes performance / predictability. As a consequence, I wrote a very simple model of async pi-calculus in Erlang, with similar semantics to the one proposed in this paper (I also added half-baked channel delegation support to allow dynamic communication patterns). Here's the code: [1]: https://github.com/yfyf/pierl -- Ignas On Mon, Sep 1, 2014 at 12:36 PM, Richard Carlsson wrote: > Some of the ideas presented in that paper became the basis for a type > analysis (without source level declarations) that I wrote with the intent of > providing more information to the compiler. Then Kostis Sagonas and Tobias > Lindahl took the ball and ran with it, and we ended up with Dialyzer. > > I remember that Sven-Olof Nystr?m also worked on typing of processes, but > the main obstacles were (as Thomas Lindgren also mentioned) the > overapproximation of which PIDs will receive which messages, the mixing of > messages for different purposes in a single mailbox, and that the type for a > process needs to describe what messages are acceptable at what times in > order to be really useful. I can't find anything published about this > research. He did however write up some of his ideas about how Erlang could > be improved, partly in order to get more precise information about > communication patterns: > http://www.it.uu.se/research/publications/reports/2009-005/ > > > /Richard > > > On Mon, Sep 1, 2014 at 12:55 AM, Anthony Ramine wrote: >> >> Found this: >> >> http://www.erlang.se/workshop/2003/paper/p56-nystrom.pdf >> >> -- >> Anthony Ramine >> >> Le 31 ao?t 2014 ? 23:47, Richard A. O'Keefe a ?crit : >> >> > Is there a reference for that? >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From wojtek@REDACTED Mon Sep 1 12:20:31 2014 From: wojtek@REDACTED (=?UTF-8?B?V29qdGVrIE5hcmN6ecWEc2tp?=) Date: Mon, 01 Sep 2014 12:20:31 +0200 Subject: [erlang-questions] Erlang and Akka In-Reply-To: <54042E2D.3080409@power.com.pl> References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <54019C80.7070000@power.com.pl> <54042E2D.3080409@power.com.pl> Message-ID: <5404486F.3040002@power.com.pl> On 01.09.2014 10:28, Wojtek Narczy?ski wrote: > It is even difficult do design a correct protocol, for example SIP has > deadlocks built-in. Correction: race conditions, not deadlocks. http://tools.ietf.org/html/rfc5407 -- Wojtek -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Sep 1 12:58:53 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 1 Sep 2014 12:58:53 +0200 Subject: [erlang-questions] Erlang and Akka In-Reply-To: <54042E2D.3080409@power.com.pl> References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <54019C80.7070000@power.com.pl> <54042E2D.3080409@power.com.pl> Message-ID: On Mon, Sep 1, 2014 at 10:28 AM, Wojtek Narczy?ski wrote: > Type checking is basically automated theorem proving. Yes and no. Trying to infer a type from a term is a search problem, more or less. Checking that a given term has a type is more akin to verifying a proof under a set of rules or policies. The hard part about session types and friends is how you: * Get the programmer burden down to a manageable level * Make it expressive so you can describe complicated protocols with ease I do think we are ready to take ventures into this, but I would definitely start with a runtime check and then extend it later with a static notion once we know what are the properties we want to eradicate at compile time. Like a normal type system, we will let things slip through - the trick is just to place the abstraction level so we reap the benefits for little cost. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From andra.dinu@REDACTED Mon Sep 1 14:26:37 2014 From: andra.dinu@REDACTED (Andra Dinu) Date: Mon, 1 Sep 2014 14:26:37 +0200 (CEST) Subject: [erlang-questions] Call for Talks: Berlin Erlang Factory Lite is back on 4 December! In-Reply-To: <544320496.515499.1409574036072.JavaMail.zimbra@erlang-solutions.com> Message-ID: <1770611257.515651.1409574397320.JavaMail.zimbra@erlang-solutions.com> Hi all, We're back in Berlin on 4 December, and we hope this year will be at least as fun as last year. The Call for Talks will be open until 10 October, but we tend to have more talks submitted than we have slots available, so the earlier you send yours the better your chances. Have you made an interesting innovation, open-source application or product with Erlang/OTP? Have you used Erlang in a real-world project and want to present a case study? Or maybe you developed a cool tool? Let the crowd know about it! IMPORTANT: Elixir talks also welcome this year :) Link to submission form: https://docs.google.com/a/erlang-solutions.com/forms/d/1oD8J66a0pjOU-5Y8vG2Ffn8twyDIyPcRMFrPEdANaY4/viewform Link to website: http://www.erlang-factory.com/berlin2014/home Best, Andra -- Andra Dinu Community & Social ERLANG SOLUTIONS LTD New Loom House 101 Back Church Lane London, E1 1LU United Kingdom Tel +44(0)2076550344 Mob +44(0)7983484387 www.erlang-solutions.com From max.lapshin@REDACTED Mon Sep 1 15:57:41 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 1 Sep 2014 17:57:41 +0400 Subject: [erlang-questions] How to debug dirty NIF? In-Reply-To: <54822.1409564709@snookles.snookles.com> References: <54822.1409564709@snookles.snookles.com> Message-ID: Thank you a lot! I'm also thinking about rewriting code to drivers + async threads. -------------- next part -------------- An HTML attachment was scrubbed... URL: From daocuong@REDACTED Mon Sep 1 15:45:50 2014 From: daocuong@REDACTED (DAO CUONG) Date: Mon, 1 Sep 2014 08:45:50 -0500 Subject: [erlang-questions] [ANN] Chicago Erlang Conference - Sept 22 - aka "Best Lineup Ever" In-Reply-To: <53F9A6DA.4030307@gmail.com> References: <53F9A6DA.4030307@gmail.com> Message-ID: Couldnt apply *ceug2014* code for early bird ticket now, anyone have any ideas? On Sun, Aug 24, 2014 at 3:48 AM, Zachary Kessin wrote: > Hey, I am flying in For Israel for this! (OK, I was already going to be > in the USA) > > --Zach > > On 8/20/14, 5:14 PM, Garrett Smith wrote: > > We will likely have video recordings as we did last year. That's the plan. > > However, if you live around Chicago or within easy flight access > (O'Hare is generally a cheaper ticket), we'll be blocking your IP > range for the videos, thereby forcing you to come. [1] > > Garrett > > [1] We won't do this. > > On Wed, Aug 20, 2014 at 8:52 AM, Federico Carrone wrote: > > I am too far away (Buenos Aires) to go. It would be AWESOME if somebody > could record the talks. > > Regards, > Federico. > > > On Tue, Aug 19, 2014 at 6:33 PM, Garrett Smith wrote: > > For a single day, September 22, Chicago will become the center of know > Erlang universe: > http://www.chicagoerlang.com > > This is a single track conference organized by the Chicago Erlang User > Group and Erlang Solutions that is focusing on "Real World Erlang". > The talks are very technical and practical: > > - Joe Armstrong: Modeling the World with Processes, Objects, Functions > or Relations > > - Francesco Cesarini: Thinking in a Concurrent Language > > - Steve Vinoski: Optimizing Native Code for Erlang > > - Fred Hebert: Keeping a System Running Forever > > - Irina Guberman: Maximizing Throughput On Multicore Systems > > - Reid Draper: Building Fault Tolerant Teams At Basho > > - Brian Troutwine: Monitoring Complex Systems > > - Jesse Gumm: Building Web Scale Apps with Nitrogen > > - Garrett Smith: Writing Quality Code in Erlang > > Zach Kessin will be hosting the Mostly Erlang podcast live at the after > party. > > Even if you're not in Chicago, really, this is worth a trip. If you > are in Chicago, you are simple required to go. It's a requirement. > > Seriously, come to this conference. > > Garrett > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > > --http://federicocarrone.com/ > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > -- > Zachary Kessin > Mostly Erlang Podcast > Skype: zachkessin > Twitter: @zkessin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean@REDACTED Mon Sep 1 18:00:01 2014 From: sean@REDACTED (Functional Jobs) Date: Mon, 1 Sep 2014 12:00:01 -0400 Subject: [erlang-questions] New Functional Programming Job Opportunities Message-ID: <540498038a025@functionaljobs.com> Here are some functional programming job opportunities that were posted recently: Senior Software Engineer (Functional) at McGraw-Hill Education http://functionaljobs.com/jobs/8737-senior-software-engineer-functional-at-mcgraw-hill-education Cheers, Sean Murphy FunctionalJobs.com From erlang@REDACTED Mon Sep 1 19:20:27 2014 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 1 Sep 2014 19:20:27 +0200 Subject: [erlang-questions] Erlang and Akka In-Reply-To: References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <1409314571.19205.YahooMailNeo@web163602.mail.gq1.yahoo.com> Message-ID: I'd thought for some time that Erlang ought to have a process declaration. The problem is that processes have no names Pid = spawn( fun() -> ... end) yields an anonymous process - we can't talk about the process Pid since it has no name. We *can* talk about modules since they have names, be we have to use a circumlocution to talk about a process, ie refer to the name of the function that started the process. Since this function can be a fun we have a double naming problem, not only has the process no name, but the start fun has no name. We could make a process declaration (analogous to a module declaration) and tell the system about the messages it can send and receive -process(mailbox). -sends( Type of messages this process can receive) - receives (Type of messages this process can receive) - max_instances(124). - ... whatever .. start(X, Y, Z) -> .... obligatory start function If we did this then we would be able to talk about "the process mailbox/3" (meaning the process started by spawn(mailbox, start, [X,Y,Z]). Once we can name processes we can use the names in annotations in other modules. so we could type functions with Pids -spec foo( PId::process(mailbox)) -> .... foo(Pid) -> receive {Pid, ...} -> ... end. and type check the code. It's difficult to infer this kind of stuff so we might as well not bother, and just *tell* the system what the Pids mean. Cheers /Joe On Sun, Aug 31, 2014 at 11:47 PM, Richard A. O'Keefe wrote: > > On 30/08/2014, at 12:16 AM, Thomas Lindgren wrote: > >> >> Second, Sven-Olof Nystr?m's old Erlang type analyzer was, at least in theory, capable of inferring types for erlang messages and processes. > > Is there a reference for that? > >> However, even if we remove a number of language features, an Erlang process can contain many receives connected by arbitrary control flow. Thus, I'd expect that a gen_fsm or suchlike where you essentially implement protocols could potentially yield a very complex type. > > Type systems aren't all-or-nothing. For example, Haskell's type > system seems pretty amazing and as precise as you could wish for, > until you meet dependent type systems like the one in Idris. > > A process type need not be *precise* to be useful. Indeed, when > you think about the fact that the type of a pid is not going to > change, while the precise set of messages it is ready to receive > *will* change from time to time, it's not clear that a precise > type would be useful at all. > > A process type that can tell you "you idiot, you're trying to > send a message that this process will NEVER accept under any > conditions" is enough to be useful. > > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From sverker.eriksson@REDACTED Mon Sep 1 19:26:40 2014 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Mon, 1 Sep 2014 19:26:40 +0200 Subject: [erlang-questions] Are binaries relocatable? In-Reply-To: References: <540079E4.5030607@erix.ericsson.se> Message-ID: <5404AC50.50701@ericsson.com> Looks like that should work if you're doing it right. * Do you update shared data (like your reference counters) in a thread safe manner? * Another way is to store the copied binary term and then use enif_make_copy(env, entry->content.bin.term) to return the binary. You can also combine it with enif_make_sub_binary if you want to return part of the binary. * Use debug built VM to ease trouble shooting: cd $ERL_TOP/erts/emulator && make TYPE=debug smp plain $ERL_TOP/bin/cerl -debug /Sverker, Erlang/OTP On 08/30/2014 07:07 AM, Hynek Vychodil wrote: > Thanks for your answer very much. If I understand it correctly this way it > should be safe: > > ErlNifBinary bin; > if (!enif_is_binary(env, argv[0])) > return enif_make_badarg(env); > > ErlNifEnv *priv_env = enif_alloc_env(); > ERL_NIF_TERM term = enif_make_copy(priv_env, argv[0]); > ErlNifBinary bin; > enif_inspect_binary(priv_env, term, &bin); > entry->content.bin = (struct my_bin){.env = priv_env, .size = > bin.size, .data = bin.data}; > > And now mine entry->content.bin.data should be safe but unfortunately it > still eventually hangs whole VM after few thousands cycles of referencing > this data > > CacheEntryRes_t *res = enif_alloc_resource(dc_entry_type, > sizeof(CacheEntryRes_t)); > unless (res) return insufficient_memory(env); > res->entry = entry; > ERL_NIF_TERM result = enif_make_resource_binary(env, res, > entry->content.bin.data, entry->content.bin.size); > enif_release_resource(res); > return result; > > I do mine own reference count and release by > > enif_free_env(entry->content.bin.env); > > It hangs only if I replace enif_alloc() + memcpy() and enif_free() version > with above one :-( > > Thanks > Hynek > > > On Fri, Aug 29, 2014 at 3:02 PM, Sverker Eriksson < > sverker.eriksson@REDACTED> wrote: > >> On 08/28/2014 03:09 PM, Hynek Vychodil wrote: >> >>> Hi all, >>> I would like to know if binaries are relocatable. If I have binary >>> parameter >>> >>> ErlNifBinary bin; >>> if (!enif_inspect_binary(env, argv[0], &bin)) >>> return enif_make_badarg(env); >>> >>> I store binary in mine own environment >>> >>> ErlNifEnv *priv_env = enif_alloc_env(); >>> enif_make_copy(priv_env, argv[0]); >>> >>> Can I then rely on bin.data pointing into same memory until I free >>> enif_free_env(priv_env) ? >>> >> No. >> Your bin.data is refering to the original term argv[0] in 'env' which is >> only valid until the NIF returns. >> You must do enif_inspect_binary on the copy in order to get a safe pointer >> that is valid until you do enif_free_env(priv_env). >> >> >> >> I'm asking because I would like to avoid copying >>> and share data or even return it using enif_make_resource_binary to catch >>> and >>> count references. >>> >> Whether a copied binary share data with its original is implementation >> dependent. Today big binaries (>64 bytes) are shared >> and small binaries are copied between environments. >> >> >> I have code which makes copy of binary data but when I switch to reference >>> bin.data it hangs whole VM. I would like to know if I make some mistake in >>> mine code and it should work or it is wrong idea at all. >>> >>> >>> There is a known bug when calling enif_make_copy after >> enif_inspect_binary >> that can cause the binary to get reallocated and thus the bin.data pointer >> to get invalid >> before the NIF returns. To detect this bug, you can call >> enif_inspect_binary(env,argv[0],&bin2) >> after enif_make_copy and check if bin2.data != bin.data. >> >> This bug has been lingering as I thought it was an exotic scenario >> (apparently not) and I have >> been a bit unsure how to fix it without ruin performance. >> >> >> /Sverker, Erlang/OTP >> >> From n.oxyde@REDACTED Mon Sep 1 19:28:40 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Mon, 1 Sep 2014 19:28:40 +0200 Subject: [erlang-questions] In praise of Erlang's 'if' In-Reply-To: <1D51BC06-E118-47CB-B91B-76D9204929C9@gmail.com> References: <6A40C846-66B1-4838-9804-049306D23B75@cs.otago.ac.nz> <1D51BC06-E118-47CB-B91B-76D9204929C9@gmail.com> Message-ID: <26ED412A-EE97-4BA1-BA45-0004F4E1AF1E@gmail.com> I just made a pull request for cond expressions: https://github.com/erlang/otp/pull/464 Regards, -- Anthony Ramine Le 21 ao?t 2014 ? 10:37, Anthony Ramine a ?crit : > Hello Richard, > > Then, let?s assume the user wants to finally use guard disjunctions in if clauses: > > if Green, Crispy -> steam() > ; Green, not Crispy -> chop() > ; not Green; Leafy, Hard -> fry(), grill() > ; not Green, Leafy, not Hard -> fry(), boil() > ; not Green, not Leafy -> fry(), bake() > end > > Now you have a guard disjunction quite hidden in the middle of a guard, and it?s tolerable only with your nonstandard indentation scheme, because in real life here is what happens: > > https://github.com/tuncer/rebar/commit/2c4d7d1d9bdc2a11d3f485f844500bf4c2aa77a2 > > Since that design flaw (which is what it is to my eyes) of the Erlang syntax was demonstrated to me, I want guard sequences to be deprecated in ?if? clauses? heads. Or maybe ?if? in its entirety, replaced by ?cond?, if I want to be fancy. > > Regards, > > -- > Anthony Ramine > > Le 21 ao?t 2014 ? 05:21, Richard A. O'Keefe a ?crit : > >> Erlang's 'if' is often criticised, so I thought it would >> be nice to show an example where it's just right. >> >> In connection with the recent thread on Potion, >> I've been re-reading >> Visual Programming Languages and the >> Empirical Evidence For and Against >> K.N.Whitley, October 1996, >> submitted to the Journal of Visual Languages and Computing. >> >> Amongst other evidence for visual programming languages, this >> paper cites Scanlan's "Structured flowcharts outperform >> pseudocode: An experimental comparison", IEEE Software, June 1989. >> Scanlan's article is, it seems to me, a classic case of >> claiming too much. The title suggests, and the text clearly >> states, that 'structured flowcharts' are superior to 'pseudocode', >> where what was established was that one particular form of >> flowchart was superior to one particular form of pseudocode. >> >> Here's one of the pseudo-code examples: >> >> IF GREEN >> THEN >> IF CRISPY >> THEN >> STEAM >> ELSE >> CHOP >> ENDIF >> ELSE >> FRY >> IF LEAFY >> THEN >> IF HARD >> THEN >> GRILL >> ELSE >> BOIL >> ENDIF >> ELSE >> BAKE >> ENDIF >> ENDIF >> >> The weird layout is none of my invention. >> Now, suppose we used Erlang as the pseudocode: >> >> >> >> if Green, Crispy -> steam() >> ; Green, not Crispy -> chop() >> ; not Green, Leafy, Hard -> fry(), grill() >> ; not Green, Leafy, not Hard -> fry(), boil() >> ; not Green, not Leafy -> fry(), bake() >> end >> >> How do you think the comparison would have come out then? >> >> There is an old experiment that showed that >> if X ... ifnot X ... end >> was easier to understand than if X ... else ... end, and >> oddly enough the 'hungry hare' examples in Scanlan appear >> to have been taken from it, without learning its lesson. >> It is at least possible that Scanlan's results were entirely >> due to this factor, not the 'visual' aspect per se.) >> >> The irony is that an Erlang 'if', laid out the way I've >> laid it out, would in Whitley's paper have been classified >> as a 'visual' notation (subclass 'matrix'). >> >> We could also do it as a 'case': >> >> case {Green, Crispy, Leafy, Hard } >> of {true, true, _, _ } -> steam() >> ; {true, false, _, _ } -> chop() >> ; {false, _, true, true } -> fry(), grill() >> ; {false, _, true, false} -> fry(), boil() >> ; {false, _, false, _ } -> fry(), bake() >> end >> >> but in this case I think 'if' is easier to read. >> >> You might want to object that Scanlan could not have been >> expected to think of this kind of layout in 1986, but this >> is just a 'decision table', and decision tables are much >> older than that and were well known in the 70s. >> >> What this illustrates above all is that proving that some >> particular language or language feature is better or worse >> than some other can be much harder than you think, and >> that even classic results in the field can collapse under >> a little inspection. >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From tony@REDACTED Mon Sep 1 22:09:10 2014 From: tony@REDACTED (Tony Rogvall) Date: Mon, 1 Sep 2014 22:09:10 +0200 Subject: [erlang-questions] In praise of Erlang's 'if' In-Reply-To: <26ED412A-EE97-4BA1-BA45-0004F4E1AF1E@gmail.com> References: <6A40C846-66B1-4838-9804-049306D23B75@cs.otago.ac.nz> <1D51BC06-E118-47CB-B91B-76D9204929C9@gmail.com> <26ED412A-EE97-4BA1-BA45-0004F4E1AF1E@gmail.com> Message-ID: <560B490C-5018-43AC-BD66-15E392A73B0F@rogvall.se> Nice try. /Tony On 1 sep 2014, at 19:28, Anthony Ramine wrote: > I just made a pull request for cond expressions: > > https://github.com/erlang/otp/pull/464 > > Regards, > > -- > Anthony Ramine > > Le 21 ao?t 2014 ? 10:37, Anthony Ramine a ?crit : > >> Hello Richard, >> >> Then, let?s assume the user wants to finally use guard disjunctions in if clauses: >> >> if Green, Crispy -> steam() >> ; Green, not Crispy -> chop() >> ; not Green; Leafy, Hard -> fry(), grill() >> ; not Green, Leafy, not Hard -> fry(), boil() >> ; not Green, not Leafy -> fry(), bake() >> end >> >> Now you have a guard disjunction quite hidden in the middle of a guard, and it?s tolerable only with your nonstandard indentation scheme, because in real life here is what happens: >> >> https://github.com/tuncer/rebar/commit/2c4d7d1d9bdc2a11d3f485f844500bf4c2aa77a2 >> >> Since that design flaw (which is what it is to my eyes) of the Erlang syntax was demonstrated to me, I want guard sequences to be deprecated in ?if? clauses? heads. Or maybe ?if? in its entirety, replaced by ?cond?, if I want to be fancy. >> >> Regards, >> >> -- >> Anthony Ramine >> >> Le 21 ao?t 2014 ? 05:21, Richard A. O'Keefe a ?crit : >> >>> Erlang's 'if' is often criticised, so I thought it would >>> be nice to show an example where it's just right. >>> >>> In connection with the recent thread on Potion, >>> I've been re-reading >>> Visual Programming Languages and the >>> Empirical Evidence For and Against >>> K.N.Whitley, October 1996, >>> submitted to the Journal of Visual Languages and Computing. >>> >>> Amongst other evidence for visual programming languages, this >>> paper cites Scanlan's "Structured flowcharts outperform >>> pseudocode: An experimental comparison", IEEE Software, June 1989. >>> Scanlan's article is, it seems to me, a classic case of >>> claiming too much. The title suggests, and the text clearly >>> states, that 'structured flowcharts' are superior to 'pseudocode', >>> where what was established was that one particular form of >>> flowchart was superior to one particular form of pseudocode. >>> >>> Here's one of the pseudo-code examples: >>> >>> IF GREEN >>> THEN >>> IF CRISPY >>> THEN >>> STEAM >>> ELSE >>> CHOP >>> ENDIF >>> ELSE >>> FRY >>> IF LEAFY >>> THEN >>> IF HARD >>> THEN >>> GRILL >>> ELSE >>> BOIL >>> ENDIF >>> ELSE >>> BAKE >>> ENDIF >>> ENDIF >>> >>> The weird layout is none of my invention. >>> Now, suppose we used Erlang as the pseudocode: >>> >>> >>> >>> if Green, Crispy -> steam() >>> ; Green, not Crispy -> chop() >>> ; not Green, Leafy, Hard -> fry(), grill() >>> ; not Green, Leafy, not Hard -> fry(), boil() >>> ; not Green, not Leafy -> fry(), bake() >>> end >>> >>> How do you think the comparison would have come out then? >>> >>> There is an old experiment that showed that >>> if X ... ifnot X ... end >>> was easier to understand than if X ... else ... end, and >>> oddly enough the 'hungry hare' examples in Scanlan appear >>> to have been taken from it, without learning its lesson. >>> It is at least possible that Scanlan's results were entirely >>> due to this factor, not the 'visual' aspect per se.) >>> >>> The irony is that an Erlang 'if', laid out the way I've >>> laid it out, would in Whitley's paper have been classified >>> as a 'visual' notation (subclass 'matrix'). >>> >>> We could also do it as a 'case': >>> >>> case {Green, Crispy, Leafy, Hard } >>> of {true, true, _, _ } -> steam() >>> ; {true, false, _, _ } -> chop() >>> ; {false, _, true, true } -> fry(), grill() >>> ; {false, _, true, false} -> fry(), boil() >>> ; {false, _, false, _ } -> fry(), bake() >>> end >>> >>> but in this case I think 'if' is easier to read. >>> >>> You might want to object that Scanlan could not have been >>> expected to think of this kind of layout in 1986, but this >>> is just a 'decision table', and decision tables are much >>> older than that and were well known in the 70s. >>> >>> What this illustrates above all is that proving that some >>> particular language or language feature is better or worse >>> than some other can be much harder than you think, and >>> that even classic results in the field can collapse under >>> a little inspection. >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Mon Sep 1 22:38:41 2014 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Mon, 1 Sep 2014 22:38:41 +0200 Subject: [erlang-questions] Are binaries relocatable? In-Reply-To: <5404AC50.50701@ericsson.com> References: <540079E4.5030607@erix.ericsson.se> <5404AC50.50701@ericsson.com> Message-ID: I think I have found what causes deadlock. I have made sort of cache for binaries and use global lock as first testing solution to prove concept. There is a scenario when I return enif_make_resource_binary and then make sub-binary of this binary (in erlang code) and then store this code again in cache. Then reference sub-binary reaches zero, it calls resource destructor, locks mutex and removes from cache which calls enif_free_env() witch causes calling destructor of its resource which tries lock mutex again. It can't happen when I have used fresh new binary in mine previous version. I have tested to remove the only possible place in erlang where any sub-binary of existing binary can be stored in cache and it works. So now I have to wrap head around how call enif_free_env() outside mutex-locked code part. Anyway thanks a lot for your help. Hynek On Mon, Sep 1, 2014 at 7:26 PM, Sverker Eriksson < sverker.eriksson@REDACTED> wrote: > Looks like that should work if you're doing it right. > > * Do you update shared data (like your reference counters) in a thread > safe manner? > > * Another way is to store the copied binary term and then use > enif_make_copy(env, entry->content.bin.term) > to return the binary. You can also combine it with enif_make_sub_binary if > you want to return part of the binary. > > * Use debug built VM to ease trouble shooting: > > cd $ERL_TOP/erts/emulator && make TYPE=debug smp plain > $ERL_TOP/bin/cerl -debug > > > /Sverker, Erlang/OTP > > > > On 08/30/2014 07:07 AM, Hynek Vychodil wrote: > >> Thanks for your answer very much. If I understand it correctly this way it >> should be safe: >> >> ErlNifBinary bin; >> if (!enif_is_binary(env, argv[0])) >> return enif_make_badarg(env); >> >> ErlNifEnv *priv_env = enif_alloc_env(); >> ERL_NIF_TERM term = enif_make_copy(priv_env, argv[0]); >> ErlNifBinary bin; >> enif_inspect_binary(priv_env, term, &bin); >> entry->content.bin = (struct my_bin){.env = priv_env, .size = >> bin.size, .data = bin.data}; >> >> And now mine entry->content.bin.data should be safe but unfortunately it >> still eventually hangs whole VM after few thousands cycles of referencing >> this data >> >> CacheEntryRes_t *res = enif_alloc_resource(dc_entry_type, >> sizeof(CacheEntryRes_t)); >> unless (res) return insufficient_memory(env); >> res->entry = entry; >> ERL_NIF_TERM result = enif_make_resource_binary(env, res, >> entry->content.bin.data, entry->content.bin.size); >> enif_release_resource(res); >> return result; >> >> I do mine own reference count and release by >> >> enif_free_env(entry->content.bin.env); >> >> It hangs only if I replace enif_alloc() + memcpy() and enif_free() version >> with above one :-( >> >> Thanks >> Hynek >> >> >> On Fri, Aug 29, 2014 at 3:02 PM, Sverker Eriksson < >> sverker.eriksson@REDACTED> wrote: >> >> On 08/28/2014 03:09 PM, Hynek Vychodil wrote: >>> >>> Hi all, >>>> I would like to know if binaries are relocatable. If I have binary >>>> parameter >>>> >>>> ErlNifBinary bin; >>>> if (!enif_inspect_binary(env, argv[0], &bin)) >>>> return enif_make_badarg(env); >>>> >>>> I store binary in mine own environment >>>> >>>> ErlNifEnv *priv_env = enif_alloc_env(); >>>> enif_make_copy(priv_env, argv[0]); >>>> >>>> Can I then rely on bin.data pointing into same memory until I free >>>> enif_free_env(priv_env) ? >>>> >>>> No. >>> Your bin.data is refering to the original term argv[0] in 'env' which is >>> only valid until the NIF returns. >>> You must do enif_inspect_binary on the copy in order to get a safe >>> pointer >>> that is valid until you do enif_free_env(priv_env). >>> >>> >>> >>> I'm asking because I would like to avoid copying >>> >>>> and share data or even return it using enif_make_resource_binary to >>>> catch >>>> and >>>> count references. >>>> >>>> Whether a copied binary share data with its original is implementation >>> dependent. Today big binaries (>64 bytes) are shared >>> and small binaries are copied between environments. >>> >>> >>> I have code which makes copy of binary data but when I switch to >>> reference >>> >>>> bin.data it hangs whole VM. I would like to know if I make some mistake >>>> in >>>> mine code and it should work or it is wrong idea at all. >>>> >>>> >>>> There is a known bug when calling enif_make_copy after >>>> >>> enif_inspect_binary >>> that can cause the binary to get reallocated and thus the bin.data >>> pointer >>> to get invalid >>> before the NIF returns. To detect this bug, you can call >>> enif_inspect_binary(env,argv[0],&bin2) >>> after enif_make_copy and check if bin2.data != bin.data. >>> >>> This bug has been lingering as I thought it was an exotic scenario >>> (apparently not) and I have >>> been a bit unsure how to fix it without ruin performance. >>> >>> >>> /Sverker, Erlang/OTP >>> >>> >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wojtek@REDACTED Mon Sep 1 23:06:06 2014 From: wojtek@REDACTED (=?UTF-8?B?V29qdGVrIE5hcmN6ecWEc2tp?=) Date: Mon, 01 Sep 2014 23:06:06 +0200 Subject: [erlang-questions] Erlang and Akka In-Reply-To: References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <1409314571.19205.YahooMailNeo@web163602.mail.gq1.yahoo.com> Message-ID: <5404DFBE.6040907@power.com.pl> On 01.09.2014 19:20, Joe Armstrong wrote: > I'd thought for some time that Erlang ought to have a process declaration. > > The problem is that processes have no names Pid = spawn( fun() -> ... end) > yields an anonymous process - we can't talk about the process Pid > since it has no name. I believe one could do all of what you describe right now rather easily in Haskell. But it wouldn't buy you much, because the key information is _in_what_state_ does the process send or receive given messages. But I fell uncomfortable writing this to the person who came up with UBF some time ago. For complex message flows, having an intermediary whose sole responsibility is to check the flow, with a grammar for example, always seemed like a good idea to me. Maybe internalizing UBF into erlang somehow would be a way to go? -- Kind regards, Wojtek Narczynski From chrisdevilliers@REDACTED Mon Sep 1 23:10:45 2014 From: chrisdevilliers@REDACTED (Chris de Villiers) Date: Mon, 1 Sep 2014 23:10:45 +0200 Subject: [erlang-questions] Process state and sensitive information Message-ID: Hello I want to make requests to a web service which authenticates users with a key and secret. I will be implementing their API with a gen_server and need to place the access credentials somewhere "safe". By safe I mean It should be out of plain sight and not accessible to a user that can attach to the VM. I also do not want them to show up in stack traces or kernel/SASL logs should the gen_server die unexpectedly. The application's environment is obviously ruled out. Storing them in the gen_server's state is also no good because sys:get_status/1 gives them away. I thought about placing them in an ETS table private to the gen_server process. Is it possible for another process to read private ETS tables somehow? Any other suggestions how I can handle this situation? I do not want to start a discussion about OS level security. Lets assume someone gets access to the user account under which the VM runs and can attach to it. Regards, chris From mark.nijhof@REDACTED Tue Sep 2 00:12:44 2014 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Tue, 2 Sep 2014 00:12:44 +0200 Subject: [erlang-questions] Process state and sensitive information In-Reply-To: References: Message-ID: Just from reading the docs (so no experience) of sys:get_status/1 it looks like you could implement Module:format_status/2 and don't return the internal state, then you could use internal state to store it. http://www.erlang.org/doc/man/gen_server.html#Module:format_status-2 -Mark On Mon, Sep 1, 2014 at 11:10 PM, Chris de Villiers < chrisdevilliers@REDACTED> wrote: > Hello > > I want to make requests to a web service which authenticates users > with a key and secret. I will be implementing their API with a > gen_server and need to place the access credentials somewhere "safe". > By safe I mean It should be out of plain sight and not accessible to a > user that can attach to the VM. I also do not want them to show up in > stack traces or kernel/SASL logs should the gen_server die > unexpectedly. > > The application's environment is obviously ruled out. Storing them in > the gen_server's state is also no good because sys:get_status/1 gives > them away. > > I thought about placing them in an ETS table private to the gen_server > process. Is it possible for another process to read private ETS > tables somehow? > > Any other suggestions how I can handle this situation? I do not want > to start a discussion about OS level security. Lets assume someone > gets access to the user account under which the VM runs and can attach > to it. > > Regards, > chris > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Mark Nijhof t: @MarkNijhof s: marknijhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.nijhof@REDACTED Tue Sep 2 00:15:17 2014 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Tue, 2 Sep 2014 00:15:17 +0200 Subject: [erlang-questions] Process state and sensitive information In-Reply-To: References: Message-ID: An other question I would have is, how are you going to supply the credentials to this gen_server to to make these API calls? -Mark On Tue, Sep 2, 2014 at 12:12 AM, Mark Nijhof wrote: > Just from reading the docs (so no experience) of sys:get_status/1 it looks > like you could implement Module:format_status/2 and don't return the > internal state, then you could use internal state to store it. > > http://www.erlang.org/doc/man/gen_server.html#Module:format_status-2 > > -Mark > > > > On Mon, Sep 1, 2014 at 11:10 PM, Chris de Villiers < > chrisdevilliers@REDACTED> wrote: > >> Hello >> >> I want to make requests to a web service which authenticates users >> with a key and secret. I will be implementing their API with a >> gen_server and need to place the access credentials somewhere "safe". >> By safe I mean It should be out of plain sight and not accessible to a >> user that can attach to the VM. I also do not want them to show up in >> stack traces or kernel/SASL logs should the gen_server die >> unexpectedly. >> >> The application's environment is obviously ruled out. Storing them in >> the gen_server's state is also no good because sys:get_status/1 gives >> them away. >> >> I thought about placing them in an ETS table private to the gen_server >> process. Is it possible for another process to read private ETS >> tables somehow? >> >> Any other suggestions how I can handle this situation? I do not want >> to start a discussion about OS level security. Lets assume someone >> gets access to the user account under which the VM runs and can attach >> to it. >> >> Regards, >> chris >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Mark Nijhof > t: @MarkNijhof > s: marknijhof > > -- Mark Nijhof t: @MarkNijhof s: marknijhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From ilya.khlopotov@REDACTED Tue Sep 2 00:42:15 2014 From: ilya.khlopotov@REDACTED (ILYA Khlopotov) Date: Mon, 1 Sep 2014 15:42:15 -0700 Subject: [erlang-questions] Process state and sensitive information In-Reply-To: References: Message-ID: Don't use gen_server since it is intro-spectable. gen_server process supports something called `secure` process however it was not designed to be none intro-spectable. If `secure` process is hybernated you can read it's state using sys module. So I would recommend implementation of a process with otp compatible loop and suspend_loop where you can block otp introspection tools (just ignore messages used by them). https://github.com/erlang/otp/blob/maint/lib/stdlib/src/sys.erl#L382 Another approach is to store data encrypted in gen_server state and pass decryption key from the caller (where it can be intercepted as well BTW). In any case it is always would be a way to get your data out. Best regards, ILYA On Mon, Sep 1, 2014 at 3:15 PM, Mark Nijhof wrote: > An other question I would have is, how are you going to supply the > credentials to this gen_server to to make these API calls? > > -Mark > > > On Tue, Sep 2, 2014 at 12:12 AM, Mark Nijhof > wrote: >> >> Just from reading the docs (so no experience) of sys:get_status/1 it looks >> like you could implement Module:format_status/2 and don't return the >> internal state, then you could use internal state to store it. >> >> http://www.erlang.org/doc/man/gen_server.html#Module:format_status-2 >> >> -Mark >> >> >> >> On Mon, Sep 1, 2014 at 11:10 PM, Chris de Villiers >> wrote: >>> >>> Hello >>> >>> I want to make requests to a web service which authenticates users >>> with a key and secret. I will be implementing their API with a >>> gen_server and need to place the access credentials somewhere "safe". >>> By safe I mean It should be out of plain sight and not accessible to a >>> user that can attach to the VM. I also do not want them to show up in >>> stack traces or kernel/SASL logs should the gen_server die >>> unexpectedly. >>> >>> The application's environment is obviously ruled out. Storing them in >>> the gen_server's state is also no good because sys:get_status/1 gives >>> them away. >>> >>> I thought about placing them in an ETS table private to the gen_server >>> process. Is it possible for another process to read private ETS >>> tables somehow? >>> >>> Any other suggestions how I can handle this situation? I do not want >>> to start a discussion about OS level security. Lets assume someone >>> gets access to the user account under which the VM runs and can attach >>> to it. >>> >>> Regards, >>> chris >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> -- >> Mark Nijhof >> t: @MarkNijhof >> s: marknijhof >> > > > > -- > Mark Nijhof > t: @MarkNijhof > s: marknijhof > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From tony@REDACTED Tue Sep 2 00:57:46 2014 From: tony@REDACTED (Tony Rogvall) Date: Tue, 2 Sep 2014 00:57:46 +0200 Subject: [erlang-questions] Process state and sensitive information In-Reply-To: References: Message-ID: <6DB816EE-EF0C-42FE-BF55-B7E8522B24CB@rogvall.se> On 1 sep 2014, at 23:10, Chris de Villiers wrote: > Hello > > I want to make requests to a web service which authenticates users > with a key and secret. I will be implementing their API with a > gen_server and need to place the access credentials somewhere "safe". > By safe I mean It should be out of plain sight and not accessible to a > user that can attach to the VM. I also do not want them to show up in > stack traces or kernel/SASL logs should the gen_server die > unexpectedly. > > The application's environment is obviously ruled out. Storing them in > the gen_server's state is also no good because sys:get_status/1 gives > them away. > > I thought about placing them in an ETS table private to the gen_server > process. Is it possible for another process to read private ETS > tables somehow? > private ets tables should be private. > Any other suggestions how I can handle this situation? I do not want > to start a discussion about OS level security. Lets assume someone > gets access to the user account under which the VM runs and can attach > to it. > consider process_flag(sensitive, true). if you have a "named" service then try find a way of protecting the service from a middle-man "attack". A fun shell example (made super easy thanks to Anthony Ramine) where a middle-man process prints the first 10 request and then restore the original process registration. > Name = file_server_2. > Orig = whereis(Name) > MiddleMan = spawn(fun() -> unregister(Name), register(Name,self()), (fun Loop(0) -> unregister(Name), register(Name,Orig); Loop(I) -> receive R -> io:format("request:~p\n", [R]), Orig ! R, Loop(I-1) end end)(10) end). With some minor hacking the replies could be captured as well. /Tony > Regards, > chris > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" From steve@REDACTED Tue Sep 2 07:27:48 2014 From: steve@REDACTED (Steve Strong) Date: Tue, 2 Sep 2014 07:27:48 +0200 Subject: [erlang-questions] Process state and sensitive information In-Reply-To: References: Message-ID: If they have access to the user account, then there's nothing you can do to secure the VM; the best you'll manage is to make it harder. For example, they could attach to the Vm, load a linked in driver, and then dump all memory to disk. Your safest approach is to assume they can see it, and to make sure that what they see is of little use. So make sure you don't store plaintext credentials, instead store salted hashes - at least now they can't steal the passwords... You could also have a second account that runs a second VM - have that VM handle all the security sensitive information and expose a thin API to the first; now the first account is less privileged and less damage can be done. Of course, that just moves the problem to the second account, but since it is doing less, it should be easier to secure. Kind of hard to suggest much more with talking about OS hardening; if your OS isn't secured then you really might as well give up ;) Cheers, Steve Sent from my iPhone > On 1 Sep 2014, at 23:10, Chris de Villiers wrote: > > Hello > > I want to make requests to a web service which authenticates users > with a key and secret. I will be implementing their API with a > gen_server and need to place the access credentials somewhere "safe". > By safe I mean It should be out of plain sight and not accessible to a > user that can attach to the VM. I also do not want them to show up in > stack traces or kernel/SASL logs should the gen_server die > unexpectedly. > > The application's environment is obviously ruled out. Storing them in > the gen_server's state is also no good because sys:get_status/1 gives > them away. > > I thought about placing them in an ETS table private to the gen_server > process. Is it possible for another process to read private ETS > tables somehow? > > Any other suggestions how I can handle this situation? I do not want > to start a discussion about OS level security. Lets assume someone > gets access to the user account under which the VM runs and can attach > to it. > > Regards, > chris > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From attila.r.nohl@REDACTED Tue Sep 2 07:53:35 2014 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 2 Sep 2014 07:53:35 +0200 Subject: [erlang-questions] Process state and sensitive information In-Reply-To: References: Message-ID: 2014-09-01 23:10 GMT+02:00 Chris de Villiers : [...] > Any other suggestions how I can handle this situation? I do not want > to start a discussion about OS level security. Lets assume someone > gets access to the user account under which the VM runs and can attach > to it. If the attacker can attach to the VM, you lost. The attacker can turn on trace and see all of your function calls with their parameters, so you can't pass the password to any functions. The attacker can also see all of your messages, so you can't send the password to any other process. The attacker can even load his/her own code and replace yours. From publicityifl@REDACTED Tue Sep 2 09:23:24 2014 From: publicityifl@REDACTED (publicityifl@REDACTED) Date: Tue, 02 Sep 2014 00:23:24 -0700 (PDT) Subject: [erlang-questions] Third call for papers, IFL 2014 Message-ID: <5405706c.2369b40a.7eb4.ffff976f@mx.google.com> Hello, Please, find below the third call for papers for IFL 2014. The submission page is now open. The submission date has been delayed to Sep. 8 2014 anywhere on the world. Please forward these to anyone you think may be interested. Apologies for any duplicates you may receive. best regards, Jurriaan Hage --- CALL FOR PAPERS 26th SYMPOSIUM ON IMPLEMENTATION AND APPLICATION OF FUNCTIONAL LANGUAGES - IFL 2014 NORTHEASTERN UNIVERSITY/BOSTON, USA OCTOBER 1-3, 2014 http://ifl2014.github.io We are pleased to announce that the 26th edition of the IFL series will be held at Northeastern University in Boston, USA. The symposium will be held from 1st to 3rd of October 2014. Scope ----- The goal of the IFL symposia is to bring together researchers actively engaged in the implementation and application of functional and function-based programming languages. IFL 2014 will be a venue for researchers to present and discuss new ideas and concepts, work in progress, and publication-ripe results related to the implementation and application of functional languages and function-based programming. Following the IFL tradition, IFL 2014 will use a post-symposium review process to produce the formal proceedings. All participants of IFL 2014 are invited to submit either a draft paper or an extended abstract describing work to be presented at the symposium. At no time may work submitted to IFL be simultaneously submitted to other venues; submissions must adhere to ACM SIGPLAN's republication policy: http://www.sigplan.org/Resources/Policies/Republication The submissions will be screened by the program committee chair to make sure they are within the scope of IFL, and will appear in the draft proceedings distributed at the symposium. Submissions appearing in the draft proceedings are not peer-reviewed publications. Hence, publications that appear only in the draft proceedings do not count as publication for the ACM SIGPLAN republication policy. After the symposium, authors will be given the opportunity to incorporate the feedback from discussions at the symposium and will be invited to submit a revised full article for the formal review process. From the revised submissions, the program committee will select papers for the formal proceedings considering their correctness, novelty, originality, relevance, significance, and clarity. Submission Details ------------------ Submission deadline draft papers: September 8 Notification of acceptance for presentation: September 10 Early registration deadline: September 11 Late registration deadline: September 17 Submission deadline for pre-symposium proceedings: September 24 26th IFL Symposium: October 1-3 Submission deadline for post-symposium proceedings: December 15 Notification of acceptance for post-symposium proceedings: January 31 2015 Camera-ready version for post-symposium proceedings: March 15 2015 Prospective authors are encouraged to submit papers or extended abstracts to be published in the draft proceedings and to present them at the symposium. All contributions must be written in English. Papers must adhere to the standard ACM two columns conference format. For the pre-symposium proceedings we adopt a 'weak' page limit of 12 pages. For the post-symposium proceedings the page limit of 12 pages is firm. A suitable document template for LaTeX can be found at: http://www.acm.org/sigs/sigplan/authorInformation.htm Papers should be submitted online at https://easychair.org/conferences/?conf=ifl2014 Topics ------ IFL welcomes submissions describing practical and theoretical work as well as submissions describing applications and tools in the context of functional programming. If you are not sure whether your work is appropriate for IFL 2014, please contact the PC chair at samth@REDACTED Topics of interest include, but are not limited to: ??? language concepts ??? type systems, type checking, type inferencing ??? compilation techniques ??? staged compilation ??? run-time function specialization ??? run-time code generation ??? partial evaluation ??? (abstract) interpretation ??? metaprogramming ??? generic programming ??? automatic program generation ??? array processing ??? concurrent/parallel programming ??? concurrent/parallel program execution ??? embedded systems ??? web applications ??? (embedded) domain specific languages ??? security ??? novel memory management techniques ??? run-time profiling performance measurements ??? debugging and tracing ??? virtual/abstract machine architectures ??? validation, verification of functional programs ??? tools and programming techniques ??? (industrial) applications Peter Landin Prize ------------------ The Peter Landin Prize is awarded to the best paper presented at the symposium every year. The honoured article is selected by the program committee based on the submissions received for the formal review process. The prize carries a cash award equivalent to 150 Euros. Programme committee ------------------- Sam Tobin-Hochstadt, Indiana University (Chair) Rinus Plasmeijer, Radboud University Nijmegen (Co-Chair) Atze Dijkstra, Utrecht University Colin Runciman, University of York Graham Hutton, University of Nottingham Mary Sheeran, Chalmers University of Technology Patricia Johann, Appalachian State University Matthew Fluet, Rochester Institute of Technology Josef Svenningsson, Chalmers University of Technology Ma??gorzata Biernacka, University of Wroclaw Peter Achten, Radboud Univerity Nijmegen Laura Castro, University of A Coru??a Hai Paul Liu, Intel Labs Kathryn Gray, Cambridge University Lars Bergstrom, Mozilla Research Lindsey Kuper, Indiana University Nicolas Wu, Oxford T. Stephen Strickland, University of Maryland Xavier Clerc, INRIA Venue ----- The 26th IFL will be held in association with the College of Computer and Information Science at Northeastern University. It can be reached quickly and easily by public transport. From ulf@REDACTED Tue Sep 2 10:13:52 2014 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 2 Sep 2014 10:13:52 +0200 Subject: [erlang-questions] Process state and sensitive information In-Reply-To: References: Message-ID: The idea behind the process_flag(sensitive, true) function was to let you disable trace and introspection. As background info, tele- and datacoms systems must usually provide support for ?Lawful Intercept? [1], i.e. wiretapping by the authorities. It should not be ?possible' for a human operator or service technician, even with console access, to see which phones are being monitored? for some pragmatic interpretation of ?possible?. This is of course hard to accomplish with Erlang, but we decided that a sufficient level of security could be achieved by combining private ets tables, process_flag(sensitive, true), secure shell, a carefully limited Bash shell and audit logging. Basically, you shouldn?t be able to inspect the sensitive data without somehow brutalizing the system, and this would show up in the audit log. Now, if the perpetrator has root access and physical access to the machine, it becomes trickier, but then, one has to rely on physical premises security and surveillance. BR, Ulf W [1] http://en.wikipedia.org/wiki/Lawful_interception On 02 Sep 2014, at 07:53, Attila Rajmund Nohl wrote: > 2014-09-01 23:10 GMT+02:00 Chris de Villiers : > [...] >> Any other suggestions how I can handle this situation? I do not want >> to start a discussion about OS level security. Lets assume someone >> gets access to the user account under which the VM runs and can attach >> to it. > > If the attacker can attach to the VM, you lost. The attacker can turn > on trace and see all of your function calls with their parameters, so > you can't pass the password to any functions. The attacker can also > see all of your messages, so you can't send the password to any other > process. The attacker can even load his/her own code and replace > yours. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From sebovick@REDACTED Tue Sep 2 12:12:10 2014 From: sebovick@REDACTED (Seb Seb) Date: Tue, 2 Sep 2014 12:12:10 +0200 Subject: [erlang-questions] SNMP manager buggy and/or partialy implemented? (snmp v3 usm problems) Message-ID: Hello, After a full week trying to make snmp application working, specially the manager, i have noticed these facts: - miss key feature for snmp v3 "agentId discovery", the code exist but is commented out (???) - normal/synchronization reports from agents are not interpreted and treated as errors (discovery) or treated as error and interpreted (in the case of sync engineBoots, engineTime), - is not yet finished (code comments are freaky: "bug bug bug","what the ****", "what about ..."), - is not maintained , as far as I can go in the OTP github repository same comments... - so complicated to configure, that i don't know if there are bugs, or if it is me who miss the point (for example, I can successfully run v3 in most modes and protocols, but agents (cisco routers) return snmpInASNParseErrs, when using auth/sha in conjunction with priv/des while it works perfectly with auth/md5 priv/des) How should I consider the development status of this application? Does someone here use it successfully for managing heterogeneous elements? Sebovick. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Tue Sep 2 12:43:16 2014 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 2 Sep 2014 12:43:16 +0200 Subject: [erlang-questions] Erlang and Akka In-Reply-To: <5404DFBE.6040907@power.com.pl> References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <1409314571.19205.YahooMailNeo@web163602.mail.gq1.yahoo.com> <5404DFBE.6040907@power.com.pl> Message-ID: > Maybe internalizing UBF into erlang somehow would be a way to go? > +1 /Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisdevilliers@REDACTED Tue Sep 2 12:41:33 2014 From: chrisdevilliers@REDACTED (Chris de Villiers) Date: Tue, 2 Sep 2014 12:41:33 +0200 Subject: [erlang-questions] Process state and sensitive information In-Reply-To: References: Message-ID: Hi, thanks for your input. @Mark > An other question I would have is, how are you going to supply the credentials to this gen_server to to make these API calls? I was thinking to call the gen_server from the shell after application startup to pass it the credentials. Another approach would be to store the credentials encrypted on disk and have the gen_server decrypt them upon use. But essentially this is just another instance of the same problem because you still have to store the private key's pass phrase or the symmetric key somewhere. @Tony The sensitive process_flag takes care of the stack traces and error reports, thanks! @Steve I realise I should be tackling the problem from the OS side first. My programming ability far exceeds my sysadmin skills that is why my first instinct is the other direction. I think I will proceed as follows: 1. Restrict user account under which the VM is executed (Any suggestions?) 2. Store access credentials encrypted on disk. 3. Set the sensitive process_flag. 4. Call the gen_server with the private key's pass phrase and store it in private ETS storage. 5. Decrypt access credentials every time upon use. Any comments? Best regards, chris On Tue, Sep 2, 2014 at 7:27 AM, Steve Strong wrote: > If they have access to the user account, then there's nothing you can do to secure the VM; the best you'll manage is to make it harder. > > For example, they could attach to the Vm, load a linked in driver, and then dump all memory to disk. > > Your safest approach is to assume they can see it, and to make sure that what they see is of little use. So make sure you don't store plaintext credentials, instead store salted hashes - at least now they can't steal the passwords... > > You could also have a second account that runs a second VM - have that VM handle all the security sensitive information and expose a thin API to the first; now the first account is less privileged and less damage can be done. Of course, that just moves the problem to the second account, but since it is doing less, it should be easier to secure. > > Kind of hard to suggest much more with talking about OS hardening; if your OS isn't secured then you really might as well give up ;) > > Cheers, > > Steve > > Sent from my iPhone > >> On 1 Sep 2014, at 23:10, Chris de Villiers wrote: >> >> Hello >> >> I want to make requests to a web service which authenticates users >> with a key and secret. I will be implementing their API with a >> gen_server and need to place the access credentials somewhere "safe". >> By safe I mean It should be out of plain sight and not accessible to a >> user that can attach to the VM. I also do not want them to show up in >> stack traces or kernel/SASL logs should the gen_server die >> unexpectedly. >> >> The application's environment is obviously ruled out. Storing them in >> the gen_server's state is also no good because sys:get_status/1 gives >> them away. >> >> I thought about placing them in an ETS table private to the gen_server >> process. Is it possible for another process to read private ETS >> tables somehow? >> >> Any other suggestions how I can handle this situation? I do not want >> to start a discussion about OS level security. Lets assume someone >> gets access to the user account under which the VM runs and can attach >> to it. >> >> Regards, >> chris >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -- chris de villiers From jesper.louis.andersen@REDACTED Tue Sep 2 13:51:50 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 2 Sep 2014 13:51:50 +0200 Subject: [erlang-questions] Process state and sensitive information In-Reply-To: References: Message-ID: On Tue, Sep 2, 2014 at 12:41 PM, Chris de Villiers < chrisdevilliers@REDACTED> wrote: > Any comments? Given what Ulf wrote, the sensitive flag serves a different purpose: it protects the administrator and not the user. An admin cannot inadvertently read credentials because it won't show up on his screen unless he forcefully bends the system. Consider if you want to protect against data which is hot in memory. If you decrypt, how do you make sure the data you decrypted is overwritten straight away, once you don't need them anymore? This approach is so much against the Erlang philosophy so it might require a NIF to pull off. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Sep 2 13:52:23 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 2 Sep 2014 13:52:23 +0200 Subject: [erlang-questions] Erlang and Akka In-Reply-To: References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <1409314571.19205.YahooMailNeo@web163602.mail.gq1.yahoo.com> <5404DFBE.6040907@power.com.pl> Message-ID: +1000 :) On Tue, Sep 2, 2014 at 12:43 PM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > > Maybe internalizing UBF into erlang somehow would be a way to go? >> > > +1 > > /Richard > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Tue Sep 2 14:13:02 2014 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 2 Sep 2014 14:13:02 +0200 Subject: [erlang-questions] Erlang and Akka In-Reply-To: <5404DFBE.6040907@power.com.pl> References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <1409314571.19205.YahooMailNeo@web163602.mail.gq1.yahoo.com> <5404DFBE.6040907@power.com.pl> Message-ID: On Mon, Sep 1, 2014 at 11:06 PM, Wojtek Narczy?ski wrote: Maybe internalizing UBF into erlang somehow would be a way to go? I?m all for that too. There are some higher-level questions to be answered: - should the contract checker be hard-linked to UBF(A), the wire format? If used only internally, it seems reasonable to let it understand the Erlang binary format - since 2002, Erlang got type descriptions and specifications; should these be used for describing the protocols? Again, it seems reasonable to be able to refer to existing types from the source code. - should it be possible to turn the contract checker on and off, preferably without performance impact when off? regards, Vlad ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Tue Sep 2 15:58:11 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 02 Sep 2014 09:58:11 -0400 Subject: [erlang-questions] Erlang and Akka In-Reply-To: References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <1409314571.19205.YahooMailNeo@web163602.mail.gq1.yahoo.com> <5404DFBE.6040907@power.com.pl> Message-ID: <5405CCF3.2010905@meetinghouse.net> Pardon my ignorance... what's UBF? Jesper Louis Andersen wrote: > +1000 :) > > > On Tue, Sep 2, 2014 at 12:43 PM, Richard Carlsson > > wrote: > > > Maybe internalizing UBF into erlang somehow would be a way to go? > > > +1 > > /Richard > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > J. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From norton@REDACTED Tue Sep 2 16:34:07 2014 From: norton@REDACTED (Norton Joseph Wayne) Date: Tue, 2 Sep 2014 23:34:07 +0900 Subject: [erlang-questions] Erlang and Akka In-Reply-To: <5405CCF3.2010905@meetinghouse.net> References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <1409314571.19205.YahooMailNeo@web163602.mail.gq1.yahoo.com> <5404DFBE.6040907@power.com.pl> <5405CCF3 .2010905@meetinghouse.net> Message-ID: <2B2ADF29-0851-4FE1-BC70-EA9DCFEE023E@lovely.email.ne.jp> Here is a link to Joe Armstrong's UBF site: - http://www.sics.se/~joe/ubf/site/home.html Plus a few links to a UBF project on GitHub: - http://ubf.github.io/ubf/ - http://ubf.github.io/ubf/ubf-user-guide.en.html This UBF project and code base started with Joe's original implementation. If you have any questions, please let me know. Joe N. On 2014/09/02, at 22:58, Miles Fidelman wrote: > Pardon my ignorance... what's UBF? > > Jesper Louis Andersen wrote: >> +1000 :) >> >> >> On Tue, Sep 2, 2014 at 12:43 PM, Richard Carlsson > wrote: >> >> >> Maybe internalizing UBF into erlang somehow would be a way to go? >> >> >> +1 >> >> /Richard >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> -- >> J. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From richard.youngkin@REDACTED Tue Sep 2 16:39:17 2014 From: richard.youngkin@REDACTED (Youngkin, Rich) Date: Tue, 2 Sep 2014 08:39:17 -0600 Subject: [erlang-questions] Erlang and Akka Message-ID: >> On 08/28, Xiao Jia wrote: >> But in such languages as C++/Java/... we are not forced to mutate variables >> so we can still employ some coding style/guidance of creating new objects >> instead of mutating existing objects most of the time. ... On 8/28 Fred Hebert wrote: > Ah but did the people writing the libraries you use think the same way? > One of the advantages of Erlang is that because of its constraints, > *everyone* programming in it ends up making decisions that are generally > amenable to concurrency and to some extent, distribution. Those who > don't tend to make libraries that hardly scale on a single node to begin > with (with heavy bottlenecks and whatnot). ... Is this true of Erlang libraries that use NIFs and or Port drivers? Specifically, I'm thinking of blocking the VM scheduler(s). Cheers, Rich -------------- next part -------------- An HTML attachment was scrubbed... URL: From norton@REDACTED Tue Sep 2 17:41:32 2014 From: norton@REDACTED (Norton Joseph Wayne) Date: Wed, 3 Sep 2014 00:41:32 +0900 Subject: [erlang-questions] Erlang and Akka In-Reply-To: References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <1409314571.19205.YahooMailNeo@web163602.mail.gq1.yahoo.com> <5404DFBE.6040907@power.com.pl> Message-ID: <9BFFFD5F-9A8F-48AF-9DD5-AF2FEE14710D@lovely.email.ne.jp> A few comments. On 2014/09/02, at 21:13, Vlad Dumitrescu wrote: > On Mon, Sep 1, 2014 at 11:06 PM, Wojtek Narczy?ski wrote: > > should the contract checker be hard-linked to UBF(A), the wire format? If used only internally, it seems reasonable to let it understand the Erlang binary format As an example, the implementation of UBF on GitHub (http://ubf.github.io/ubf/ubf-user-guide.en.html) supports several network formats including UBF(A), Erlang binary format (a.k.a. EBF), and a local procedure call (a.k.a. LPC). The network formats share the same contract checker implementation. This contract checker implementation operates directly on Erlang terms. There is a lot of surrounding code but the core of the checker is a function that checks Erlang terms against a contract (https://github.com/ubf/ubf/blob/master/src/contracts.erl#L108). > since 2002, Erlang got type descriptions and specifications; should these be used for describing the protocols? Again, it seems reasonable to be able to refer to existing types from the source code. Yes, agreed. > should it be possible to turn the contract checker on and off, preferably without performance impact when off? Yes, agreed. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Sep 2 18:07:41 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 2 Sep 2014 18:07:41 +0200 Subject: [erlang-questions] Erlang and Akka In-Reply-To: References: Message-ID: On Tue, Sep 2, 2014 at 4:39 PM, Youngkin, Rich wrote: > Is this true of Erlang libraries that use NIFs and or Port drivers? > Specifically, I'm thinking of blocking the VM scheduler(s). No, not in general. Correctly written NIFs executing on dirty schedulers and correctly written port drivers does not have these problems. Also, just because you have the Erlang tooling available doesn't automatically give you the magic: * You can still overflow the atom table * You can still spawn non-linked processes that never stop leading to memory exhaustion * You can still deadlock the system Mind you, it is *hard* to do these things in Erlang if you just adhere to some simple basic rules. But you end up seeing it all over time. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Tue Sep 2 20:52:15 2014 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 2 Sep 2014 20:52:15 +0200 Subject: [erlang-questions] Erlang and Akka In-Reply-To: <9BFFFD5F-9A8F-48AF-9DD5-AF2FEE14710D@lovely.email.ne.jp> References: <2ec11b1134cd6c8104ee3996bd75d475@webmail.webfaction.com> <3270b8a25ba1dd2d2686afeb0800cae3@webmail.webfaction.com> <5A3AC1B4-18C5-482B-B172-4BB6F9A72809@cs.otago.ac.nz> <78B008BE-7BC3-4B0B-BC61-9A1D1F9A9EE1@cs.otago.ac.nz> <1409314571.19205.YahooMailNeo@web163602.mail.gq1.yahoo.com> <5404DFBE.6040907@power.com.pl> <9BFFFD5F-9A8F-48AF-9DD5-AF2FEE14710D@lovely.email.ne.jp> Message-ID: On Tue, Sep 2, 2014 at 5:41 PM, Norton Joseph Wayne < norton@REDACTED> wrote: > As an example, the implementation of UBF on GitHub ( > http://ubf.github.io/ubf/ubf-user-guide.en.html) supports several network > formats including UBF(A), Erlang binary format (a.k.a. EBF), and a local > procedure call (a.k.a. LPC). The network formats share the same contract > checker implementation. > I saw later your note about the project. Great stuff! It would need a bit more of marketing, though :-) best regards, Vlad On Tue, Sep 2, 2014 at 5:41 PM, Norton Joseph Wayne < norton@REDACTED> wrote: > > A few comments. > > On 2014/09/02, at 21:13, Vlad Dumitrescu wrote: > > On Mon, Sep 1, 2014 at 11:06 PM, Wojtek Narczy?ski > wrote: > > - should the contract checker be hard-linked to UBF(A), the wire > format? If used only internally, it seems reasonable to let it understand > the Erlang binary format > > As an example, the implementation of UBF on GitHub ( > http://ubf.github.io/ubf/ubf-user-guide.en.html) supports several network > formats including UBF(A), Erlang binary format (a.k.a. EBF), and a local > procedure call (a.k.a. LPC). The network formats share the same contract > checker implementation. > > This contract checker implementation operates directly on Erlang terms. > There is a lot of surrounding code but the core of the checker is a > function that checks Erlang terms against a contract ( > https://github.com/ubf/ubf/blob/master/src/contracts.erl#L108). > > > - since 2002, Erlang got type descriptions and specifications; should > these be used for describing the protocols? Again, it seems reasonable to > be able to refer to existing types from the source code. > > > Yes, agreed. > > > - should it be possible to turn the contract checker on and off, > preferably without performance impact when off? > > Yes, agreed. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Tobias.Schlager@REDACTED Wed Sep 3 10:08:54 2014 From: Tobias.Schlager@REDACTED (Tobias Schlager) Date: Wed, 3 Sep 2014 08:08:54 +0000 Subject: [erlang-questions] [ANN] rebar_escript_plugin Message-ID: <12F2115FD1CCEE4294943B2608A18FA30122360EBF@MAIL01.win.lbaum.eu> Hi, the rebar_escript_plugin [1] is an alternative attempt on escriptizing applications and releases with rebar. It was written out of necessity to be able to package this fileserver utility [2] into one executable file. I hope you will find it useful. Regards Tobias [1] https://github.com/schlagert/rebar_escript_plugin [2] https://github.com/schlagert/serve_it From tuncer.ayaz@REDACTED Wed Sep 3 13:15:43 2014 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 3 Sep 2014 13:15:43 +0200 Subject: [erlang-questions] [ANN] rebar_escript_plugin In-Reply-To: <12F2115FD1CCEE4294943B2608A18FA30122360EBF@MAIL01.win.lbaum.eu> References: <12F2115FD1CCEE4294943B2608A18FA30122360EBF@MAIL01.win.lbaum.eu> Message-ID: On Wed, Sep 3, 2014 at 10:08 AM, Tobias Schlager wrote: > Hi, > > the rebar_escript_plugin [1] is an alternative attempt on > escriptizing applications and releases with rebar. It was written > out of necessity to be able to package this fileserver utility [2] > into one executable file. > > I hope you will find it useful. > > Regards > Tobias > > [1] https://github.com/schlagert/rebar_escript_plugin > [2] https://github.com/schlagert/serve_it Hi Tobias, thanks for sharing the project(s), and here are some comments based on a quick look at the sources, which I hope you'll find useful: (1) runs automatically as post_compile or post_generate hook We've been discussing support for user defined aliases like the built-in prepare-deps and refresh-deps. The motivation is something else, but it could also make this more convenient. ==> can be made more convenient in rebar, and this also touches on the topic of being able to select config variants on the command line. Related: https://github.com/rebar/rebar/issues/58 https://github.com/rebar/rebar/issues/206 https://github.com/rebar/rebar/issues/283 (2) avoids the necessity of duplicate configuration rebar escriptize cannot unconditionally include all deps in the escript for obvious reasons, but if this is a common use case, it'd be easy to add a rebar.config switch that default to off. ==> can be folded into rebar upstream (3) provides a default main/1 function, starting all needed applications as a normal Erlang release would We could do the same, but I'm not sure there is a single way to start an escript. Your runner seems to spin up all bundled apps and wait for termination, but is this generally applicable? Does it work with all all kinds of escripts, including interactive tools? ==> can be folded into rebar upstream, if there's consensus on runner behavior (4) makes your application's priv directory contents accessible at runtime The plugin extracts priv into a temp directory, and while that works, the long-term solution is to teach escript and erl_prim_loader to do the right thing. We should also add a convention (filename, etc.) based way to bundle arch-specific files (e.g. .so, .dll). ==> not sure this can be folded into rebar upstream (5) additionally allows escript creation based on release specifications ==> can be folded into rebar upstream From madhukars@REDACTED Wed Sep 3 12:55:42 2014 From: madhukars@REDACTED (Madhukar Sah) Date: Wed, 03 Sep 2014 16:25:42 +0530 Subject: [erlang-questions] mnesia feasibility Message-ID: <5406F3AE.5070507@utl.in> I am not able to decide if mnesia can handle my requirement or not. Please help. The requirement is as described below: I am running 16 nodes that are constantly pushing data in mnesia database located in a card powered by erlang. The card is running at 260 MHz with 128 MB ram. The data is being maintained as RAM_COPY as well as DISC_COPY in the card. Each of the 16 nodes running on a PC are pumping records every 1 sec. The result is mnesia is overloading and after some time mnesia is taking around 10 seconds to write one record into mnesia table. I have checked the same thing by putting the data as RAM_COPY as well but mnesia is taking again around 10 seconds to write one record in mnesia table. I have also used dirty read and write but with little improvement only. Please suggest Regards, Madhukar Sah From jesper.louis.andersen@REDACTED Wed Sep 3 13:34:21 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 3 Sep 2014 13:34:21 +0200 Subject: [erlang-questions] mnesia feasibility In-Reply-To: <5406F3AE.5070507@utl.in> References: <5406F3AE.5070507@utl.in> Message-ID: On Wed, Sep 3, 2014 at 12:55 PM, Madhukar Sah wrote: > I am running 16 nodes that are constantly pushing data in mnesia database > located in a card powered by erlang. The card is running at 260 MHz with > 128 MB ram. The data is being maintained as RAM_COPY as well as DISC_COPY > in the card. Each of the 16 nodes running on a PC are pumping records every > 1 sec. The result is mnesia is overloading and after some time mnesia is > taking around 10 seconds to write one record into mnesia table. I have > checked the same thing by putting the data as RAM_COPY as well but mnesia > is taking again around 10 seconds to write one record in mnesia table. Something does not make sense here. disc_copies also stores the table in RAM but backs it on disc, so you don't need ram_copies in that situation. The overload messages vary, but the most common one is when mnesia is not able to dump the in-memory table to disk before it has to dump again. You say you have a 260mhz CPU and 128 MB ram, which is a fairly small system. What kind of disc I/O are you getting in such a system? Some napkin math usually helps here to figure out the location of an eventual bottleneck. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Wed Sep 3 13:41:37 2014 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 3 Sep 2014 13:41:37 +0200 Subject: [erlang-questions] mnesia feasibility In-Reply-To: <5406F3AE.5070507@utl.in> References: <5406F3AE.5070507@utl.in> Message-ID: <079056A6-C69F-44F7-B693-1EA058C881B3@feuerlabs.com> So you have 16 or 17 copies of the table(s)? Or did I misunderstand? You can write to a table without having a local copy. Replicating to 16 nodes will be very expensive, and is overkill at least from a survivability standpoint. BR, Ulf W On 03 Sep 2014, at 12:55, Madhukar Sah wrote: > I am not able to decide if mnesia can handle my requirement or not. Please help. The requirement is as described below: > > I am running 16 nodes that are constantly pushing data in mnesia database located in a card powered by erlang. The card is running at 260 MHz with 128 MB ram. The data is being maintained as RAM_COPY as well as DISC_COPY in the card. Each of the 16 nodes running on a PC are pumping records every 1 sec. The result is mnesia is overloading and after some time mnesia is taking around 10 seconds to write one record into mnesia table. I have checked the same thing by putting the data as RAM_COPY as well but mnesia is taking again around 10 seconds to write one record in mnesia table. > > I have also used dirty read and write but with little improvement only. > > Please suggest > > Regards, > Madhukar Sah > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From mononcqc@REDACTED Wed Sep 3 15:04:24 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 3 Sep 2014 09:04:24 -0400 Subject: [erlang-questions] [ANN] rebar_escript_plugin In-Reply-To: References: <12F2115FD1CCEE4294943B2608A18FA30122360EBF@MAIL01.win.lbaum.eu> Message-ID: <20140903130406.GB69118@ferdair.local> On 09/03, Tuncer Ayaz wrote: > (3) provides a default main/1 function, starting all needed > applications as a normal Erlang release would > > We could do the same, but I'm not sure there is a single way to start > an escript. Your runner seems to spin up all bundled apps and wait for > termination, but is this generally applicable? Does it work with all > all kinds of escripts, including interactive tools? > For this kind of stuff, I'd rather have an eventual harness a bit like this: https://github.com/ferd/start_wrap This thing uses a supervisor bridge to supervise your `main/1` function and then you're able to ship a release using it, giving you the simple part about an escript, while having the ability to ship and boot OTP applications along with it. You can eventually move to a more proper release fairly easily as soon as you OTP-ize all you've got. It's still pretty damn unusable at this point, but this was a proof of concept I quickly hacked together back in March and had forgotten to share around. Regards, Fred. From daniel.goertzen@REDACTED Wed Sep 3 15:47:24 2014 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 3 Sep 2014 08:47:24 -0500 Subject: [erlang-questions] SNMP manager buggy and/or partialy implemented? (snmp v3 usm problems) In-Reply-To: References: Message-ID: Regarding auth/sha: It sounds like you might not be truncating your privacy key to 16 bytes. The md5 hash yields 16 bytes, the sha hash is a bit longer. Make sure you are doing something like... PrivKey = lists:sublist(snmp:passwd2localized_key(sha, PrivPass, EngineId), 16), I am successfully using the snmp app as an agent. The documentation is not bad, but the RFCs were also helpful in gaining understanding. On Tue, Sep 2, 2014 at 5:12 AM, Seb Seb wrote: > Hello, > > After a full week trying to make snmp application working, specially the > manager, i have noticed these facts: > - miss key feature for snmp v3 "agentId discovery", the code exist but is > commented out (???) > - normal/synchronization reports from agents are not interpreted and > treated as errors (discovery) or treated as error and interpreted (in the > case of sync engineBoots, engineTime), > - is not yet finished (code comments are freaky: "bug bug bug","what the > ****", "what about ..."), > - is not maintained , as far as I can go in the OTP github repository same > comments... > - so complicated to configure, that i don't know if there are bugs, or if > it is me who miss the point (for example, I can successfully run v3 in most > modes and protocols, but agents (cisco routers) return snmpInASNParseErrs, > when using auth/sha in conjunction with priv/des while it works perfectly > with auth/md5 priv/des) > > How should I consider the development status of this application? Does > someone here use it successfully for managing heterogeneous elements? > > Sebovick. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Tobias.Schlager@REDACTED Wed Sep 3 15:49:04 2014 From: Tobias.Schlager@REDACTED (Tobias Schlager) Date: Wed, 3 Sep 2014 13:49:04 +0000 Subject: [erlang-questions] [ANN] rebar_escript_plugin In-Reply-To: References: <12F2115FD1CCEE4294943B2608A18FA30122360EBF@MAIL01.win.lbaum.eu>, Message-ID: <12F2115FD1CCEE4294943B2608A18FA30122360F2D@MAIL01.win.lbaum.eu> Hi Tuncer, thank you very much for your feedback! Let me make clear, that I didn't want to bash rebar's escriptizer in any way. rebar is a great project and I would be glad to contribute some of the plugins functionality if you would like. The only reason I went the plugin way was because I wanted something that everybody can use immediately without patching their (committed) rebar. Your comments and critics are always welcome! My thoughts about your points: (2) Why not? As of today, there's no way to specify different types of dependencies (test, compile, etc.) which is no problem for me. I think that including none of the dependecies is not better or worse as including all of them. I wanted to make sure that everything the user may need is in place without configuration. Therefore, the rebar_escript_plugin even searches the configured lib_dirs for non-system applications to include. In my opinion, if you want to avoid, e.g. proper, from being packaged along, you should generate the escript from a release specification. (3) Sorry, this is not yet documented in the README. In order to unpack your 'priv' content the plugin needs to have its own main/1 function called first. However, the default runner (starting all needed applications and block) is only used if you don't specify your own main function. If the module with the name of the escript exports a 'main/1' function the plugin will call it (from the main process) after priv extraction with the original arguments (as rebar would). When writing the runner I had services like the serve_it example application in mind. I assumed we all write proper OTP applications ;-) therefore the simple default runner. (4) This is the tricky part of the plugin and I agree with you it would be awesome if escript and erl_prim_loader would do the right thing automagically. Additionally, escript has no possibility to register signal handlers to run when someone terminates the process, e.g. CTRL-C. Therefore, the plugin (miss-)uses the heart process and proxies the HEART_COMMAND to schedule cleanup of its temporary directory... I couldn't come up with a nicer solution. Signal or at least cleanup handlers would also be a cool OTP contribution. Thanks again for taking the time to share your thoughts. Regards Tobias ________________________________________ Von: Tuncer Ayaz [tuncer.ayaz@REDACTED] Gesendet: Mittwoch, 3. September 2014 13:15 An: Tobias Schlager Cc: Erlang Betreff: Re: [erlang-questions] [ANN] rebar_escript_plugin On Wed, Sep 3, 2014 at 10:08 AM, Tobias Schlager wrote: > Hi, > > the rebar_escript_plugin [1] is an alternative attempt on > escriptizing applications and releases with rebar. It was written > out of necessity to be able to package this fileserver utility [2] > into one executable file. > > I hope you will find it useful. > > Regards > Tobias > > [1] https://github.com/schlagert/rebar_escript_plugin > [2] https://github.com/schlagert/serve_it Hi Tobias, thanks for sharing the project(s), and here are some comments based on a quick look at the sources, which I hope you'll find useful: (1) runs automatically as post_compile or post_generate hook We've been discussing support for user defined aliases like the built-in prepare-deps and refresh-deps. The motivation is something else, but it could also make this more convenient. ==> can be made more convenient in rebar, and this also touches on the topic of being able to select config variants on the command line. Related: https://github.com/rebar/rebar/issues/58 https://github.com/rebar/rebar/issues/206 https://github.com/rebar/rebar/issues/283 (2) avoids the necessity of duplicate configuration rebar escriptize cannot unconditionally include all deps in the escript for obvious reasons, but if this is a common use case, it'd be easy to add a rebar.config switch that default to off. ==> can be folded into rebar upstream (3) provides a default main/1 function, starting all needed applications as a normal Erlang release would We could do the same, but I'm not sure there is a single way to start an escript. Your runner seems to spin up all bundled apps and wait for termination, but is this generally applicable? Does it work with all all kinds of escripts, including interactive tools? ==> can be folded into rebar upstream, if there's consensus on runner behavior (4) makes your application's priv directory contents accessible at runtime The plugin extracts priv into a temp directory, and while that works, the long-term solution is to teach escript and erl_prim_loader to do the right thing. We should also add a convention (filename, etc.) based way to bundle arch-specific files (e.g. .so, .dll). ==> not sure this can be folded into rebar upstream (5) additionally allows escript creation based on release specifications ==> can be folded into rebar upstream From kenneth@REDACTED Wed Sep 3 15:50:51 2014 From: kenneth@REDACTED (Kenneth Lundin) Date: Wed, 3 Sep 2014 15:50:51 +0200 Subject: [erlang-questions] SNMP manager buggy and/or partialy implemented? (snmp v3 usm problems) In-Reply-To: <54070261.8070808@ericsson.com> References: <54070261.8070808@ericsson.com> Message-ID: Hi See embedded answer below. On Tue, Sep 2, 2014 at 12:12 PM, Seb Seb wrote: > Hello, > > After a full week trying to make snmp application working, specially the > manager, i have noticed these facts: > - miss key feature for snmp v3 "agentId discovery", the code exist but is > commented out (???) > - normal/synchronization reports from agents are not interpreted and > treated as errors (discovery) or treated as error and interpreted (in the > case of sync engineBoots, engineTime), > - is not yet finished (code comments are freaky: "bug bug bug","what the > ****", "what about ..."), > - is not maintained , as far as I can go in the OTP github repository same > comments... > - so complicated to configure, that i don't know if there are bugs, or if > it is me who miss the point (for example, I can successfully run v3 in most > modes and protocols, but agents (cisco routers) return snmpInASNParseErrs, > when using auth/sha in conjunction with priv/des while it works perfectly > with auth/md5 priv/des) > > How should I consider the development status of this application? Does > someone here use it successfully for managing heterogeneous elements? > > Sebovick. > > The SNMP agent for sure and the SNMP mgr is used in products since 10 years or more and it actively maintained but this is without use of the SNMP V3. As far as I know we (Ericsson) have never used the SNMP application with V3 in products. The SNMP V3 implementation has been available for +10 years but since it is not or almost not used we have not actively maintained it except the test cases we have which are not exhaustive. Contributions with bugfixes and improvements in this area are are very welcome. /Kenneth, Erlang/OTP Ericsson > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Wed Sep 3 16:16:51 2014 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 3 Sep 2014 16:16:51 +0200 Subject: [erlang-questions] SNMP manager buggy and/or partialy implemented? (snmp v3 usm problems) In-Reply-To: References: Message-ID: 2014-09-02 12:12 GMT+02:00 Seb Seb : > Hello, > > After a full week trying to make snmp application working, specially the > manager, i have noticed these facts: > - miss key feature for snmp v3 "agentId discovery", the code exist but is > commented out (???) I used to work on a product using SNMPv3 (both manager and agent) and we've implemented agentId discovery. Some of it had to be implemented in OTP (it was done back in R14B days) and some of it in our product. A module using the snmpa_discovery_handler had to be implemented and eventually it worked. As it was done about 5 years ago, my memory about the details is quite sketchy. Also we abused the SNMP framework in OTP (e.g. we simulated up to 128 agents in simple agent), so lots of hack was used. [...] > - so complicated to configure, that i don't know if there are bugs, or if it > is me who miss the point (for example, I can successfully run v3 in most > modes and protocols, but agents (cisco routers) return snmpInASNParseErrs, > when using auth/sha in conjunction with priv/des while it works perfectly > with auth/md5 priv/des) That's a long standing bug, exists at least since OTP R14B02. From dominik_pawlak@REDACTED Wed Sep 3 22:27:08 2014 From: dominik_pawlak@REDACTED (Dominik Pawlak) Date: Wed, 03 Sep 2014 22:27:08 +0200 Subject: [erlang-questions] SNMP manager buggy and/or partialy implemented? (snmp v3 usm problems) In-Reply-To: References: Message-ID: <5407799C.8020302@yahoo.co.uk> Hi, I was recently implementing the USM discovery and time synchronization feature on the manager side. You can find my work here (snmp_* branches). This code is based on snmp-4.25.1 - the latest version. We are using this successfully in our product - managing different device types. Dominik On 02.09.2014 12:12, Seb Seb wrote: > Hello, > > After a full week trying to make snmp application working, specially > the manager, i have noticed these facts: > - miss key feature for snmp v3 "agentId discovery", the code exist but > is commented out (???) > - normal/synchronization reports from agents are not interpreted and > treated as errors (discovery) or treated as error and interpreted (in > the case of sync engineBoots, engineTime), > - is not yet finished (code comments are freaky: "bug bug bug","what > the ****", "what about ..."), > - is not maintained , as far as I can go in the OTP github repository > same comments... > - so complicated to configure, that i don't know if there are bugs, or > if it is me who miss the point (for example, I can successfully run v3 > in most modes and protocols, but agents (cisco routers) return > snmpInASNParseErrs, when using auth/sha in conjunction with priv/des > while it works perfectly with auth/md5 priv/des) > > How should I consider the development status of this application? Does > someone here use it successfully for managing heterogeneous elements? > > Sebovick. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From yoursurrogategod@REDACTED Thu Sep 4 02:43:34 2014 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Wed, 3 Sep 2014 20:43:34 -0400 Subject: [erlang-questions] Unable to call functions that are in a module in the same location In-Reply-To: References: Message-ID: Hello, I completely forgot about this problem. Here are some more files that will hopefully get rid of some of the confusion: records.hrl: http://textsnip.com/a148rx polis.erl: http://textsnip.com/bk8xao morphology.erl: http://textsnip.com/8rlag9 genotype.erl: http://textsnip.com/ax1n49 Let me know if there's anything else that I can hopefully shed light on. On Sun, Aug 17, 2014 at 9:59 PM, Gustavo Pires wrote: > Where's the code of construct_Agent(Specie_Id, Agent_Id, SpecCon)? You > only sent construct_Cortex. Probably the problem is there. > > > On Sun, Aug 17, 2014 at 1:31 PM, Steve Strong wrote: > >> The record will be initialised with the defaults as specified in the >> hrl. The code and stacktrace in the gist still suggest that >> SpecCon#constraint.morphology is ?undefined? - so something between the >> record construction below and the crash within the gist must be modifying >> the record. >> >> -- >> Steve Strong >> Sent with Sparrow >> >> On Sunday, 17 August 2014 at 15:36, Yves S. Garret wrote: >> >> The record in question is "constraint". The thing is, when I call it, it >> looks like this with none of the values initialized: >> >> test() -> >> Specie_Id = test, >> Agent_Id = test, >> CloneAgent_Id = test_clone, >> SpecCon = #constraint{}, % <---- HERE IS WHERE THE FIRST INSTANCE IS >> CREATED >> F = fun() -> >> construct_Agent(Specie_Id, Agent_Id, SpecCon), >> clone_Agent(Specie_Id, CloneAgent_Id), >> print(Agent_Id), >> print(CloneAgent_Id), >> delete_Agent(Agent_Id), >> delete_Agent(CloneAgent_Id) >> end, >> >> mnesia:transaction(F). >> >> My question is thus, if I make a record with #constraint{}, does that >> mean that all of its values are by default set to undefined or the default >> values that were specified in records.hrl? >> >> >> >> >> On Sat, Aug 16, 2014 at 2:35 AM, Steve Strong wrote: >> >> I'm guessing that this: >> >> Morphology = SpecCon#constraint.morphology, >> >> >> Is setting Morphology to 'undefined' - that's what the stack trace looks >> to be saying. >> >> Cheers, >> >> Steve >> >> Sent from my iPhone >> >> On 16 Aug 2014, at 04:54, "Yves S. Garret" >> wrote: >> >> Hello, >> >> This is my code: >> https://gist.github.com/anonymous/5b9b2c55a7684f5386e2 >> >> When I run my code inside of erlang (17.0), this is the error that I get: >> 7> genotype:create_test(). >> {aborted,{undef,[{morphology,undefined,[sensors],[]}, >> {morphology,get_InitSensors,1, >> [{file,"morphology.erl"},{line,8}]}, >> {genotype,construct_Cortex,3, >> [{file,"genotype.erl"},{line,41}]}, >> {genotype,construct_Agent,3, >> [{file,"genotype.erl"},{line,20}]}, >> {genotype,'-create_test/0-fun-0-',0, >> [{file,"genotype.erl"},{line,434}]}, >> >> {mnesia_tm,apply_fun,3,[{file,"mnesia_tm.erl"},{line,833}]}, >> {mnesia_tm,execute_transaction,5, >> [{file,"mnesia_tm.erl"},{line,813}]}, >> >> {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,661}]}]}} >> >> genotype.erl and morphology.erl are in the directory, so why can't >> genotype's function call morphology's functions? Or am I missing something >> else entirely? >> >> Thanks. >> >> _______________________________________________ >> >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Thu Sep 4 04:21:26 2014 From: zxq9@REDACTED (zxq9) Date: Thu, 04 Sep 2014 11:21:26 +0900 Subject: [erlang-questions] Work solicitation from Oncam Message-ID: <5325511.8c8y8a2BO2@burrito> Apologies in advance -- I'm not trying to muck up the list with work offers, but the community is mighty small... I received a solicitation for rush, short-term (a few weeks) Erlang consulting work in California. I'm currently engaged, but if anyone else is interested let me know off-list and I'll pass the note along. It appears some help is needed with straightening out infrastructure bits prior to a public release. -Craig From ecotter@REDACTED Thu Sep 4 04:45:42 2014 From: ecotter@REDACTED (Eric Cotter) Date: Wed, 3 Sep 2014 19:45:42 -0700 Subject: [erlang-questions] Windows 64bit erlang install actually 32bit? Message-ID: <5407d26a.6560440a.0f69.1483@mx.google.com> Hello, I am using RabbitMQ on a Windows 64bit OS. I installed the 64 bit most recent windows distro. I ran the erlang console and verified that it is 64 bit. I installed rabbitmq, which is written in erlang and the installed components are a bunch of beam files. When I run the server and execute the rabbitmqctl eval erlang:system-info(system-architecture). I get win32. I'm facing a problem in my dev environment where erlang is crashing at just 2gb of ram. I have the crash dump files. This Windows VM has 25gb ram and 8 cores. I am running at 30 to 50% CPU utilization when erlang crashes. I have verified all previous erlang versions have been removed and the registry prior to install. Any thoughts other then the obvious to install on Linux and I will be. I have an urgent need to be able to run rabbitmq in a larger memory constraint. Rabbitmq management UI tells me the erlang instance is 64 bit, however it acts like its 32bit and the architecture tells me so. What can I do to run RabbitMQ in 64 bit. Thank you very much! Eric Cotter -------------- next part -------------- An HTML attachment was scrubbed... URL: From mloftis@REDACTED Thu Sep 4 05:53:44 2014 From: mloftis@REDACTED (Michael Loftis) Date: Wed, 3 Sep 2014 20:53:44 -0700 Subject: [erlang-questions] Work solicitation from Oncam In-Reply-To: <5325511.8c8y8a2BO2@burrito> References: <5325511.8c8y8a2BO2@burrito> Message-ID: They spammed almost anyone whose posted on the list (got one too). On Wed, Sep 3, 2014 at 7:21 PM, zxq9 wrote: > Apologies in advance -- I'm not trying to muck up the list with work offers, > but the community is mighty small... > > I received a solicitation for rush, short-term (a few weeks) Erlang consulting > work in California. I'm currently engaged, but if anyone else is interested > let me know off-list and I'll pass the note along. > > It appears some help is needed with straightening out infrastructure bits > prior to a public release. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- "Genius might be described as a supreme capacity for getting its possessors into trouble of all kinds." -- Samuel Butler From s.j.thompson@REDACTED Thu Sep 4 16:58:59 2014 From: s.j.thompson@REDACTED (Simon Thompson) Date: Thu, 4 Sep 2014 15:58:59 +0100 Subject: [erlang-questions] Jobs at the University of Kent Message-ID: <79E7D413-78B2-4FE5-9A00-70C86D72A322@kent.ac.uk> We've been doing functional programming at Kent for more than thirty years, and it would be great to recruit more people in FP, theory, type theory, programming languages, verification, concurrency ... Details of three positions here: https://jobs.kent.ac.uk/fe/tpl_kent01.asp?s=4A515F4E5A565B1A&jobid=38618,1240724854&key=41286990&c=545876250256&pagestamp=sedufjhfxtuvnjtwsc If you're interested in discussing, just send us an email. Simon Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt From egil@REDACTED Thu Sep 4 18:50:37 2014 From: egil@REDACTED (=?windows-1252?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Thu, 4 Sep 2014 18:50:37 +0200 Subject: [erlang-questions] Windows 64bit erlang install actually 32bit? In-Reply-To: <5407d26a.6560440a.0f69.1483@mx.google.com> References: <5407d26a.6560440a.0f69.1483@mx.google.com> Message-ID: <5408985D.1060506@erlang.org> Our naming of Windows is very misleading. In this context "win32" means Windows OS not wordsize. For instance os:type/0 will return {win32,_} on 64 bit and that is because we don't want to make the distinction between win32 and win64. It should probably just have been 'windows' from the first implementation but .. C'est la vie. Check for wordsize to be sure. Eshell V7.0 (abort with ^G) 1> erlang:system_info(system_architecture). "win32" 2> os:type(). {win32,nt} 3> erlang:system_info(wordsize). 8 // Bj?rn-Egil On 2014-09-04 04:45, Eric Cotter wrote: > Hello, > > I am using RabbitMQ on a Windows 64bit OS. > I installed the 64 bit most recent windows distro. > > I ran the erlang console and verified that it is 64 bit. > > I installed rabbitmq, which is written in erlang and the installed > components are a bunch of beam files. When I run the server and > execute the rabbitmqctl eval erlang:system-info(system-architecture). > I get win32. > > I'm facing a problem in my dev environment where erlang is crashing at > just 2gb of ram. I have the crash dump files. This Windows VM has > 25gb ram and 8 cores. I am running at 30 to 50% CPU utilization when > erlang crashes. > > I have verified all previous erlang versions have been removed and the > registry prior to install. > > Any thoughts other then the obvious to install on Linux and I will > be. I have an urgent need to be able to run rabbitmq in a larger > memory constraint. > > Rabbitmq management UI tells me the erlang instance is 64 bit, however > it acts like its 32bit and the architecture tells me so. > > What can I do to run RabbitMQ in 64 bit. > > Thank you very much! > > Eric Cotter > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From devgutt@REDACTED Thu Sep 4 19:27:34 2014 From: devgutt@REDACTED (Gustavo Pires) Date: Thu, 4 Sep 2014 14:27:34 -0300 Subject: [erlang-questions] Unable to call functions that are in a module in the same location In-Reply-To: References: Message-ID: You shouldn't be using the record #actuator instead of #sensor in xor_mimic(actuators)? There is a error in construct_Cortex because of this xor_mimic(sensors) -> [#sensor{name = xor_GetInput, scape = {private, xor_sim}, vl = 2}]; xor_mimic(actuators) -> [#sensor{name = xor_SendOutput, scape = {private, xor_sim}, vl = 1}]. On Wed, Sep 3, 2014 at 9:43 PM, Yves S. Garret wrote: > Hello, > > I completely forgot about this problem. > > Here are some more files that will hopefully get rid of some of the > confusion: > records.hrl: > http://textsnip.com/a148rx > polis.erl: > http://textsnip.com/bk8xao > morphology.erl: > http://textsnip.com/8rlag9 > genotype.erl: > http://textsnip.com/ax1n49 > > Let me know if there's anything else that I can hopefully shed light on. > > > On Sun, Aug 17, 2014 at 9:59 PM, Gustavo Pires wrote: > >> Where's the code of construct_Agent(Specie_Id, Agent_Id, SpecCon)? You >> only sent construct_Cortex. Probably the problem is there. >> >> >> On Sun, Aug 17, 2014 at 1:31 PM, Steve Strong wrote: >> >>> The record will be initialised with the defaults as specified in the >>> hrl. The code and stacktrace in the gist still suggest that >>> SpecCon#constraint.morphology is ?undefined? - so something between the >>> record construction below and the crash within the gist must be modifying >>> the record. >>> >>> -- >>> Steve Strong >>> Sent with Sparrow >>> >>> On Sunday, 17 August 2014 at 15:36, Yves S. Garret wrote: >>> >>> The record in question is "constraint". The thing is, when I call it, >>> it looks like this with none of the values initialized: >>> >>> test() -> >>> Specie_Id = test, >>> Agent_Id = test, >>> CloneAgent_Id = test_clone, >>> SpecCon = #constraint{}, % <---- HERE IS WHERE THE FIRST INSTANCE IS >>> CREATED >>> F = fun() -> >>> construct_Agent(Specie_Id, Agent_Id, SpecCon), >>> clone_Agent(Specie_Id, CloneAgent_Id), >>> print(Agent_Id), >>> print(CloneAgent_Id), >>> delete_Agent(Agent_Id), >>> delete_Agent(CloneAgent_Id) >>> end, >>> >>> mnesia:transaction(F). >>> >>> My question is thus, if I make a record with #constraint{}, does that >>> mean that all of its values are by default set to undefined or the default >>> values that were specified in records.hrl? >>> >>> >>> >>> >>> On Sat, Aug 16, 2014 at 2:35 AM, Steve Strong >>> wrote: >>> >>> I'm guessing that this: >>> >>> Morphology = SpecCon#constraint.morphology, >>> >>> >>> >>> Is setting Morphology to 'undefined' - that's what the stack trace looks >>> to be saying. >>> >>> Cheers, >>> >>> Steve >>> >>> Sent from my iPhone >>> >>> On 16 Aug 2014, at 04:54, "Yves S. Garret" >>> wrote: >>> >>> Hello, >>> >>> This is my code: >>> https://gist.github.com/anonymous/5b9b2c55a7684f5386e2 >>> >>> When I run my code inside of erlang (17.0), this is the error that I get: >>> 7> genotype:create_test(). >>> {aborted,{undef,[{morphology,undefined,[sensors],[]}, >>> {morphology,get_InitSensors,1, >>> [{file,"morphology.erl"},{line,8}]}, >>> {genotype,construct_Cortex,3, >>> [{file,"genotype.erl"},{line,41}]}, >>> {genotype,construct_Agent,3, >>> [{file,"genotype.erl"},{line,20}]}, >>> {genotype,'-create_test/0-fun-0-',0, >>> [{file,"genotype.erl"},{line,434}]}, >>> >>> {mnesia_tm,apply_fun,3,[{file,"mnesia_tm.erl"},{line,833}]}, >>> {mnesia_tm,execute_transaction,5, >>> [{file,"mnesia_tm.erl"},{line,813}]}, >>> >>> {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,661}]}]}} >>> >>> genotype.erl and morphology.erl are in the directory, so why can't >>> genotype's function call morphology's functions? Or am I missing something >>> else entirely? >>> >>> Thanks. >>> >>> _______________________________________________ >>> >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clonejo@REDACTED Thu Sep 4 21:46:05 2014 From: clonejo@REDACTED (clonejo) Date: Thu, 04 Sep 2014 21:46:05 +0200 Subject: [erlang-questions] Unable to call functions that are in a module in the same location In-Reply-To: References: Message-ID: <5408C17D.2040402@clnj.de> On 16.08.2014 04:54, Yves S. Garret wrote: > genotype.erl and morphology.erl are in the directory, so why can't > genotype's function call morphology's functions? Or am I missing > something else entirely? I suppose this is not your problem, but the Erlang VM does not care about .erl files. All it wants is .beam bytecode files. From yoursurrogategod@REDACTED Fri Sep 5 03:39:17 2014 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Thu, 4 Sep 2014 21:39:17 -0400 Subject: [erlang-questions] Unable to call functions that are in a module in the same location In-Reply-To: References: Message-ID: Hi, Thanks for your reply. You're right, I did screw that up. This is after I made the proper fix: 6> genotype:test(). Morphology: xor_mimic {aborted,{undef,[{random,unifrom,[],[]}, {genotype,construct_InitialNeurons,6, [{file,"genotype.erl"},{line,101}]}, {genotype,construct_InitialNeuroLayer,7, [{file,"genotype.erl"},{line,84}]}, {genotype,construct_Cortex,3, [{file,"genotype.erl"},{line,51}]}, {genotype,construct_Agent,3, [{file,"genotype.erl"},{line,20}]}, {genotype,'-test/0-fun-0-',0, [{file,"genotype.erl"},{line,420}]}, {mnesia_tm,apply_fun,3,[{file,"mnesia_tm.erl"},{line,830}]}, {mnesia_tm,execute_transaction,5, [{file,"mnesia_tm.erl"},{line,810}]}]}} I kept digging in here and then this is what I've found. The AgentId (in test/0) is set to test. However, when the attempt is made to retrieve it, obviously, it gives me the above error. I think I need to keep reading and try to better understand what Gene Sher is trying to pull together here. On Thu, Sep 4, 2014 at 1:27 PM, Gustavo Pires wrote: > You shouldn't be using the record #actuator instead of #sensor in > xor_mimic(actuators)? There is a error in construct_Cortex because of this > > xor_mimic(sensors) -> > [#sensor{name = xor_GetInput, scape = {private, xor_sim}, vl = 2}]; > xor_mimic(actuators) -> > [#sensor{name = xor_SendOutput, scape = {private, xor_sim}, vl = 1}]. > > > On Wed, Sep 3, 2014 at 9:43 PM, Yves S. Garret > wrote: > >> Hello, >> >> I completely forgot about this problem. >> >> Here are some more files that will hopefully get rid of some of the >> confusion: >> records.hrl: >> http://textsnip.com/a148rx >> polis.erl: >> http://textsnip.com/bk8xao >> morphology.erl: >> http://textsnip.com/8rlag9 >> genotype.erl: >> http://textsnip.com/ax1n49 >> >> Let me know if there's anything else that I can hopefully shed light on. >> >> >> On Sun, Aug 17, 2014 at 9:59 PM, Gustavo Pires wrote: >> >>> Where's the code of construct_Agent(Specie_Id, Agent_Id, SpecCon)? You >>> only sent construct_Cortex. Probably the problem is there. >>> >>> >>> On Sun, Aug 17, 2014 at 1:31 PM, Steve Strong >>> wrote: >>> >>>> The record will be initialised with the defaults as specified in the >>>> hrl. The code and stacktrace in the gist still suggest that >>>> SpecCon#constraint.morphology is ?undefined? - so something between the >>>> record construction below and the crash within the gist must be modifying >>>> the record. >>>> >>>> -- >>>> Steve Strong >>>> Sent with Sparrow >>>> >>>> On Sunday, 17 August 2014 at 15:36, Yves S. Garret wrote: >>>> >>>> The record in question is "constraint". The thing is, when I call it, >>>> it looks like this with none of the values initialized: >>>> >>>> test() -> >>>> Specie_Id = test, >>>> Agent_Id = test, >>>> CloneAgent_Id = test_clone, >>>> SpecCon = #constraint{}, % <---- HERE IS WHERE THE FIRST INSTANCE IS >>>> CREATED >>>> F = fun() -> >>>> construct_Agent(Specie_Id, Agent_Id, SpecCon), >>>> clone_Agent(Specie_Id, CloneAgent_Id), >>>> print(Agent_Id), >>>> print(CloneAgent_Id), >>>> delete_Agent(Agent_Id), >>>> delete_Agent(CloneAgent_Id) >>>> end, >>>> >>>> mnesia:transaction(F). >>>> >>>> My question is thus, if I make a record with #constraint{}, does that >>>> mean that all of its values are by default set to undefined or the default >>>> values that were specified in records.hrl? >>>> >>>> >>>> >>>> >>>> On Sat, Aug 16, 2014 at 2:35 AM, Steve Strong >>>> wrote: >>>> >>>> I'm guessing that this: >>>> >>>> Morphology = SpecCon#constraint.morphology, >>>> >>>> >>>> >>>> >>>> Is setting Morphology to 'undefined' - that's what the stack trace >>>> looks to be saying. >>>> >>>> Cheers, >>>> >>>> Steve >>>> >>>> Sent from my iPhone >>>> >>>> On 16 Aug 2014, at 04:54, "Yves S. Garret" >>>> wrote: >>>> >>>> Hello, >>>> >>>> This is my code: >>>> https://gist.github.com/anonymous/5b9b2c55a7684f5386e2 >>>> >>>> When I run my code inside of erlang (17.0), this is the error that I >>>> get: >>>> 7> genotype:create_test(). >>>> {aborted,{undef,[{morphology,undefined,[sensors],[]}, >>>> {morphology,get_InitSensors,1, >>>> [{file,"morphology.erl"},{line,8}]}, >>>> {genotype,construct_Cortex,3, >>>> [{file,"genotype.erl"},{line,41}]}, >>>> {genotype,construct_Agent,3, >>>> [{file,"genotype.erl"},{line,20}]}, >>>> {genotype,'-create_test/0-fun-0-',0, >>>> [{file,"genotype.erl"},{line,434}]}, >>>> >>>> {mnesia_tm,apply_fun,3,[{file,"mnesia_tm.erl"},{line,833}]}, >>>> {mnesia_tm,execute_transaction,5, >>>> [{file,"mnesia_tm.erl"},{line,813}]}, >>>> >>>> {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,661}]}]}} >>>> >>>> genotype.erl and morphology.erl are in the directory, so why can't >>>> genotype's function call morphology's functions? Or am I missing something >>>> else entirely? >>>> >>>> Thanks. >>>> >>>> _______________________________________________ >>>> >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Fri Sep 5 10:25:25 2014 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 5 Sep 2014 10:25:25 +0200 Subject: [erlang-questions] Unable to call functions that are in a module in the same location In-Reply-To: References: Message-ID: On Fri, Sep 5, 2014 at 3:39 AM, Yves S. Garret wrote: > Hi, > > Thanks for your reply. You're right, I did screw that up. This is > after I made the proper fix: > > 6> genotype:test(). > Morphology: xor_mimic > {aborted,{undef,[{random,unifrom,[],[]}, unifrom -> uniform > {genotype,construct_InitialNeurons,6, > [{file,"genotype.erl"},{line,101}]}, > {genotype,construct_InitialNeuroLayer,7, > [{file,"genotype.erl"},{line,84}]}, > {genotype,construct_Cortex,3, > [{file,"genotype.erl"},{line,51}]}, > {genotype,construct_Agent,3, > [{file,"genotype.erl"},{line,20}]}, > {genotype,'-test/0-fun-0-',0, > [{file,"genotype.erl"},{line,420}]}, > > {mnesia_tm,apply_fun,3,[{file,"mnesia_tm.erl"},{line,830}]}, > {mnesia_tm,execute_transaction,5, > [{file,"mnesia_tm.erl"},{line,810}]}]}} > > I kept digging in here and then this is what I've found. The AgentId > (in test/0) is set to test. However, when the attempt is made to > retrieve it, obviously, it gives me the above error. > > I think I need to keep reading and try to better understand what > Gene Sher is trying to pull together here. From devgutt@REDACTED Fri Sep 5 14:14:55 2014 From: devgutt@REDACTED (Gustavo Pires) Date: Fri, 5 Sep 2014 09:14:55 -0300 Subject: [erlang-questions] Unable to call functions that are in a module in the same location In-Reply-To: References: Message-ID: It seems a very interesting project. Congrats On Fri, Sep 5, 2014 at 5:25 AM, Tuncer Ayaz wrote: > On Fri, Sep 5, 2014 at 3:39 AM, Yves S. Garret wrote: > > Hi, > > > > Thanks for your reply. You're right, I did screw that up. This is > > after I made the proper fix: > > > > 6> genotype:test(). > > Morphology: xor_mimic > > {aborted,{undef,[{random,unifrom,[],[]}, > > unifrom -> uniform > > > {genotype,construct_InitialNeurons,6, > > [{file,"genotype.erl"},{line,101}]}, > > {genotype,construct_InitialNeuroLayer,7, > > [{file,"genotype.erl"},{line,84}]}, > > {genotype,construct_Cortex,3, > > [{file,"genotype.erl"},{line,51}]}, > > {genotype,construct_Agent,3, > > [{file,"genotype.erl"},{line,20}]}, > > {genotype,'-test/0-fun-0-',0, > > [{file,"genotype.erl"},{line,420}]}, > > > > {mnesia_tm,apply_fun,3,[{file,"mnesia_tm.erl"},{line,830}]}, > > {mnesia_tm,execute_transaction,5, > > [{file,"mnesia_tm.erl"},{line,810}]}]}} > > > > I kept digging in here and then this is what I've found. The AgentId > > (in test/0) is set to test. However, when the attempt is made to > > retrieve it, obviously, it gives me the above error. > > > > I think I need to keep reading and try to better understand what > > Gene Sher is trying to pull together here. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mats.cronqvist@REDACTED Fri Sep 5 13:31:40 2014 From: mats.cronqvist@REDACTED (mats cronqvist) Date: Fri, 5 Sep 2014 13:31:40 +0200 Subject: [erlang-questions] apparent memory leak in OTP 17 Message-ID: I have an application (a web server using cowboy and jiffy) that keeps getting killed by the Linux out-of-memory assassin. This is rather unsurprising given that the emulator seems to allocate a factor of 1000 times more memory than it uses. E.g., in the snapshot below, erlang:memory/0 reports memory usage of ~500 Mb, while /proc/self/stat reports ~600 Gb. Digging into the allocator info reveals that the binary_alloc / mbcs_pool allocator has booked 606086.75 Mb, and uses 357.35 Mb. For the brave, I have attached a file full of system data. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: erlang.dump Type: application/octet-stream Size: 246717 bytes Desc: not available URL: From jesper.louis.andersen@REDACTED Fri Sep 5 17:55:19 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 5 Sep 2014 17:55:19 +0200 Subject: [erlang-questions] apparent memory leak in OTP 17 In-Reply-To: References: Message-ID: One jiffy problem I've seen is that the binaries it produces are subbinaries over the original. So if one process decodes and then stuffs data into ETS, you keep the large binaries around. Since it is you reporting it, I guess you have ruled out the possibility, but it is one thing to look out for. On Fri, Sep 5, 2014 at 1:31 PM, mats cronqvist wrote: > I have an application (a web server using cowboy and jiffy) that keeps > getting killed by the Linux out-of-memory assassin. > This is rather unsurprising given that the emulator seems to allocate a > factor of 1000 times more memory than it uses. > E.g., in the snapshot below, erlang:memory/0 reports memory usage of > ~500 Mb, while /proc/self/stat reports ~600 Gb. > Digging into the allocator info reveals that the binary_alloc / > mbcs_pool allocator has booked 606086.75 Mb, and uses 357.35 Mb. > For the brave, I have attached a file full of system data. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.joseph.davis@REDACTED Fri Sep 5 18:27:41 2014 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Fri, 5 Sep 2014 11:27:41 -0500 Subject: [erlang-questions] apparent memory leak in OTP 17 In-Reply-To: References: Message-ID: On Fri, Sep 5, 2014 at 10:55 AM, Jesper Louis Andersen wrote: > One jiffy problem I've seen is that the binaries it produces are subbinaries > over the original. So if one process decodes and then stuffs data into ETS, > you keep the large binaries around. Since it is you reporting it, I guess > you have ruled out the possibility, but it is one thing to look out for. Is that enough of a problem you'd want to see an option to disable the sub-binary and instead make out of the source binary? From sverker.eriksson@REDACTED Fri Sep 5 18:33:15 2014 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 5 Sep 2014 18:33:15 +0200 Subject: [erlang-questions] apparent memory leak in OTP 17 In-Reply-To: References: Message-ID: <5409E5CB.8070603@erix.ericsson.se> The statistics says there are a *lot* of poorly utilized carriers (mmap'ed regions) waiting (in mbcs_pool) to die or be revived. Do you have any notion about the previous peak memory load of the node? Has it been loaded with hundreds of Gb with binaries? /Sverker, Erlang/OTP On 09/05/2014 01:31 PM, mats cronqvist wrote: > I have an application (a web server using cowboy and jiffy) that keeps > getting killed by the Linux out-of-memory assassin. > This is rather unsurprising given that the emulator seems to allocate a > factor of 1000 times more memory than it uses. > E.g., in the snapshot below, erlang:memory/0 reports memory usage of ~500 > Mb, while /proc/self/stat reports ~600 Gb. > Digging into the allocator info reveals that the binary_alloc / mbcs_pool > allocator has booked 606086.75 Mb, and uses 357.35 Mb. > For the brave, I have attached a file full of system data. > > From jesper.louis.andersen@REDACTED Fri Sep 5 19:44:21 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 5 Sep 2014 19:44:21 +0200 Subject: [erlang-questions] apparent memory leak in OTP 17 In-Reply-To: References: Message-ID: On Fri, Sep 5, 2014 at 6:27 PM, Paul Davis wrote: > Is that enough of a problem you'd want to see an option to disable the > sub-binary and instead make out of the source binary? > Yes and no. If you end up copying the data to another process, the problem goes away. It is a hard call, because the safest solution is to copy. But at times, you want the other option and then you pin a couple binaries with binary:copy/1 in order to achieve speed. I don't know how many have been hit by this in practice. My call is that if it is a lot, the default should probably be the safe bet. And then you have an option for the faster code path. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ecotter@REDACTED Fri Sep 5 18:41:31 2014 From: ecotter@REDACTED (Eric Cotter) Date: Fri, 5 Sep 2014 09:41:31 -0700 Subject: [erlang-questions] Windows OS crash dump at 2GB (64bit OTP) Message-ID: Hello, I am using the Windows 64 bit distribution of the latest Elrang vm. I am using RabbitMQ's Windows installation as well. When I start load testing RabbitMQ I'm sending a modest volume of messages onto RabbitMQ 10,000 messages avg size < 1K. When I get about 8K onboard i'm running just about 2GB of Ram and the Erlang VM crashes. I'm using the C# NuGet packgage via NServiceBus to make all of my calls onto RabbitMQ transport. I am not using Erlang at this time for anything other then RabbitMQ and plan on moving it eventually to CentOS. For now I need to get RabbitMQ and Erlang to be able to stay up when more then 2GB of memory is allocated. I have included the crashdump on the machine in QA that is crashing. I'm using AppDymamics to stress the system. 8 cores 3.3ghz 25 GB ram 18.7 Terabytes drive Windows 64bit server. --------------------------- =erl_crash_dump:0.3 Tue Sep 02 16:11:36 2014 Slogan: eheap_alloc: Cannot allocate 762886488 bytes of memory (of type "heap"). System version: Erlang/OTP 17 [erts-6.0] [64-bit] [smp:4:4] [async-threads:30] Compiled: Mon Apr 7 19:35:35 2014 Taints: Atoms: 23356 =memory total: 1972553664 processes: 1446544768 processes_used: 1446486544 system: 526008896 atom: 703377 atom_used: 685539 binary: 476176112 code: 20066158 ets: 24685176 =hash_table:atom_tab size: 19289 used: 13572 objs: 23356 depth: 8 =index_table:atom_tab size: 23552 limit: 1048576 entries: 23356 =hash_table:module_code size: 797 used: 476 objs: 565 depth: 3 =index_table:module_code size: 1024 limit: 65536 entries: 565 =hash_table:export_list size: 6421 used: 4963 objs: 9312 depth: 7 =index_table:export_list size: 10240 limit: 524288 entries: 9312 =hash_table:export_list size: 6421 used: 4955 objs: 9285 depth: 7 =hash_table:process_reg size: 97 used: 73 objs: 120 depth: 5 =hash_table:fun_table size: 3203 used: 2257 objs: 4143 depth: 6 =hash_table:node_table size: 5 used: 1 objs: 1 depth: 1 =hash_table:dist_table size: 5 used: 1 objs: 1 depth: 1 =allocated_areas sys_misc: 37549 static: 524928 atom_space: 360536 342698 atom_table: 342841 module_table: 205008 export_table: 400212 export_list: 1638912 register_table: 868 fun_table: 25714 module_refs: 27072 loaded_code: 17769240 dist_table: 547 node_table: 179 bits_bufs_size: 0 bif_timer: 99064 link_lh: 0 process_table: 12582912 port_table: 98304 ets_misc: 49192 =allocator:sys_alloc option e: true option m: libc =allocator:temp_alloc[0] versions: 2.1 3.0 option e: true option t: 5 option ramv: false option sbct: 524288 option rsbcmt: 80 option rmbcmt: 100 option mmbcs: 131072 option lmbcs: 10485760 option smbcs: 1048576 option mbcgs: 10 option acul: 0 option mbsd: 3 option as: gf mbcs blocks: 0 0 4 mbcs blocks size: 0 0 9968 mbcs carriers: 1 1 1 mbcs sys_alloc carriers: 1 mbcs carriers size: 131200 131200 131200 mbcs sys_alloc carriers size: 131200 sbcs blocks: 0 0 0 sbcs blocks size: 0 0 0 sbcs carriers: 0 0 0 sbcs sys_alloc carriers: 0 sbcs carriers size: 0 0 0 sbcs sys_alloc carriers size: 0 temp_alloc calls: 8663 temp_free calls: 8663 temp_realloc calls: 1 sys_alloc calls: 1 sys_free calls: 0 sys_realloc calls: 0 =allocator:temp_alloc[1] -------------------------------- The actual crashdump is 25mb so I am not including it entirely. I can see the "eheap_alloc: Cannot allocate 762886488 bytes of memory (of type "heap")." But that is not even a gig. Any guidance would be greatly appreciated. Thank you very much, Eric Cotter -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri Sep 5 23:25:58 2014 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 05 Sep 2014 14:25:58 -0700 Subject: [erlang-questions] Logging methods in Erlang (FUNCTION macro) Message-ID: <540A2A66.6090604@gmail.com> When you care about the efficiency of logging within Erlang source code (i.e., only using Erlang source code for a logging solution) there are two main solutions: 1) Dynamically compile a module based on the logging level that is currently set and reload it. Lower log levels are naturally ignored because their function doesn't do anything within the updated module. This was done in older versions of ejabberd (https://github.com/processone/ejabberd/blob/2.1.x/src/ejabberd_loglevel.erl), but ejabberd has since switched to using lager. CloudI uses this approach (https://github.com/CloudI/CloudI/blob/develop/src/lib/cloudi_core/src/cloudi_core_i_logger.erl). 2) Use a parse transform to rewrite all the source code that wants to do logging so that it can determine whether it needs to log by checking global state, probably in ets. This approach is taken in lager with the parse transform at https://github.com/basho/lager/blob/master/src/lager_transform.erl . I strongly believe you should not be required to dynamically change all your source code to facilitate logging, since that can be a source of problems for stability and transparency. It is good to have the assurance that running module X is actually X's source code, since you can then make clear assertions about the behaviour of execution within module X. Adding random source code (i.e., source code you don't control in a common dependency) to everything via a parse transform weakens your ability to understand the system with debugging, profiling, and basic human understanding, in addition to providing an opportunity for all source code to contain a difficult to discover error previously unknown within the transform. The additional problem with the parse transform is the usage of global data to get the log level. Performance-wise it is better to have the log level setting the source code in a way that the change is a constant (i.e., a version of a module loaded) instead of dynamic data in ets which is volatile. The main data missing from the logging approach #1 that is present in #2 is the function name. Since Erlang is a functional programming language, functions are the most important element, not modules. We already have a MODULE macro that helps to facilitate code reloading (while also being useful for logging), though we currently don't have a FUNCTION macro to provide the function name with no overhead. I understand it is possible to get the function name from the stack trace with an exception like so: current_function() -> catch throw(x), [_, {_, F, _, _} | _] = erlang:get_stacktrace(), F. A method of getting the function name from the stack trace with less overhead is: erlang:element(2, erlang:element(2, erlang:process_info(self(), current_function)))). However, it is much better if we can avoid any overhead when getting the current function name. That means it would be best to have a FUNCTION preprocessor macro available within Erlang source code. Is there any clear reason why this should not be done? Thanks, Michael From essen@REDACTED Sat Sep 6 00:41:00 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 06 Sep 2014 00:41:00 +0200 Subject: [erlang-questions] Logging methods in Erlang (FUNCTION macro) In-Reply-To: <540A2A66.6090604@gmail.com> References: <540A2A66.6090604@gmail.com> Message-ID: <540A3BFC.7050504@ninenines.eu> On 09/05/2014 11:25 PM, Michael Truog wrote: > When you care about the efficiency of logging within Erlang source code > (i.e., only using Erlang source code for a logging solution) there are > two main solutions: > > 1) Dynamically compile a module based on the logging level that is > currently set and reload it. Lower log levels are naturally ignored > because their function doesn't do anything within the updated module. > This was done in older versions of ejabberd > (https://github.com/processone/ejabberd/blob/2.1.x/src/ejabberd_loglevel.erl), > but ejabberd has since switched to using lager. CloudI uses this > approach > (https://github.com/CloudI/CloudI/blob/develop/src/lib/cloudi_core/src/cloudi_core_i_logger.erl). I like this approach. I do not know how you use it but I hope you don't have to type cloudi_core_i_logger:log(...) everytime you want to log something. > 2) Use a parse transform to rewrite all the source code that wants to do > logging so that it can determine whether it needs to log by checking > global state, probably in ets. This approach is taken in lager with the > parse transform at > https://github.com/basho/lager/blob/master/src/lager_transform.erl . I do not like this one, especially since lager will simply fail if modules were not compiled with the parse_transform (the functions aren't defined). So you have to compile everything with it, can't call from the shell (at least not the same way), yada yada. Very impractical. Efficiency is always good to have but that one is quite costy. [...] > However, it is much better if we can avoid any overhead when getting the > current function name. That means it would be best to have a FUNCTION > preprocessor macro available within Erlang source code. Is there any > clear reason why this should not be done? I think we should *definitely* have a ?FUNCTION macro. The main problem I can see with such a macro however is that the preprocessor would need some knowledge of Erlang syntax, which it probably doesn't have today. But I do not think it is impossible considering the advantages a ?FUNCTION would give. -- Lo?c Hoguin http://ninenines.eu From vinoski@REDACTED Sat Sep 6 01:19:03 2014 From: vinoski@REDACTED (Steve Vinoski) Date: Fri, 5 Sep 2014 19:19:03 -0400 Subject: [erlang-questions] Logging methods in Erlang (FUNCTION macro) In-Reply-To: <540A3BFC.7050504@ninenines.eu> References: <540A2A66.6090604@gmail.com> <540A3BFC.7050504@ninenines.eu> Message-ID: On Fri, Sep 5, 2014 at 6:41 PM, Lo?c Hoguin wrote: > On 09/05/2014 11:25 PM, Michael Truog wrote: > >> >> 2) Use a parse transform to rewrite all the source code that wants to do >> logging so that it can determine whether it needs to log by checking >> global state, probably in ets. This approach is taken in lager with the >> parse transform at >> https://github.com/basho/lager/blob/master/src/lager_transform.erl . >> > > I do not like this one, especially since lager will simply fail if modules > were not compiled with the parse_transform (the functions aren't defined). > So you have to compile everything with it, can't call from the shell (at > least not the same way), yada yada. Very impractical. Efficiency is always > good to have but that one is quite costy. > Instead of complaining in text, how about contributing in code? If there's something you don't like about lager, Andrew is easy to find in IRC or on email -- discuss your concerns with him, propose a solution, and submit a PR. Andrew is quite reasonable and pragmatic. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sat Sep 6 01:29:19 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 06 Sep 2014 01:29:19 +0200 Subject: [erlang-questions] Logging methods in Erlang (FUNCTION macro) In-Reply-To: References: <540A2A66.6090604@gmail.com> <540A3BFC.7050504@ninenines.eu> Message-ID: <540A474F.5010908@ninenines.eu> On 09/06/2014 01:19 AM, Steve Vinoski wrote: > > > > On Fri, Sep 5, 2014 at 6:41 PM, Lo?c Hoguin > wrote: > > On 09/05/2014 11:25 PM, Michael Truog wrote: > > > 2) Use a parse transform to rewrite all the source code that > wants to do > logging so that it can determine whether it needs to log by checking > global state, probably in ets. This approach is taken in lager > with the > parse transform at > https://github.com/basho/__lager/blob/master/src/lager___transform.erl > > . > > > I do not like this one, especially since lager will simply fail if > modules were not compiled with the parse_transform (the functions > aren't defined). So you have to compile everything with it, can't > call from the shell (at least not the same way), yada yada. Very > impractical. Efficiency is always good to have but that one is quite > costy. > > > Instead of complaining in text, how about contributing in code? If > there's something you don't like about lager, Andrew is easy to find in > IRC or on email -- discuss your concerns with him, propose a solution, > and submit a PR. Andrew is quite reasonable and pragmatic. That wasn't a complaint. I am not a lager user. There was two different ways of doing things in the original post, and I simply explained which one I prefer and why. In general anything involving parse transforms will receive a negative opinion from me. I am not going to go to every single project I am not using to tell them why I don't think they're doing the right thing, though. -- Lo?c Hoguin http://ninenines.eu From mjtruog@REDACTED Sat Sep 6 02:04:49 2014 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 05 Sep 2014 17:04:49 -0700 Subject: [erlang-questions] Logging methods in Erlang (FUNCTION macro) In-Reply-To: <540A3BFC.7050504@ninenines.eu> References: <540A2A66.6090604@gmail.com> <540A3BFC.7050504@ninenines.eu> Message-ID: <540A4FA1.5000600@gmail.com> On 09/05/2014 03:41 PM, Lo?c Hoguin wrote: > On 09/05/2014 11:25 PM, Michael Truog wrote: >> When you care about the efficiency of logging within Erlang source code >> (i.e., only using Erlang source code for a logging solution) there are >> two main solutions: >> >> 1) Dynamically compile a module based on the logging level that is >> currently set and reload it. Lower log levels are naturally ignored >> because their function doesn't do anything within the updated module. >> This was done in older versions of ejabberd >> (https://github.com/processone/ejabberd/blob/2.1.x/src/ejabberd_loglevel.erl), >> but ejabberd has since switched to using lager. CloudI uses this >> approach >> (https://github.com/CloudI/CloudI/blob/develop/src/lib/cloudi_core/src/cloudi_core_i_logger.erl). > > I like this approach. I do not know how you use it but I hope you don't have to type cloudi_core_i_logger:log(...) everytime you want to log something. There is a header file with macros to provide MODULE and LINE at the line where the logging statement is, in addition to macros that allow you to call erlang:apply/2 or erlang:apply/3 based on the current logging level (https://github.com/CloudI/CloudI/blob/develop/src/lib/cloudi_core/include/cloudi_logger.hrl). So, a macro call like ?LOG_ERROR("This is an error with data: ~p", [Date]), will work as expected. Elixir requires the macros be in a module, but the module is in the same include directory to make it easy to utilize (https://github.com/CloudI/CloudI/blob/develop/src/lib/cloudi_core/include/CloudILogger.ex). > >> 2) Use a parse transform to rewrite all the source code that wants to do >> logging so that it can determine whether it needs to log by checking >> global state, probably in ets. This approach is taken in lager with the >> parse transform at >> https://github.com/basho/lager/blob/master/src/lager_transform.erl . > > I do not like this one, especially since lager will simply fail if modules were not compiled with the parse_transform (the functions aren't defined). So you have to compile everything with it, can't call from the shell (at least not the same way), yada yada. Very impractical. Efficiency is always good to have but that one is quite costy. > > [...] >> However, it is much better if we can avoid any overhead when getting the >> current function name. That means it would be best to have a FUNCTION >> preprocessor macro available within Erlang source code. Is there any >> clear reason why this should not be done? > > I think we should *definitely* have a ?FUNCTION macro. > > The main problem I can see with such a macro however is that the preprocessor would need some knowledge of Erlang syntax, which it probably doesn't have today. But I do not think it is impossible considering the advantages a ?FUNCTION would give. > From mjtruog@REDACTED Sat Sep 6 02:07:17 2014 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 05 Sep 2014 17:07:17 -0700 Subject: [erlang-questions] Logging methods in Erlang (FUNCTION macro) In-Reply-To: <540A474F.5010908@ninenines.eu> References: <540A2A66.6090604@gmail.com> <540A3BFC.7050504@ninenines.eu> <540A474F.5010908@ninenines.eu> Message-ID: <540A5035.9080404@gmail.com> On 09/05/2014 04:29 PM, Lo?c Hoguin wrote: > On 09/06/2014 01:19 AM, Steve Vinoski wrote: >> >> >> >> On Fri, Sep 5, 2014 at 6:41 PM, Lo?c Hoguin > > wrote: >> >> On 09/05/2014 11:25 PM, Michael Truog wrote: >> >> >> 2) Use a parse transform to rewrite all the source code that >> wants to do >> logging so that it can determine whether it needs to log by checking >> global state, probably in ets. This approach is taken in lager >> with the >> parse transform at >> https://github.com/basho/__lager/blob/master/src/lager___transform.erl >> >> . >> >> >> I do not like this one, especially since lager will simply fail if >> modules were not compiled with the parse_transform (the functions >> aren't defined). So you have to compile everything with it, can't >> call from the shell (at least not the same way), yada yada. Very >> impractical. Efficiency is always good to have but that one is quite >> costy. >> >> >> Instead of complaining in text, how about contributing in code? If >> there's something you don't like about lager, Andrew is easy to find in >> IRC or on email -- discuss your concerns with him, propose a solution, >> and submit a PR. Andrew is quite reasonable and pragmatic. > > That wasn't a complaint. I am not a lager user. There was two different ways of doing things in the original post, and I simply explained which one I prefer and why. > > In general anything involving parse transforms will receive a negative opinion from me. I am not going to go to every single project I am not using to tell them why I don't think they're doing the right thing, though. > I agree. If you believe the first approach is better, to improve lager would be to rewrite it. From stephane@REDACTED Sat Sep 6 12:36:00 2014 From: stephane@REDACTED (=?utf-8?q?St=C3=A9phane?= Wirtel) Date: Sat, 06 Sep 2014 12:36:00 +0200 Subject: [erlang-questions] escript and dependencies Message-ID: <0BB6AC76-C1C6-4648-9F25-1D1A2DA3C524@wirtel.be> Hi all, How to use some external dependencies with an escript? Here is a script and I don't see my path to the dependencies with code:get_path(). https://www.friendpaste.com/2PBNBucysjLeXq51uhGh3y Where is my error? Thank you, Regards, Stephane -- St?phane Wirtel - http://wirtel.be - @matrixise From tuncer.ayaz@REDACTED Sat Sep 6 13:07:31 2014 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 6 Sep 2014 13:07:31 +0200 Subject: [erlang-questions] [ANN] rebar_escript_plugin In-Reply-To: <12F2115FD1CCEE4294943B2608A18FA30122360F2D@MAIL01.win.lbaum.eu> References: <12F2115FD1CCEE4294943B2608A18FA30122360EBF@MAIL01.win.lbaum.eu> <12F2115FD1CCEE4294943B2608A18FA30122360F2D@MAIL01.win.lbaum.eu> Message-ID: On Wed, Sep 3, 2014 at 3:49 PM, Tobias Schlager wrote: > Hi Tuncer, > > thank you very much for your feedback! Let me make clear, that I > didn't want to bash rebar's escriptizer in any way. rebar is a great > project and I would be glad to contribute some of the plugins > functionality if you would like. The only reason I went the plugin > way was because I wanted something that everybody can use > immediately without patching their (committed) rebar. Plugins are a good choice for trying out ideas, and in this case most of it is upstream-worthy. > Your comments and critics are always welcome! My thoughts about your > points: > > (2) Why not? As of today, there's no way to specify different types > of dependencies (test, compile, etc.) which is no problem for me. I The fastest way to teach rebar selectable config sets is to join the discussion in the linked tickets (and write patch(es)) :). > think that including none of the dependecies is not better or worse > as including all of them. I wanted to make sure that everything the > user may need is in place without configuration. Therefore, the > rebar_escript_plugin even searches the configured lib_dirs for > non-system applications to include. In my opinion, if you want to > avoid, e.g. proper, from being packaged along, you should generate > the escript from a release specification. %% list of apps {escript_incl_apps, [app1, app2]}. %% include release spec apps (might require asking reltool/relx) {escript_incl_apps, release_apps}. %% include all apps configured via rebar.config {escript_incl_apps, project_apps}. > (3) Sorry, this is not yet documented in the README. In order to > unpack your 'priv' content the plugin needs to have its own main/1 > function called first. However, the default runner (starting all > needed applications and block) is only used if you don't specify > your own main function. If the module with the name of the escript > exports a 'main/1' function the plugin will call it (from the main > process) after priv extraction with the original arguments (as rebar > would). When writing the runner I had services like the serve_it > example application in mind. I assumed we all write proper OTP > applications ;-) therefore the simple default runner. We can and should promote one good way of doing things, but we cannot assume all projects are like that. That said, we can probably make this work reliably. From a.brandon.clark@REDACTED Sat Sep 6 14:35:31 2014 From: a.brandon.clark@REDACTED (Brandon Clark) Date: Sat, 6 Sep 2014 05:35:31 -0700 Subject: [erlang-questions] escript and dependencies In-Reply-To: <0BB6AC76-C1C6-4648-9F25-1D1A2DA3C524@wirtel.be> References: <0BB6AC76-C1C6-4648-9F25-1D1A2DA3C524@wirtel.be> Message-ID: File globbing, i.e. the '*', is a feature of your login shell, not escript. If I run this: $ erl -pa _rel/lib/*/ebin ...the shell expands the '*' before launching erl. The command that is actually run is this: $ erl -pa _rel/lib/cowboy-0.10.0/ebin _rel/lib/cowlib-0.6.2/ebin _rel/lib/crypto-3.4/ebin _rel/lib/hello_cowboy-1.0/ebin _rel/lib/kernel-3.0.1/ebin _rel/lib/ranch-0.10.0/ebin _rel/lib/stdlib-2.1/ebin There may be an elegant way to perform the expansion inside your script but I don't know what it is. I usually rely on the quick-and-dirty solution of running: $ echo -pa _rel/lib/*/ebin Then I copy and paste the output into the script: #!/usr/bin/env escript %% -*- erlang -*- %%! -pa _rel/lib/cowboy-0.10.0/ebin _rel/lib/cowlib-0.6.2/ebin _rel/lib/crypto-3.4/ebin _rel/lib/hello_cowboy-1.0/ebin _rel/lib/kernel-3.0.1/ebin _rel/lib/ranch-0.10.0/ebin _rel/lib/stdlib-2.1/ebin -module(democtl). . . . ~BC On Sat, Sep 6, 2014 at 3:36 AM, St?phane Wirtel wrote: > Hi all, > > How to use some external dependencies with an escript? > Here is a script and I don't see my path to the dependencies with > code:get_path(). > > https://www.friendpaste.com/2PBNBucysjLeXq51uhGh3y > > Where is my error? > > Thank you, > > Regards, > > Stephane > -- > St?phane Wirtel - http://wirtel.be - @matrixise > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Sat Sep 6 14:39:30 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 6 Sep 2014 08:39:30 -0400 Subject: [erlang-questions] escript and dependencies In-Reply-To: References: <0BB6AC76-C1C6-4648-9F25-1D1A2DA3C524@wirtel.be> Message-ID: <20140906123929.GA784@ferdmbp.local> On 09/06, Brandon Clark wrote: > There may be an elegant way to perform the expansion inside your script but > I don't know what it is. I usually rely on the quick-and-dirty solution of > running: > -env ERL_LIBS _rel/lib/ This can also just be a variable in your environment. From tuncer.ayaz@REDACTED Sat Sep 6 15:39:43 2014 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 6 Sep 2014 15:39:43 +0200 Subject: [erlang-questions] ERL_LIBS equivalent code:fun? Message-ID: Is there really no official way to extend the lib dir path at runtime like we can do with the env var ERL_LIBS before starting a node? We have code:addpathsa/1, but it's not for extending the lib dir path. From yoursurrogategod@REDACTED Sat Sep 6 17:19:47 2014 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Sat, 6 Sep 2014 11:19:47 -0400 Subject: [erlang-questions] Unable to call functions that are in a module in the same location In-Reply-To: References: Message-ID: Yes! That was another mistake. Now, when I run it, this is what I see (which doesn't surprise me): > genotype:test(). Morphology: xor_mimic {aborted,{no_exists,neuron}} On Fri, Sep 5, 2014 at 4:25 AM, Tuncer Ayaz wrote: > On Fri, Sep 5, 2014 at 3:39 AM, Yves S. Garret wrote: > > Hi, > > > > Thanks for your reply. You're right, I did screw that up. This is > > after I made the proper fix: > > > > 6> genotype:test(). > > Morphology: xor_mimic > > {aborted,{undef,[{random,unifrom,[],[]}, > > unifrom -> uniform > > > {genotype,construct_InitialNeurons,6, > > [{file,"genotype.erl"},{line,101}]}, > > {genotype,construct_InitialNeuroLayer,7, > > [{file,"genotype.erl"},{line,84}]}, > > {genotype,construct_Cortex,3, > > [{file,"genotype.erl"},{line,51}]}, > > {genotype,construct_Agent,3, > > [{file,"genotype.erl"},{line,20}]}, > > {genotype,'-test/0-fun-0-',0, > > [{file,"genotype.erl"},{line,420}]}, > > > > {mnesia_tm,apply_fun,3,[{file,"mnesia_tm.erl"},{line,830}]}, > > {mnesia_tm,execute_transaction,5, > > [{file,"mnesia_tm.erl"},{line,810}]}]}} > > > > I kept digging in here and then this is what I've found. The AgentId > > (in test/0) is set to test. However, when the attempt is made to > > retrieve it, obviously, it gives me the above error. > > > > I think I need to keep reading and try to better understand what > > Gene Sher is trying to pull together here. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From genesislive@REDACTED Sun Sep 7 04:23:18 2014 From: genesislive@REDACTED (Genesislive) Date: Sun, 7 Sep 2014 10:23:18 +0800 (CST) Subject: [erlang-questions] escript and dependencies In-Reply-To: <0BB6AC76-C1C6-4648-9F25-1D1A2DA3C524@wirtel.be> References: <0BB6AC76-C1C6-4648-9F25-1D1A2DA3C524@wirtel.be> Message-ID: <224a6066.168f.1484dec3288.Coremail.genesislive@126.com> I have met the same problem before, I think you can add the codes like below into main/1:

code:add_path("./ebin"),
code:add_paths(filelib:wildcard("./deps/*/ebin", "./"))

--
ShionRyuu - https://github.com/genesislive ? 2014-09-06 06:36:00?"St?phane Wirtel" ??? >Hi all, > >How to use some external dependencies with an escript? >Here is a script and I don't see my path to the dependencies with >code:get_path(). > >https://www.friendpaste.com/2PBNBucysjLeXq51uhGh3y > >Where is my error? > >Thank you, > >Regards, > >Stephane >-- >St?phane Wirtel - http://wirtel.be - @matrixise >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions From justin.hanekom@REDACTED Sun Sep 7 18:29:20 2014 From: justin.hanekom@REDACTED (Justin HANEKOM) Date: Sun, 7 Sep 2014 09:29:20 -0700 Subject: [erlang-questions] Not looping after processing read_file request Message-ID: <1410107360.80196.YahooMailNeo@web121506.mail.ne1.yahoo.com> Hi, BEGINNER QUESTION ALERT I'm trying to create a file server that loops, responding to 2 requests: list_dir; and read_file. For some reason the server loops sometimes, but for some unknown reason after successfully responding to a read_file request does not receive/respond to any further requests. I've studied this code extensively looking for my error, but to no avail. Could someone please put me out of my misery? Thanks, Justin -module(afile_server). -export([start/1]). start(Dir) -> spawn_loop_in_dir(Dir). spawn_loop_in_dir(Dir) -> Fn = fun() -> loop(Dir) end, spawn(Fn). loop(Dir) -> receive {Client, list_dir} -> list_dir(Client, Dir); {Client, {read_file, File}} -> read_file(Client, Dir, File) end, loop(Dir). list_dir(Client, Dir) -> Client ! prefix_with_self(list_dir_reply(Dir)). list_dir_reply(Dir) -> DirList = file:list_dir(Dir), {list_dir, DirList}. read_file(Client, Dir, File) -> Client ! prefix_with_self(read_file_reply(Dir, File)). read_file_reply(Dir, File) -> Fullname = filename:join(Dir, File), Contents = file:read_file(Fullname), {read_file, Contents}. prefix_with_self(Msg) -> {self(), Msg}. Justin http://180movie.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sun Sep 7 18:52:53 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 7 Sep 2014 18:52:53 +0200 Subject: [erlang-questions] Not looping after processing read_file request In-Reply-To: <1410107360.80196.YahooMailNeo@web121506.mail.ne1.yahoo.com> References: <1410107360.80196.YahooMailNeo@web121506.mail.ne1.yahoo.com> Message-ID: On Sun, Sep 7, 2014 at 6:29 PM, Justin HANEKOM wrote: > I'm trying to create a file server that loops, responding to 2 requests: > list_dir; and read_file. For some reason the server loops sometimes, but > for some unknown reason after successfully responding to a read_file > request does not receive/respond to any further requests. > Hi! I just tried your code here and it seems to work like it should, in the happy-path case. Where your strategy seems to go a bit off is that you have no way of knowing if your process is still alive, and if it dies, why it died. This is where you want to use a supervisor to keep your process alive, and handle errors. Alternatively, you want to have a monitoring process using erlang:monitor(Process, Pid) in order to figure out if your server exits. For quick debugging, you can easily check for its status: Eshell V6.1.2 (abort with ^G) 1> afile_server:start("."). <0.44.0> 2> exit(v(1), foo). true 3> 3> is_process_alive(v(1)). false but note that for a serious server, you need to have someone watching over your process and report if it goes wrong along the way. There is really no other way. You may be lured into having the process itself catch errors, but this leads to convoluted code quickly: start(Dir) -> spawn(fun() -> try loop(Dir) catch Class:Error -> error_logger:error_report([{class, Class}, {reason, Reason}]) end end). (not tested, but you get the gist of it, surely). Yet another way, is to trace any event on the server process: Eshell V6.1.2 (abort with ^G) 1> Pid = afile_server:start(). ** exception error: undefined function afile_server:start/0 2> Pid = afile_server:start("."). <0.46.0> 3> dbg:tracer(). {ok,<0.48.0>} 4> dbg:p(Pid, all). {ok,[{matched,nonode@REDACTED,1}]} 5> Pid ! {self(), list_dir}. (<0.46.0>) << {<0.44.0>,list_dir} (Timestamp: {1410,108607,886896}) (<0.46.0>) in {afile_server,loop,1} (Timestamp: {1410,108607,886905}) (<0.46.0>) <0.19.0> ! {'$gen_call',{<0.46.0>,#Ref<0.0.0.76>},{list_dir,"."}} (Timestamp: {1410, 108607, 886912}) (<0.46.0>) out {gen,do_call,4} (Timestamp: {1410,108607,886915}) (<0.46.0>) << {#Ref<0.0.0.76>, {ok,["transit-erlang","afile_server.erl","afile_server.beam", "zoo.txt"]}} (Timestamp: {1410,108607,887026}) (<0.46.0>) in {gen,do_call,4} (Timestamp: {1410,108607,887075}) (<0.46.0>) <0.44.0> ! {<0.46.0>, {list_dir,{ok,["transit-erlang","afile_server.erl", "afile_server.beam","zoo.txt"]}}} (Timestamp: {1410, 108607, 887078}) {<0.44.0>,list_dir} (<0.46.0>) out {afile_server,loop,1} (Timestamp: {1410,108607,887081}) [...] 7> exit(Pid, foo). true (<0.46.0>) exit foo (Timestamp: {1410,108643,568435}) The idiomatic Erlang way is to have someone else clean up after a failed process, but at least you have a couple of methods to debug your current situation. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jj@REDACTED Sun Sep 7 18:57:44 2014 From: jj@REDACTED (Giovanni Giorgi) Date: Sun, 7 Sep 2014 18:57:44 +0200 Subject: [erlang-questions] Best Berkley db driver and qlc tutorial Message-ID: Hi, I am searching a berkley db driver interface. Googling I found Edtk (dated 2007) http://www.snookles.com/erlang/edtk/ and Bets (dated 2012) https://github.com/krestenkrab/bets But... which do you suggest? Where can I find a short tutorial on them and on qlc module usage? -- [ [ [ Giovanni ``Daitan`` Giorgi ] ] ] jj@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From mats.cronqvist@REDACTED Sun Sep 7 19:51:56 2014 From: mats.cronqvist@REDACTED (mats cronqvist) Date: Sun, 7 Sep 2014 10:51:56 -0700 (PDT) Subject: [erlang-questions] apparent memory leak in OTP 17 In-Reply-To: <5409E5CB.8070603@erix.ericsson.se> References: <5409E5CB.8070603@erix.ericsson.se> Message-ID: On Fri, Sep 5, 2014 at 6:33 PM, Sverker Eriksson wrote: > The statistics says there are a *lot* of poorly utilized carriers (mmap'ed regions) waiting (in mbcs_pool) to die or be revived. yes, that seems to be tha pattern. > Do you have any notion about the previous peak memory load of the node? Has it been loaded with hundreds of Gb with binaries? i have the full history of the node. the memory usage increases monotonically; there have not been any previou peaks. From sdl.web@REDACTED Mon Sep 8 00:55:35 2014 From: sdl.web@REDACTED (Leo Liu) Date: Mon, 08 Sep 2014 06:55:35 +0800 Subject: [erlang-questions] erl_tidy:file gives surprising output for -spec or -type Message-ID: running erl_tidy:file/1 on the attached file gives me: --8<---------------cut here---------------start------------->8--- [snipped 7 lines] -type({filename_all, {type, 7, union, [{type, 7, string, []}, {type, 7, binary, []}]}, []}). -spec({{pid2name, 1}, [{type, 9, bounded_fun, [{type, 9, 'fun', [{type, 9, product, [{var, 9, 'Pid'}]}, {type, 9, union, [{type, 9, tuple, [{atom, 9, ok}, {var, 9, 'Filename'}]}, {atom, 9, undefined}]}]}, [{type, 10, constraint, [{atom, 10, is_subtype}, [{var, 10, 'Filename'}, {user_type, 10, filename_all, []}]]}, {type, 11, constraint, [{atom, 11, is_subtype}, [{var, 11, 'Pid'}, {type, 11, pid, []}]]}]]}]}). [snipped 9 lines] --8<---------------cut here---------------end--------------->8--- Is this a bug or feature? Thanks, Leo -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: bug.erl URL: From sean@REDACTED Mon Sep 8 01:11:31 2014 From: sean@REDACTED (Sean Cribbs) Date: Sun, 7 Sep 2014 18:11:31 -0500 Subject: [erlang-questions] erl_tidy:file gives surprising output for -spec or -type In-Reply-To: References: Message-ID: erl_syntax doesn't understand -type or -spec either. I'm tempted to write a patch since I'll be needing to generate -spec entries soon. On Sun, Sep 7, 2014 at 5:55 PM, Leo Liu wrote: > running erl_tidy:file/1 on the attached file gives me: > > --8<---------------cut here---------------start------------->8--- > [snipped 7 lines] > > -type({filename_all, > {type, 7, union, > [{type, 7, string, []}, {type, 7, binary, []}]}, > []}). > > -spec({{pid2name, 1}, > [{type, 9, bounded_fun, > [{type, 9, 'fun', > [{type, 9, product, [{var, 9, 'Pid'}]}, > {type, 9, union, > [{type, 9, tuple, > [{atom, 9, ok}, {var, 9, 'Filename'}]}, > {atom, 9, undefined}]}]}, > [{type, 10, constraint, > [{atom, 10, is_subtype}, > [{var, 10, 'Filename'}, > {user_type, 10, filename_all, []}]]}, > {type, 11, constraint, > [{atom, 11, is_subtype}, > [{var, 11, 'Pid'}, {type, 11, pid, []}]]}]]}]}). > > [snipped 9 lines] > --8<---------------cut here---------------end--------------->8--- > > Is this a bug or feature? Thanks, Leo > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Sean Cribbs Software Engineer Basho Technologies, Inc. http://basho.com/ From sdl.web@REDACTED Mon Sep 8 01:39:55 2014 From: sdl.web@REDACTED (Leo Liu) Date: Mon, 08 Sep 2014 07:39:55 +0800 Subject: [erlang-questions] erl_tidy:file gives surprising output for -spec or -type References: Message-ID: On 2014-09-07 18:11 -0500, Sean Cribbs wrote: > erl_syntax doesn't understand -type or -spec either. I'm tempted to > write a patch since I'll be needing to generate -spec entries soon. Thanks for the info and I'm looking forward to your patch ;) Leo From ciprian.craciun@REDACTED Mon Sep 8 08:37:14 2014 From: ciprian.craciun@REDACTED (Ciprian Dorin Craciun) Date: Mon, 8 Sep 2014 09:37:14 +0300 Subject: [erlang-questions] Best Berkley db driver and qlc tutorial In-Reply-To: References: Message-ID: On Sun, Sep 7, 2014 at 7:57 PM, Giovanni Giorgi wrote: > Hi, > I am searching a berkley db driver interface. > Googling I found > Edtk (dated 2007) > http://www.snookles.com/erlang/edtk/ > > and Bets (dated 2012) > https://github.com/krestenkrab/bets > > But... which do you suggest? > Where can I find a short tutorial on them and on qlc module usage? [The reply is kind of off-topic, however it might be relevant especially for "production".] Until some time ago I was also very fond of BerkeleyDB (although I didn't find any usable Erlang interfaces, I did used it in other languages), however please note that since a few versions (maybe a year now?) it has switched from GPL to AGPL: http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/downloads/oslicense-093458.html Ciprian. From drormein@REDACTED Mon Sep 8 11:08:55 2014 From: drormein@REDACTED (Dror Mein) Date: Mon, 8 Sep 2014 02:08:55 -0700 Subject: [erlang-questions] cnode interface memory allocation Message-ID: <1410167335.29492.YahooMailNeo@web126701.mail.ne1.yahoo.com> Hi all, I'm writing a c application that communicates with an erlang node. In order to send messages to the node I need to create erlang terms (ETERM structure). it appears that all functions in the C library "erl_eterm" allocate memory when creating erlang terms. For example, erl_mk_binary copies the binary provided to it, instead of only creating the pointer. I want to prevent this memory allocation to optimize speed. Has anyone done this before? Is there a better way to create eterms? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Mon Sep 8 11:34:39 2014 From: g@REDACTED (Guilherme Andrade) Date: Mon, 08 Sep 2014 10:34:39 +0100 Subject: [erlang-questions] cnode interface memory allocation In-Reply-To: <1410167335.29492.YahooMailNeo@web126701.mail.ne1.yahoo.com> References: <1410167335.29492.YahooMailNeo@web126701.mail.ne1.yahoo.com> Message-ID: <540D782F.9020402@gandrade.net> Hi Drod, Assuming your application is a C node, and that the library codebase is the same as for NIFs and whatnot, I'd expect that allocating structures / copying data is the appropriate behaviour when creating Erlang terms -- it's the simple way to transfer that data back to Erlang avoiding ownership issues and mutable state (or at least making it much harder to unintentionally do it.) I've only dealt with NIFs though, and am not an expert on them, so probably someone else can probably clarify this better. Cheers, On 08/09/14 10:08, Dror Mein wrote: > Hi all, > I'm writing a c application that communicates with an erlang node. In > order to send messages to the node I need to create erlang terms > (ETERM structure). > it appears that all functions in the C library "erl_eterm" allocate > memory when creating erlang terms. > For example, erl_mk_binary copies the binary provided to it, instead > of only creating the pointer. > I want to prevent this memory allocation to optimize speed. > Has anyone done this before? > Is there a better way to create eterms? > > Thanks > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Guilherme https://www.gandrade.net/ PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From n.oxyde@REDACTED Mon Sep 8 11:36:33 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Mon, 8 Sep 2014 11:36:33 +0200 Subject: [erlang-questions] erl_tidy:file gives surprising output for -spec or -type In-Reply-To: References: Message-ID: <6255BC5D-6967-443B-B5C4-E4DA0E5D6CC6@gmail.com> In my opinion, such a patch should also make syntax_tools aware of specs and types, so it should wrap these nodes in its own tree terms. -- Anthony Ramine Le 8 sept. 2014 ? 01:11, Sean Cribbs a ?crit : > erl_syntax doesn't understand -type or -spec either. I'm tempted to > write a patch since I'll be needing to generate -spec entries soon. From n.oxyde@REDACTED Mon Sep 8 11:38:38 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Mon, 8 Sep 2014 11:38:38 +0200 Subject: [erlang-questions] erl_tidy:file gives surprising output for -spec or -type In-Reply-To: <6255BC5D-6967-443B-B5C4-E4DA0E5D6CC6@gmail.com> References: <6255BC5D-6967-443B-B5C4-E4DA0E5D6CC6@gmail.com> Message-ID: <4076A857-9F82-4673-AB5D-F3551677CCA7@gmail.com> Also, this might be useful: https://github.com/erlang/otp/pull/230 -- Anthony Ramine Le 8 sept. 2014 ? 11:36, Anthony Ramine a ?crit : > In my opinion, such a patch should also make syntax_tools aware of specs and types, so it should wrap these nodes in its own tree terms. From g@REDACTED Mon Sep 8 11:55:48 2014 From: g@REDACTED (Guilherme Andrade) Date: Mon, 08 Sep 2014 10:55:48 +0100 Subject: [erlang-questions] cnode interface memory allocation In-Reply-To: <540D782F.9020402@gandrade.net> References: <1410167335.29492.YahooMailNeo@web126701.mail.ne1.yahoo.com> <540D782F.9020402@gandrade.net> Message-ID: <540D7D24.5080703@gandrade.net> (Sorry, missed the subject -- you are indeed talking about a C node. Please ignore some of my previous phrasing.) On 08/09/14 10:34, Guilherme Andrade wrote: > Hi Drod, > > Assuming your application is a C node, and that the library codebase > is the same as for NIFs and whatnot, I'd expect that allocating > structures / copying data is the appropriate behaviour when creating > Erlang terms -- it's the simple way to transfer that data back to > Erlang avoiding ownership issues and mutable state (or at least making > it much harder to unintentionally do it.) > > I've only dealt with NIFs though, and am not an expert on them, so > probably someone else can probably clarify this better. > > > Cheers, > > On 08/09/14 10:08, Dror Mein wrote: >> Hi all, >> I'm writing a c application that communicates with an erlang node. In >> order to send messages to the node I need to create erlang terms >> (ETERM structure). >> it appears that all functions in the C library "erl_eterm" allocate >> memory when creating erlang terms. >> For example, erl_mk_binary copies the binary provided to it, instead >> of only creating the pointer. >> I want to prevent this memory allocation to optimize speed. >> Has anyone done this before? >> Is there a better way to create eterms? >> >> Thanks >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > Guilherme > > https://www.gandrade.net/ > PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Guilherme https://www.gandrade.net/ PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From drormein@REDACTED Mon Sep 8 12:35:05 2014 From: drormein@REDACTED (Dror Mein) Date: Mon, 8 Sep 2014 03:35:05 -0700 Subject: [erlang-questions] cnode interface memory allocation In-Reply-To: <540D7D24.5080703@gandrade.net> References: <1410167335.29492.YahooMailNeo@web126701.mail.ne1.yahoo.com> <540D782F.9020402@gandrade.net> <540D7D24.5080703@gandrade.net> Message-ID: <1410172505.17155.YahooMailNeo@web126706.mail.ne1.yahoo.com> I'll try to clarify the problem. I have a char * in my code, which I want to erl_send to the erlang node. The erl_send function accepts as input an ETERM* pointer. I need to turn char* to ETERM*. The current way is by erl_mk_binary, which allocates memory. Is there a way to pass the char * to erl_send without the unnecessary memory allocation. I thought of pre-creating an ETERM struct, and only changing the binary pointer to the char* (before running erl_send). This way I can send multiple char* buffers without the need to create multiple ETERM pointers. The point of all this is to reduce run-time for the erl_send process On Monday, September 8, 2014 12:55 PM, Guilherme Andrade wrote: (Sorry, missed the subject -- you are indeed talking about a C node. Please ignore some of my previous phrasing.) On 08/09/14 10:34, Guilherme Andrade wrote: Hi Drod, Assuming your application is a C node, and that the library codebase is the same as for NIFs and whatnot, I'd expect that allocating structures / copying data is the appropriate behaviour when creating Erlang terms -- it's the simple way to transfer that data back to Erlang avoiding ownership issues and mutable state (or at least making it much harder to unintentionally do it.) I've only dealt with NIFs though, and am not an expert on them, so probably someone else can probably clarify this better. Cheers, On 08/09/14 10:08, Dror Mein wrote: Hi all, >I'm writing a c application that communicates with an erlang node. In order to send messages to the node I need to create erlang terms (ETERM structure). >it appears that all functions in the C library "erl_eterm" allocate memory when creating erlang terms. >For example, erl_mk_binary copies the binary provided to it, instead of only creating the pointer. >I want to prevent this memory allocation to optimize speed. >Has anyone done this before? >Is there a better way to create eterms? > > >Thanks > > >_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -- Guilherme https://www.gandrade.net/ PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -- Guilherme https://www.gandrade.net/ PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 -------------- next part -------------- An HTML attachment was scrubbed... URL: From drormein@REDACTED Mon Sep 8 12:59:04 2014 From: drormein@REDACTED (Dror Mein) Date: Mon, 8 Sep 2014 03:59:04 -0700 Subject: [erlang-questions] Inter-node communication bottleneck Message-ID: <1410173944.42603.YahooMailNeo@web126704.mail.ne1.yahoo.com> I encountered the same problem. any updates or new successes? the binary_to_term over tcp sounds promising. -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Mon Sep 8 13:16:49 2014 From: g@REDACTED (Guilherme Andrade) Date: Mon, 08 Sep 2014 12:16:49 +0100 Subject: [erlang-questions] cnode interface memory allocation In-Reply-To: <1410172505.17155.YahooMailNeo@web126706.mail.ne1.yahoo.com> References: <1410167335.29492.YahooMailNeo@web126701.mail.ne1.yahoo.com> <540D782F.9020402@gandrade.net> <540D7D24.5080703@gandrade.net> <1410172505.17155.YahooMailNeo@web126706.mail.ne1.yahoo.com> Message-ID: <540D9021.2080708@gandrade.net> You can always use a secondary channel to transmit the heavier data; serialising libs like Protocol Buffers et. al. come to mind. As a very unsafe alternative, you can also use enif_make_new_binary() first and only then write your data over the allocated area. But, in my limited experience, I wouldn't recomend it. On 08/09/14 11:35, Dror Mein wrote: > I'll try to clarify the problem. > I have a char * in my code, which I want to erl_send to the erlang > node. The erl_send function accepts as input an ETERM* pointer. I need > to turn char* to ETERM*. The current way is by erl_mk_binary, which > allocates memory. > Is there a way to pass the char * to erl_send without the unnecessary > memory allocation. > > I thought of pre-creating an ETERM struct, and only changing the > binary pointer to the char* (before running erl_send). This way I can > send multiple char* buffers without the need to create multiple ETERM > pointers. > > The point of all this is to reduce run-time for the erl_send process > > > On Monday, September 8, 2014 12:55 PM, Guilherme Andrade > wrote: > > > (Sorry, missed the subject -- you are indeed talking about a C node. > Please ignore some of my previous phrasing.) > > On 08/09/14 10:34, Guilherme Andrade wrote: > Hi Drod, > > Assuming your application is a C node, and that the library codebase > is the same as for NIFs and whatnot, I'd expect that allocating > structures / copying data is the appropriate behaviour when creating > Erlang terms -- it's the simple way to transfer that data back to > Erlang avoiding ownership issues and mutable state (or at least making > it much harder to unintentionally do it.) > > I've only dealt with NIFs though, and am not an expert on them, so > probably someone else can probably clarify this better. > > > Cheers, > > On 08/09/14 10:08, Dror Mein wrote: >> Hi all, >> I'm writing a c application that communicates with an erlang node. In >> order to send messages to the node I need to create erlang terms >> (ETERM structure). >> it appears that all functions in the C library "erl_eterm" allocate >> memory when creating erlang terms. >> For example, erl_mk_binary copies the binary provided to it, instead >> of only creating the pointer. >> I want to prevent this memory allocation to optimize speed. >> Has anyone done this before? >> Is there a better way to create eterms? >> >> Thanks >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > Guilherme > > https://www.gandrade.net/ > PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- > Guilherme > > https://www.gandrade.net/ > PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 > > -- Guilherme https://www.gandrade.net/ PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From yjh0502@REDACTED Mon Sep 8 14:12:39 2014 From: yjh0502@REDACTED (Jihyun Yu) Date: Mon, 8 Sep 2014 21:12:39 +0900 Subject: [erlang-questions] Inter-node communication bottleneck In-Reply-To: <1410173944.42603.YahooMailNeo@web126704.mail.ne1.yahoo.com> References: <1410173944.42603.YahooMailNeo@web126704.mail.ne1.yahoo.com> Message-ID: <20140908121239.GA9558@debian> Hi, I couldn't spend much time after last update. It seems that using multiple TCP sockets is not that scalable as well. I ran a test with following configurations. - 12 cores, 24 threads server (Intel L5640 x 2) - Two Erlang VM on two docker containers - Connects with 12 TCP connections and send/recv packets - [{delay_send,true}, {high_watermark,1024000}] With single integer term, total bandwidth is capped on about ~10MB/s. With single binary term <<"hello">>, total bandwidth is capped on about 20~30MB/s. Two Erlang VM consume almost all CPU resources (2400% total), so bandwidth per CPU usage is much worse than Erlang messaging case. I didn't run perf yet, so I don't know where is an bottleneck. I attached test source code so you can reproduce the result. Please tell me if there is an error on configurations/test codes/... [1] http://www.erlang.org/doc/man/inet.html On Mon, Sep 08, 2014 at 03:59:04AM -0700, Dror Mein wrote: > I encountered the same problem. any updates or new successes? the binary_to_term over tcp sounds promising. -------------- next part -------------- -module(test_tcp). -export([run_tcp/3, recv_tcp/3, send_tcp/4]). -define(SOCKET_OPTS, [{active, false}, binary, {packet, 4}]). run_tcp(Port, P, N) -> spawn_link(fun() -> recv_tcp(Port, P, N) end), send_tcp(localhost, 10000, P, N). recv_tcp(Port, P, N) -> {ok, ListenSocket} = gen_tcp:listen(Port, ?SOCKET_OPTS), lists:foreach(fun(Idx) -> {ok, Socket} = gen_tcp:accept(ListenSocket), spawn_monitor(fun() -> io:format("receiving ~p/~p~n", [Idx, P]), recv_n(Socket, N), io:format("receiving ~p/~p done~n", [Idx, P]), ok = gen_tcp:close(Socket) end) end, lists:seq(1, P)), waitdown(P), io:format("closing listen socket~n"), ok = gen_tcp:close(ListenSocket). recv_n(_Socket, 0) -> ok; recv_n(Socket, N) -> {ok, _Data} = gen_tcp:recv(Socket, 0), recv_n(Socket, N-1). send_tcp(Host, Port, P, N) -> lists:foreach(fun(Idx) -> spawn_monitor(fun() -> {ok, Socket} = gen_tcp:connect(Host, Port, ?SOCKET_OPTS), io:format("sending ~p/~p~n", [Idx, P]), send_n(Socket, N), io:format("sending ~p/~p done~n", [Idx, P]), ok = gen_tcp:close(Socket) end) end, lists:seq(1, P)), waitdown(P). send_n(_, 0) -> ok; send_n(Socket, N) -> gen_tcp:send(Socket, term_to_binary(N)), send_n(Socket, N-1). waitdown(0) -> ok; waitdown(Count) -> receive {'DOWN', _, process, _, normal} -> waitdown(Count-1) end. From mats.cronqvist@REDACTED Mon Sep 8 14:18:13 2014 From: mats.cronqvist@REDACTED (mats cronqvist) Date: Mon, 8 Sep 2014 05:18:13 -0700 (PDT) Subject: [erlang-questions] cnode interface memory allocation In-Reply-To: <1410172505.17155.YahooMailNeo@web126706.mail.ne1.yahoo.com> References: <1410167335.29492.YahooMailNeo@web126701.mail.ne1.yahoo.com> <540D782F.9020402@gandrade.net> <540D7D24.5080703@gandrade.net> <1410172505.17155.YahooMailNeo@web126706.mail.ne1.yahoo.com> Message-ID: On Monday, September 8, 2014 12:38:07 PM UTC+2, Dror Mein wrote: > I'll try to clarify the problem. > I have a char * in my code, which I want to erl_send to the erlang node. Apologies in advance for not answering the actual question... I did at one time write a fairly intricate cnode (https://github.com/massemanet/gtknode/tree/master/c_src) and had very little success with the erl_* stuff. I found the ei_* framework much more reliable. From jesper.louis.andersen@REDACTED Mon Sep 8 15:11:13 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 8 Sep 2014 15:11:13 +0200 Subject: [erlang-questions] Process state and sensitive information In-Reply-To: References: Message-ID: On Tue, Sep 2, 2014 at 1:51 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > Consider if you want to protect against data which is hot in memory. If > you decrypt, how do you make sure the data you decrypted is overwritten > straight away, once you don't need them anymore? This approach is so much > against the Erlang philosophy so it might require a NIF to pull off. > So, Colin Percival has been up to this for a while. The goal is to protect against an enemy grabbing hold of the memory area later. For example through a heartbleed-like attack. He starts out with some C code: http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html the problem is to make sure an aggressive optimization doesn't remove the memory zero of the buffer. He thinks he has a clever construction by threading the memset call through an unoptimizable pointer, but alas: http://www.daemonology.net/blog/2014-09-05-erratum.html No, this doesn't work :) And then he correctly identifies the real problem: http://www.daemonology.net/blog/2014-09-06-zeroing-buffers-is-insufficient.html which is that sensitive information can be in many locations and memory and you have no way of knowing where: stacks, registers, ... you name it. The consequence is that it is hard to guarantee a given piece of memory is overwritten and never used again. There is no good way of doing this right now, and it is a problem in computing in general. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From drormein@REDACTED Mon Sep 8 15:14:02 2014 From: drormein@REDACTED (Dror Mein) Date: Mon, 8 Sep 2014 06:14:02 -0700 Subject: [erlang-questions] Inter-node communication bottleneck In-Reply-To: <20140908121239.GA9558@debian> References: <1410173944.42603.YahooMailNeo@web126704.mail.ne1.yahoo.com> <20140908121239.GA9558@debian> Message-ID: <1410182042.55529.YahooMailNeo@web126702.mail.ne1.yahoo.com> active false worked better than active once? On Monday, September 8, 2014 3:12 PM, Jihyun Yu wrote: Hi, I couldn't spend much time after last update. It seems that using multiple TCP sockets is not that scalable as well. I ran a test with following configurations. - 12 cores, 24 threads server (Intel L5640 x 2) - Two Erlang VM on two docker containers - Connects with 12 TCP connections and send/recv packets - [{delay_send,true}, {high_watermark,1024000}] With single integer term, total bandwidth is capped on about ~10MB/s. With single binary term <<"hello">>, total bandwidth is capped on about 20~30MB/s. Two Erlang VM consume almost all CPU resources (2400% total), so bandwidth per CPU usage is much worse than Erlang messaging case. I didn't run perf yet, so I don't know where is an bottleneck. I attached test source code so you can reproduce the result. Please tell me if there is an error on configurations/test codes/... [1] http://www.erlang.org/doc/man/inet.html On Mon, Sep 08, 2014 at 03:59:04AM -0700, Dror Mein wrote: > I encountered the same problem. any updates or new successes? the binary_to_term over tcp sounds promising. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yjh0502@REDACTED Mon Sep 8 15:25:34 2014 From: yjh0502@REDACTED (Jihyun Yu) Date: Mon, 8 Sep 2014 22:25:34 +0900 Subject: [erlang-questions] Inter-node communication bottleneck In-Reply-To: <1410182042.55529.YahooMailNeo@web126702.mail.ne1.yahoo.com> References: <1410173944.42603.YahooMailNeo@web126704.mail.ne1.yahoo.com> <20140908121239.GA9558@debian> <1410182042.55529.YahooMailNeo@web126702.mail.ne1.yahoo.com> Message-ID: <20140908132534.GA1312@debian> I didn't tested, but AFAIK {active, once} is useful when process wants to handle messages other than TCP packet without blocking. On Mon, Sep 08, 2014 at 06:14:02AM -0700, Dror Mein wrote: > active false worked better than active once? > > > On Monday, September 8, 2014 3:12 PM, Jihyun Yu wrote: > > > > Hi, > > I couldn't spend much time after last > update. It seems that using > multiple TCP sockets is not that scalable as well. I ran a test with > following configurations. > > - 12 cores, 24 threads server (Intel L5640 x 2) > - Two Erlang VM on two docker containers > - Connects with 12 TCP connections and send/recv packets > - [{delay_send,true}, {high_watermark,1024000}] > > With single integer term, total bandwidth is capped on about ~10MB/s. > With single binary term <<"hello">>, total bandwidth is capped on about > 20~30MB/s. Two Erlang VM consume almost all CPU resources (2400% total), > so bandwidth per CPU usage is much worse than Erlang messaging case. I > didn't run perf yet, so I don't know where is an bottleneck. > > I attached test source code so you can > reproduce the result. Please tell > me if there is an error on configurations/test codes/... > > > [1] http://www.erlang.org/doc/man/inet.html > > > On Mon, Sep 08, 2014 at 03:59:04AM -0700, Dror Mein wrote: > > I encountered the same problem. any updates or new successes? the binary_to_term over tcp sounds promising. From sean@REDACTED Mon Sep 8 15:45:17 2014 From: sean@REDACTED (Sean Cribbs) Date: Mon, 8 Sep 2014 08:45:17 -0500 Subject: [erlang-questions] erl_tidy:file gives surprising output for -spec or -type In-Reply-To: <6255BC5D-6967-443B-B5C4-E4DA0E5D6CC6@gmail.com> References: <6255BC5D-6967-443B-B5C4-E4DA0E5D6CC6@gmail.com> Message-ID: Yes, that is my main use-case. On Mon, Sep 8, 2014 at 4:36 AM, Anthony Ramine wrote: > In my opinion, such a patch should also make syntax_tools aware of specs and types, so it should wrap these nodes in its own tree terms. > > -- > Anthony Ramine > > Le 8 sept. 2014 ? 01:11, Sean Cribbs a ?crit : > >> erl_syntax doesn't understand -type or -spec either. I'm tempted to >> write a patch since I'll be needing to generate -spec entries soon. > -- Sean Cribbs Software Engineer Basho Technologies, Inc. http://basho.com/ From kunthar@REDACTED Mon Sep 8 15:46:23 2014 From: kunthar@REDACTED (Gokhan Boranalp) Date: Mon, 8 Sep 2014 16:46:23 +0300 Subject: [erlang-questions] Inter-node communication bottleneck In-Reply-To: <20140908132534.GA1312@debian> References: <1410173944.42603.YahooMailNeo@web126704.mail.ne1.yahoo.com> <20140908121239.GA9558@debian> <1410182042.55529.YahooMailNeo@web126702.mail.ne1.yahoo.com> <20140908132534.GA1312@debian> Message-ID: Hello, I think the main problem is docker here. You should test it without docker. If you still insist to go with docker please check the items below: - How docker container is configured? - What kind of network setup you use within docker? Did you use Open switch etc.? - Did you do general tweak on machine, before you run the docker containers? Like incrementing file descriptors, tweaking TCP stack in sysconfig etc. See my example sysctl.conf below. Please do not use these values directly :) #kunthar tuning values net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 16384 16777216 net.core.somaxconn = 4096 net.core.netdev_max_backlog = 16384 net.ipv4.tcp_max_syn_backlog = 8192 net.ipv4.tcp_syncookies = 1 net.ipv4.ip_local_port_range = 1024 65535 net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_congestion_control = cubic #Disable ipv6 for now net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1 SNIP from /etc/init/docker.conf description "Docker daemon" start on (local-filesystems and net-device-up IFACE!=lo) stop on runlevel [!2345] This part should be added >>> limit nofile 524288 1048576 >>> limit nproc 524288 1048576 You also should pass needed params when run docker containers properly. Please read: http://docs.docker.com/reference/run/#runtime-constraints-on-cpu-and-memory http://docs.docker.com/reference/run/#runtime-privilege-linux-capabilities-and-lxc-configuration http://docs.docker.com/reference/run/ http://stackoverflow.com/questions/16084741/how-do-i-set-resources-allocated-to-a-container-using-docker Also, i strongly advise you to change your base docker image to this: https://github.com/phusion/baseimage-docker Good luck. On Mon, Sep 8, 2014 at 4:25 PM, Jihyun Yu wrote: > I didn't tested, but AFAIK {active, once} is useful when process wants > to handle messages other than TCP packet without blocking. > > On Mon, Sep 08, 2014 at 06:14:02AM -0700, Dror Mein wrote: >> active false worked better than active once? >> >> >> On Monday, September 8, 2014 3:12 PM, Jihyun Yu wrote: >> >> >> >> Hi, >> >> I couldn't spend much time after last >> update. It seems that using >> multiple TCP sockets is not that scalable as well. I ran a test with >> following configurations. >> >> - 12 cores, 24 threads server (Intel L5640 x 2) >> - Two Erlang VM on two docker containers >> - Connects with 12 TCP connections and send/recv packets >> - [{delay_send,true}, {high_watermark,1024000}] >> >> With single integer term, total bandwidth is capped on about ~10MB/s. >> With single binary term <<"hello">>, total bandwidth is capped on about >> 20~30MB/s. Two Erlang VM consume almost all CPU resources (2400% total), >> so bandwidth per CPU usage is much worse than Erlang messaging case. I >> didn't run perf yet, so I don't know where is an bottleneck. >> >> I attached test source code so you can >> reproduce the result. Please tell >> me if there is an error on configurations/test codes/... >> >> >> [1] http://www.erlang.org/doc/man/inet.html >> >> >> On Mon, Sep 08, 2014 at 03:59:04AM -0700, Dror Mein wrote: >> > I encountered the same problem. any updates or new successes? the binary_to_term over tcp sounds promising. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- BR, \|/ Kunthar From roger@REDACTED Mon Sep 8 17:16:19 2014 From: roger@REDACTED (Roger Lipscombe) Date: Mon, 8 Sep 2014 16:16:19 +0100 Subject: [erlang-questions] Using ssl (via ranch) {active, once} behaving like {active, twice} Message-ID: In R16B03-1, using SSL (with ranch) I'm seeing {active, once} behave as if it was {active, twice}. The client sends a message in 3 packets; I `receive` one packet, I call `recv`, which delivers the 3rd packet; process fails to parse the message and crashes; the crash log reveals that the 2nd packet is in the process message queue. What, if anything, am I doing wrong? Cheers, Roger. From jesper.louis.andersen@REDACTED Mon Sep 8 17:51:24 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 8 Sep 2014 17:51:24 +0200 Subject: [erlang-questions] Inter-node communication bottleneck In-Reply-To: <20140908121239.GA9558@debian> References: <1410173944.42603.YahooMailNeo@web126704.mail.ne1.yahoo.com> <20140908121239.GA9558@debian> Message-ID: On Mon, Sep 8, 2014 at 2:12 PM, Jihyun Yu wrote: > I attached test source code so you can reproduce the result. Please tell > me if there is an error on configurations/test codes/... > So I toyed around with this example for a while. My changes are here: https://gist.github.com/jlouis/0cbdd8581fc0651827d0 Test machine is a fairly old laptop: [jlouis@REDACTED ~/test_tcp]$ uname -a FreeBSD dragon.lan 10.0-RELEASE-p7 FreeBSD 10.0-RELEASE-p7 #0: Tue Jul 8 06:37:44 UTC 2014 root@REDACTED:/usr/obj/usr/src/sys/GENERIC amd64 [jlouis@REDACTED ~]$ sysctl hw.model hw.model: Intel(R) Core(TM)2 Duo CPU P8600 @ 2.40GHz All measurements are happening in both directions. We are sending bits to the kernel and receiving bits from the kernel as well. The base rate of this system was around 23 megabit per second running 4 sender processes and 4 receiver processes. Adding [{delay_send, true}] immediately sent this to 31 megabit per second, which kind of hints what is going in. This is not a bandwidth problem, it is a problem of latency and synchronous communication. Utilizing the {active, N} feature in 17+ by Steve Vinoski, removes the synchronicity bottleneck in the receiver direction. Eprof shows that CPU utilization falls from 10% per process to 1.5% on this machine. And then we run at 58 megabit. The reason we don't run any faster is due to the send path. A gen_tcp:send/2 only continues when the socket responds back that the message was sent with success. Since we only have one process per core, we end up dying of messaging overhead due to the messages being small and the concurrency of the system being bad. You can hack yourself out of this one with a bit of trickery and port_command/3 but I am not sure it is worth it. I also suspect this is why it doesn't help with a higher watermark. Your 4/12 processes are waiting for the underlying layer to send out data before it will send off the next piece of data to the underlying socket. Then the kernel gets to work, and gives the data to the receivers which then gets to consume it. At no point is the TCP send buffers filled up, really. To play the bandwidth game, you need to saturate your outgoing TCP buffers, so when the kernel goes to work, it has a lot of stuff to work with. What you are seeing is a common symptom: you are trading off latency, bandwidth utilization and correctness for each other. For messages of this small size and no other processing, you are essentially measuring a code path which includes a lot of context switches: between erlang processes and back'n'forth to the kernel. Since the concurrency of the system is fairly low (P is small), and we have a tight sending loop, you are going to lose with Erlang, every time. In a large system, the overhead you are seeing is fairly constant and thus it becomes irrelevant to the measurement. If we change send_n to contain bigger terms[0]: send_n(_, 0) -> ok; send_n(Socket, N) -> N1 = {N, N}, N2 = {N1, N1}, N3 = {N2, N2}, N4 = {N3, N3}, N5 = {N4, N4}, N6 = {N5, N5}, N7 = {N6, N6}, gen_tcp:send(Socket, term_to_binary(N7)), send_n(Socket, N-1). Then we hit 394 megabit on this machine. Furthermore, we can't even maximize the two CPU cores anymore as they are only running at 50% utilization. So now we are hitting the OS bottlenecks instead, which you have to tune for otherwise. In this example, we avoid the synchronicity of the gen_tcp:send/2 path since we are sending more work to the underlying system. You can probably run faster, but then you need to tune the TCP socket options as they are not made for gigabit speed operation by default. In order to figure all this out, I just ran eprof:profile(fun() -> tcp_test:run_tcp(2000, 4, 1000*10) end). eprof:log("foo.txt"). eprof:analyze(). and went to work by analyzing the profile output. Docker or not, I believe there are other factors at play here... [0] Small nice persistence trick used here: building a tree of exponential size in linear time. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean@REDACTED Mon Sep 8 18:00:01 2014 From: sean@REDACTED (Functional Jobs) Date: Mon, 8 Sep 2014 12:00:01 -0400 Subject: [erlang-questions] New Functional Programming Job Opportunities Message-ID: <540dd28304681@functionaljobs.com> Here are some functional programming job opportunities that were posted recently: Erlang/OTP Expert at A10 Networks http://functionaljobs.com/jobs/8739-erlang-otp-expert-at-a10-networks Cheers, Sean Murphy FunctionalJobs.com From drormein@REDACTED Mon Sep 8 18:11:17 2014 From: drormein@REDACTED (Dror Mein) Date: Mon, 8 Sep 2014 09:11:17 -0700 Subject: [erlang-questions] Inter-node communication bottleneck In-Reply-To: References: <1410173944.42603.YahooMailNeo@web126704.mail.ne1.yahoo.com> <20140908121239.GA9558@debian> Message-ID: <1410192677.91164.YahooMailNeo@web126705.mail.ne1.yahoo.com> Do you any advice for those who haven't migrated to 17 yet for the receive side? Has anyone tried to hack out the synchronous troubles of gen_tcp:send? I can always aggregate messages to achieve higher throughput, but why not add a asynchronous gen_tcp:send option? On Monday, September 8, 2014 6:51 PM, Jesper Louis Andersen wrote: On Mon, Sep 8, 2014 at 2:12 PM, Jihyun Yu wrote: I attached test source code so you can reproduce the result. Please tell >me if there is an error on configurations/test codes/... So I toyed around with this example for a while. My changes are here: https://gist.github.com/jlouis/0cbdd8581fc0651827d0 Test machine is a fairly old laptop: [jlouis@REDACTED ~/test_tcp]$ uname -a FreeBSD dragon.lan 10.0-RELEASE-p7 FreeBSD 10.0-RELEASE-p7 #0: Tue Jul 8 06:37:44 UTC 2014 root@REDACTED:/usr/obj/usr/src/sys/GENERIC amd64 [jlouis@REDACTED ~]$ sysctl hw.model hw.model: Intel(R) Core(TM)2 Duo CPU P8600 @ 2.40GHz All measurements are happening in both directions. We are sending bits to the kernel and receiving bits from the kernel as well. The base rate of this system was around 23 megabit per second running 4 sender processes and 4 receiver processes. Adding [{delay_send, true}] immediately sent this to 31 megabit per second, which kind of hints what is going in. This is not a bandwidth problem, it is a problem of latency and synchronous communication. Utilizing the {active, N} feature in 17+ by Steve Vinoski, removes the synchronicity bottleneck in the receiver direction. Eprof shows that CPU utilization falls from 10% per process to 1.5% on this machine. And then we run at 58 megabit. The reason we don't run any faster is due to the send path. A gen_tcp:send/2 only continues when the socket responds back that the message was sent with success. Since we only have one process per core, we end up dying of messaging overhead due to the messages being small and the concurrency of the system being bad. You can hack yourself out of this one with a bit of trickery and port_command/3 but I am not sure it is worth it. I also suspect this is why it doesn't help with a higher watermark. Your 4/12 processes are waiting for the underlying layer to send out data before it will send off the next piece of data to the underlying socket. Then the kernel gets to work, and gives the data to the receivers which then gets to consume it. At no point is the TCP send buffers filled up, really. To play the bandwidth game, you need to saturate your outgoing TCP buffers, so when the kernel goes to work, it has a lot of stuff to work with. What you are seeing is a common symptom: you are trading off latency, bandwidth utilization and correctness for each other. For messages of this small size and no other processing, you are essentially measuring a code path which includes a lot of context switches: between erlang processes and back'n'forth to the kernel. Since the concurrency of the system is fairly low (P is small), and we have a tight sending loop, you are going to lose with Erlang, every time. In a large system, the overhead you are seeing is fairly constant and thus it becomes irrelevant to the measurement. If we change send_n to contain bigger terms[0]: send_n(_, 0) -> ok; send_n(Socket, N) -> N1 = {N, N}, N2 = {N1, N1}, N3 = {N2, N2}, N4 = {N3, N3}, N5 = {N4, N4}, N6 = {N5, N5}, N7 = {N6, N6}, gen_tcp:send(Socket, term_to_binary(N7)), send_n(Socket, N-1). Then we hit 394 megabit on this machine. Furthermore, we can't even maximize the two CPU cores anymore as they are only running at 50% utilization. So now we are hitting the OS bottlenecks instead, which you have to tune for otherwise. In this example, we avoid the synchronicity of the gen_tcp:send/2 path since we are sending more work to the underlying system. You can probably run faster, but then you need to tune the TCP socket options as they are not made for gigabit speed operation by default. In order to figure all this out, I just ran eprof:profile(fun() -> tcp_test:run_tcp(2000, 4, 1000*10) end). eprof:log("foo.txt"). eprof:analyze(). and went to work by analyzing the profile output. Docker or not, I believe there are other factors at play here... [0] Small nice persistence trick used here: building a tree of exponential size in linear time. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Sep 8 19:36:47 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 8 Sep 2014 19:36:47 +0200 Subject: [erlang-questions] Inter-node communication bottleneck In-Reply-To: <1410192677.91164.YahooMailNeo@web126705.mail.ne1.yahoo.com> References: <1410173944.42603.YahooMailNeo@web126704.mail.ne1.yahoo.com> <20140908121239.GA9558@debian> <1410192677.91164.YahooMailNeo@web126705.mail.ne1.yahoo.com> Message-ID: On Mon, Sep 8, 2014 at 6:11 PM, Dror Mein wrote: > Do you any advice for those who haven't migrated to 17 yet for the receive > side? Try {active, once}. It was optimized in Release 16.0, so you may be able to reap some of the benefits there already. As for a faster gen_tcp:send/2 path, I believe it is a question of documentation, support and effort. It is one of the places where a well-designed and well-placed patch is worth the work. Do note, however that for typical work-loads this might not be a bottleneck. Measure before you start designing around it. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Sep 8 20:57:02 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Mon, 08 Sep 2014 20:57:02 +0200 Subject: [erlang-questions] Using ssl (via ranch) {active, once} behaving like {active, twice} In-Reply-To: References: Message-ID: <540DFBFE.2000803@ninenines.eu> Does it do it with normal SSL too or just with Ranch? Depending on how you open the socket (for example using the {socket, Socket} option) it may have active set to once already, which means it will send the message beforehand. Ranch itself shouldn't, but if you open it manually it may happen. On 09/08/2014 05:16 PM, Roger Lipscombe wrote: > In R16B03-1, using SSL (with ranch) I'm seeing {active, once} behave > as if it was {active, twice}. > > The client sends a message in 3 packets; I `receive` one packet, I > call `recv`, which delivers the 3rd packet; process fails to parse the > message and crashes; the crash log reveals that the 2nd packet is in > the process message queue. > > What, if anything, am I doing wrong? > > Cheers, > Roger. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu From H.Li@REDACTED Mon Sep 8 22:30:16 2014 From: H.Li@REDACTED (H.Li@REDACTED) Date: Mon, 8 Sep 2014 21:30:16 +0100 (BST) Subject: [erlang-questions] processes stuck in erlang:bif_return_trap/1 Message-ID: <1fd2497815c1319d242043ce8f42b3f7.squirrel@webmail.cs.kent.ac.uk> Dear All, I wonder if anyone could help me with this. We have been experiencing some problems with an Erlang node running Mnesia. For some reason, the node becomes unresponsive every few minutes, but the system does recover after a short period of time. We did some profiling using ETop, and it shows that when the system freezes, there are about 5000 processes accumulated on this node getting stuck in the erlang:bif_return_trap/1 function. These processes are spawned by a rpc_server running on this node; each process computes a M:F(A) then sends the result back via gen_tcp or stores the result in an ets table. I don't quite understand what erlang:bif_return_trap/1 does, and am confused why so many processes got stuck in this function. The Erlang node is running on a 12 physical core machine (hence 24 schedulers), and the version of Erlang is R15B03. Here is part of the ETop output: Load: cpu 317 Memory: total 66521736 binary 393593 procs 5785 processes 1253072 code 12280 runq 0 atom 493 ets 64843631 Pid Name or Initial Func Time Reds Memory MsgQ Current Function ---------------------------------------------------------------------------------------- <5291.115.0> mnesia_tm '-' 0******** 0 mnesia_tm:doit_loop/ <5291.6.0> error_logger '-' 012342496 0 gen_event:fetch_msg/ <5291.61.0> memsup '-' 138 4714912 0 gen_server:loop/6 <5291.1464.0> proc_lib:init_p/5 '-' 103 2914384 0 gen_fsm:loop/7 <5291.1466.0> proc_lib:init_p/5 '-' 105 2914384 0 gen_fsm:loop/7 . . . ***************rpc_socket:worker/6 '-' 8141 101144 0 erlang:bif_return_tr ***************rpc_socket:worker/6 '-' 7900 101144 0 erlang:bif_return_tr ***************rpc_socket:worker/6 '-' 7901 101144 0 erlang:bif_return_tr ***************rpc_socket:worker/6 '-' 7954 101144 0 erlang:bif_return_tr ***************rpc_socket:worker/6 '-' 7975 101144 0 erlang:bif_return_tr ***************rpc_socket:worker/6 '-' 8068 101144 0 erlang:bif_return_tr ***************rpc_socket:worker/6 '-' 7956 101144 0 erlang:bif_return_tr ***************rpc_socket:worker/6 '-' 8212 101144 0 erlang:bif_return_tr Many Thanks! Huiqing -------------- next part -------------- An HTML attachment was scrubbed... URL: From yjh0502@REDACTED Mon Sep 8 23:46:16 2014 From: yjh0502@REDACTED (Jihyun Yu) Date: Tue, 9 Sep 2014 06:46:16 +0900 Subject: [erlang-questions] Inter-node communication bottleneck In-Reply-To: References: <1410173944.42603.YahooMailNeo@web126704.mail.ne1.yahoo.com> <20140908121239.GA9558@debian> Message-ID: <20140908214616.GA1337@debian> Hi, Summerizing benchmark result in terms of bandwidth is misleading and my mistake. I wanted to talk about overhead of small messages, and how many inter-node messages Erlang can handle. My initial question was about bottleneck on inter-node messaging [1], and with some tuning, throughput could be increased about 0.5M messages per second. I thought that the problem is on using single TCP connections to handle all messages from multiple processes, and throughput (# of msgs/sec) might be increased with multiple TCP connections [2]. It seems that Erlang messaging with multiple nodes does not scale with number of CPUs, even with multiple TCP connections. For example in my test environment, P=2 shows 1.5M msgs/sec with ~18% usr+sys CPU, and P=12 shows 1.8M msgs/sec with ~90% usr+sys CPU. It seems that Erlang VM cannot handle more than 2M messages without explicit tuning like message batching. 2M msgs/sec, or even 0.5M msgs/sec might enough for most use cases, so it might not a problem. [1] http://erlang.org/pipermail/erlang-questions/2014-August/080598.html [2] http://erlang.org/pipermail/erlang-questions/2014-August/080643.html On Mon, Sep 08, 2014 at 05:51:24PM +0200, Jesper Louis Andersen wrote: > On Mon, Sep 8, 2014 at 2:12 PM, Jihyun Yu wrote: > > > I attached test source code so you can reproduce the result. Please tell > > me if there is an error on configurations/test codes/... > > > > So I toyed around with this example for a while. My changes are here: > > https://gist.github.com/jlouis/0cbdd8581fc0651827d0 > > Test machine is a fairly old laptop: > > [jlouis@REDACTED ~/test_tcp]$ uname -a > FreeBSD dragon.lan 10.0-RELEASE-p7 FreeBSD 10.0-RELEASE-p7 #0: Tue Jul 8 > 06:37:44 UTC 2014 > root@REDACTED:/usr/obj/usr/src/sys/GENERIC > amd64 > [jlouis@REDACTED ~]$ sysctl hw.model > hw.model: Intel(R) Core(TM)2 Duo CPU P8600 @ 2.40GHz > > All measurements are happening in both directions. We are sending bits to > the kernel and receiving bits from the kernel as well. > > The base rate of this system was around 23 megabit per second running 4 > sender processes and 4 receiver processes. Adding [{delay_send, true}] > immediately sent this to 31 megabit per second, which kind of hints what is > going in. This is not a bandwidth problem, it is a problem of latency and > synchronous communication. Utilizing the {active, N} feature in 17+ by > Steve Vinoski, removes the synchronicity bottleneck in the receiver > direction. Eprof shows that CPU utilization falls from 10% per process to > 1.5% on this machine. And then we run at 58 megabit. > > The reason we don't run any faster is due to the send path. A > gen_tcp:send/2 only continues when the socket responds back that the > message was sent with success. Since we only have one process per core, we > end up dying of messaging overhead due to the messages being small and the > concurrency of the system being bad. You can hack yourself out of this one > with a bit of trickery and port_command/3 but I am not sure it is worth it. > I also suspect this is why it doesn't help with a higher watermark. Your > 4/12 processes are waiting for the underlying layer to send out data before > it will send off the next piece of data to the underlying socket. Then the > kernel gets to work, and gives the data to the receivers which then gets to > consume it. At no point is the TCP send buffers filled up, really. > > To play the bandwidth game, you need to saturate your outgoing TCP buffers, > so when the kernel goes to work, it has a lot of stuff to work with. > > What you are seeing is a common symptom: you are trading off latency, > bandwidth utilization and correctness for each other. For messages of this > small size and no other processing, you are essentially measuring a code > path which includes a lot of context switches: between erlang processes and > back'n'forth to the kernel. Since the concurrency of the system is fairly > low (P is small), and we have a tight sending loop, you are going to lose > with Erlang, every time. In a large system, the overhead you are seeing is > fairly constant and thus it becomes irrelevant to the measurement. > > If we change send_n to contain bigger terms[0]: > > send_n(_, 0) -> ok; > send_n(Socket, N) -> > N1 = {N, N}, > N2 = {N1, N1}, > N3 = {N2, N2}, > N4 = {N3, N3}, > N5 = {N4, N4}, > N6 = {N5, N5}, > N7 = {N6, N6}, > gen_tcp:send(Socket, term_to_binary(N7)), > send_n(Socket, N-1). > > Then we hit 394 megabit on this machine. Furthermore, we can't even > maximize the two CPU cores anymore as they are only running at 50% > utilization. So now we are hitting the OS bottlenecks instead, which you > have to tune for otherwise. In this example, we avoid the synchronicity of > the gen_tcp:send/2 path since we are sending more work to the underlying > system. You can probably run faster, but then you need to tune the TCP > socket options as they are not made for gigabit speed operation by default. > > In order to figure all this out, I just ran > > eprof:profile(fun() -> tcp_test:run_tcp(2000, 4, 1000*10) end). > eprof:log("foo.txt"). > eprof:analyze(). > > and went to work by analyzing the profile output. > > Docker or not, I believe there are other factors at play here... > > [0] Small nice persistence trick used here: building a tree of exponential > size in linear time. > > -- > J. From essen@REDACTED Mon Sep 8 23:47:20 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Mon, 08 Sep 2014 23:47:20 +0200 Subject: [erlang-questions] Feedback on inaka/erlang_standards Message-ID: <540E23E8.1060500@ninenines.eu> Hello, I want to initiate a larger discussion that Twitter allows for the inaka/erlang_standards repository. I hope you don't mind that I do it here, Github issues is quite terrible for discussions and I feel that talking about these things with a larger number of people can bring better feedback. I will take all points listed in the README at https://github.com/inaka/erlang_standards and comment if I have something (hopefully) interesting to say about it. I know they are "Inaka's coding standards" but they are also used for the Elvis coding standard checker project which would be interesting to try and integrate as part of more people's workflow (regardless of projects using the same standards or not, simply checking that contributions respect it is already quite nice to have). An example output can be seen there: https://gist.github.com/elbrujohalcon/332db8bc343f81235462 So let me start now. > Spaces over tabs, 2 space indentation. Whatever floats your boat. I just want to point out here that 2-4 space vs tab vs mixed never lead anywhere. My problem with it though is that the Elvis tool has an option "no_tabs". That's very opinionated, instead you should have an option that allows you to define indentation. For example I want to know if people use spaces instead of tabs in my projects. > Surround operators and commas with spaces. As much as I like having these with spaces around, I have an issue with that. I feel that the default should be "whatever Erlang is doing". For example maps, it does "#{a => b,c => d}". The default your tool should do is exactly what Erlang does, because you can then just copy paste everything and not have to worry. We could extend that logic to doing what erl_pp does but people really like their tabs/spaces/goats so that'd probably be taking it a bit too far. > Try to always separate PRIVATE and PUBLIC functions in groups, with the public ones first, unless it helps readability and code discovery. I have to disagree with that. This is what I have done for a *long time* but I experimented and changed my mind recently. What I do now is for *each* exported function (nice OOP language btw), start with the exported function, then the internal functions it calls, and then the unit tests (and performance tests). Then do the same for the next exported function and so on. It doesn't fit all kinds of modules (of course!) but for modules that aren't doing processes stuff (like gen_servers) it's very nice. For example: https://github.com/ninenines/cowlib/blob/master/src/cow_qs.erl - if that module was doing the old public, private, tests, it would be pretty much unreadable. Having the unit tests right there with the code is very nice because you can easily see what the code takes and returns. > Don't design your system using god modules (modules that have a huge number of functions and/or deal with very unrelated things) Well that's fine if your problem domain is simple, but not so much otherwise. The problem with your approach comes from being able to set a maximum number of functions project-wide. In Cowboy for example most modules are small, and cowboy_req is the exception. I know cowboy_req has a lot of exported functions, I don't want a tool to tell me that. It's meant to be this way. > Don't write the same code in many places, use functions and variables for that I have for a long time been a proponent of applying DRY only when this code exists more than 2 times, and the code is significant enough. I care little about repeated code that's 2 lines long for example. I don't have hard limits there but it would be nice to analyze a few projects with different settings and find what's the most useful. > When having lots of modules, use subdirectories for them, named with a nice descriptive name for what that "package" does. Sounds like you need to separate into different applications. > Don't write spaghetti code (A list comprehension with a case inside, or blocks with begin/end, and nested stuff) Not what I call spaghetti code? I'm not even sure what this is about. List comprehensions with a case or begin/end inside can be immensely useful. Especially the begin/end ones. > Don't use if . I would say the opposite. ROK has already argued on that, no need to duplicate. :-) > Macros should be named in ALL_UPPER_CASE: I would say "Don't use macros" instead. I would love the tool to tell me when it sees macros that aren't ?MODULE (and a way to whitelist a few small ones). > Write the -spec's for your public fun's, and for private fun's when it adds real value for documentation purposes. Define as many types as needed. Honestly I'd remove the comment about internal functions (wow OOP man), they're generally not very useful, unless Dialyzer complains. > Module comments go with %%%, function comments with %%, and code comments with %. Or you could stop worrying and enjoy life. Seriously that's some nitpicking you got there. :-) Instead of saying that you should probably worry that people *actually* comment their code. > Try to write functions with a small number of lines. 12 lines per function except for integration tests is a good measure. 12, seriously? If I'm writing a game the init function or main loop function will hardly be less than 20 lines. > Help dialyzer and xref as much as you can, so that they can work for you Like with most things in life, there's an optimal middle ground. Don't go out of your way to make tools happy. Tools are supposed to help you not hinder you. > When having many nested "include files", use -ifndef(HEADER_FILE_HRL) .... -endif so they can be included in any order without conflicts. I'll need an option that complains when -include is used (again, with whitelist, probably never warn about file.hrl either). :-) That's it. Hope to spark some (not too pointless) discussion. Enjoy! -- Lo?c Hoguin http://ninenines.eu From jesper.louis.andersen@REDACTED Tue Sep 9 00:06:54 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 9 Sep 2014 00:06:54 +0200 Subject: [erlang-questions] processes stuck in erlang:bif_return_trap/1 In-Reply-To: <1fd2497815c1319d242043ce8f42b3f7.squirrel@webmail.cs.kent.ac.uk> References: <1fd2497815c1319d242043ce8f42b3f7.squirrel@webmail.cs.kent.ac.uk> Message-ID: On Mon, Sep 8, 2014 at 10:30 PM, wrote: > I don't quite understand what erlang:bif_return_trap/1 does, and am > confused why so many processes got stuck in this function. > It is a Red Herring(tm)[0]. Suppose the VM has to execute some BIF, link for example, for a process P. In the course of linking P to another process, the VM will figure out it needs to send a distributed message and that the distribution channel is clogged up. So the VM suspends P until the link message has succeeded. The problem now, is that we can't return to P, since it is suspended, and we have a result from the link function we wish to tell P once it is unsuspended. We can bump the reduction counter to 0, but this will still have P execute until the next reduction counter bump, which is not what we want. This is where bif_return_trap/1 comes in. We mangle the state of P such that when it unsuspends, it is executing bif_return_trap/1 and the value is ready for returning. That is, we arrange it such that bif_return_trap/1 is an immediate continuation of the program. This is why it is a red herring. Some other function has executed and mangled your current state. If distribution/ports are in play, consider using erlang:system_monitor/2 to watch over the errornous state. There are some old posts by Paul Davis and Scott L. Fritchie which could be somewhat related: http://erlang.org/pipermail/erlang-questions/2012-December/071157.html [0] Ni! Ni! Ni! Ni! Ni! -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Hal.Snyder@REDACTED Tue Sep 9 00:46:41 2014 From: Hal.Snyder@REDACTED (Snyder, Hal) Date: Mon, 8 Sep 2014 22:46:41 +0000 Subject: [erlang-questions] Feedback on inaka/erlang_standards In-Reply-To: <540E23E8.1060500@ninenines.eu> References: <540E23E8.1060500@ninenines.eu> Message-ID: You might ask inaka to start a chat room for inaka/erlang_standards on gitter.im. See https://gitter.im/sympy/sympy for an example On 9/8/14, 4:47 PM, "Lo?c Hoguin" wrote: >I want to initiate a larger discussion that Twitter allows for the >inaka/erlang_standards repository. I hope you don't mind that I do it >here, Github issues is quite terrible for discussions and I feel that >talking about these things with a larger number of people can bring >better feedback. From ingela.andin@REDACTED Tue Sep 9 08:56:41 2014 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 9 Sep 2014 08:56:41 +0200 Subject: [erlang-questions] Using ssl (via ranch) {active, once} behaving like {active, twice} In-Reply-To: References: Message-ID: Hi! You are aware of that sockets by default are active? So you want to set the socket active option already in listen. Regards Ingela Erlang/OTP team 2014-09-08 17:16 GMT+02:00 Roger Lipscombe : > In R16B03-1, using SSL (with ranch) I'm seeing {active, once} behave > as if it was {active, twice}. > > The client sends a message in 3 packets; I `receive` one packet, I > call `recv`, which delivers the 3rd packet; process fails to parse the > message and crashes; the crash log reveals that the 2nd packet is in > the process message queue. > > What, if anything, am I doing wrong? > > Cheers, > Roger. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Tue Sep 9 09:10:44 2014 From: roger@REDACTED (Roger Lipscombe) Date: Tue, 9 Sep 2014 08:10:44 +0100 Subject: [erlang-questions] Using ssl (via ranch) {active, once} behaving like {active, twice} In-Reply-To: <540DFBFE.2000803@ninenines.eu> References: <540DFBFE.2000803@ninenines.eu> Message-ID: On 8 September 2014 19:57, Lo?c Hoguin wrote: > Does it do it with normal SSL too or just with Ranch? I haven't tested with normal SSL at the moment -- I'm only seeing this from a single client, and only on one server (the servers are chef-identical, btw), so I've not succeeded in reproducing it yet. > Depending on how you open the socket (for example using the {socket, Socket} > option) it may have active set to once already, which means it will send the > message beforehand. Ranch itself shouldn't, but if you open it manually it > may happen. We are using {socket, Socket} with ranch, but we're definitely setting {active, false} on the listening socket. Is this correctly inherited by the connected socket the first time round? Either way, a bunch of single-packet messages arrive and are processed successfully first, which suggests that we're getting to set {active, once} on that socket before this problem occurs. The loop looks (more-or-less) like this: loop() -> {OK, Closed, Error} = Transport:messages(), % Transport == ranch_ssl Transport:setopts(Socket, [{active, once}]), receive {OK, Socket, Data} -> start_request(Data), % etc. end end. start_request(Data) -> % message has a size prefix, and might span more than one packet. case needs_more(Data) of true -> get_more_with_recv(), % if you see what I mean end. So, it appears to be in the transition from {active, once} using receive to {active, false} -- which should be automatic -- using recv. But it only seems to happen when a single message from the client spans more than one packet. And it's only happening from this one particular client. I don't have any metrics for how often messages span more than one packet, unfortunately. If I test it locally using ssl:send, then it's correct. Wireshark shows nothing odd about the packets, as far as I can tell, it's just that the first one arrives as a message to the controlling process, as does the second one (but shouldn't), and the third one appears in recv. We're using R16 -- is this a bug in ssl that's fixed in 17? Is there something wrong with the way I'm using it? Should I implement some kind of workaround? The reason that I'm using this active/passive thing is that certain kinds of outbound messages are sent as messages to the process from elsewhere in the application, and we don't want to send them while it's processing an inbound message (because they'll get ahead of the response to the inbound message), and since an inbound message can be split across multiple packets, we opted to do it this way. Regards, Roger. From essen@REDACTED Tue Sep 9 09:22:19 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 09 Sep 2014 09:22:19 +0200 Subject: [erlang-questions] Feedback on inaka/erlang_standards In-Reply-To: References: <540E23E8.1060500@ninenines.eu> Message-ID: <540EAAAB.2070100@ninenines.eu> Please read again what I said instead of spamming. I want to bring more people into the discussion, that can't happen if I post on either github or random chat service. Especially considering you need to create an account to talk on random chat service. And that random chat service looks just as bad as github issues, if not worse, as you can't seem to just use email with it. On 09/09/2014 12:46 AM, Snyder, Hal wrote: > You might ask inaka to start a chat room for inaka/erlang_standards on > gitter.im. > > > See https://gitter.im/sympy/sympy for an example > > On 9/8/14, 4:47 PM, "Lo?c Hoguin" wrote: > >> I want to initiate a larger discussion that Twitter allows for the >> inaka/erlang_standards repository. I hope you don't mind that I do it >> here, Github issues is quite terrible for discussions and I feel that >> talking about these things with a larger number of people can bring >> better feedback. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu From ingela.andin@REDACTED Tue Sep 9 09:54:59 2014 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 9 Sep 2014 09:54:59 +0200 Subject: [erlang-questions] Using ssl (via ranch) {active, once} behaving like {active, twice} In-Reply-To: References: <540DFBFE.2000803@ninenines.eu> Message-ID: Hi! 2014-09-09 9:10 GMT+02:00 Roger Lipscombe : > On 8 September 2014 19:57, Lo?c Hoguin wrote: > > Does it do it with normal SSL too or just with Ranch? > > I haven't tested with normal SSL at the moment -- I'm only seeing this > from a single client, and only on one server (the servers are > chef-identical, btw), so I've not succeeded in reproducing it yet. > > > Depending on how you open the socket (for example using the {socket, > Socket} > > option) it may have active set to once already, which means it will send > the > > message beforehand. Ranch itself shouldn't, but if you open it manually > it > > may happen. > > We are using {socket, Socket} with ranch, but we're definitely setting > {active, false} on the listening socket. Is this correctly inherited > by the connected socket the first time round? If Ranch uses a pool of accept sockets it is possible that you you are seeing an inheritance bug in ssl that is fixed in 17. > Either way, a bunch of > single-packet messages arrive and are processed successfully first, > which suggests that we're getting to set {active, once} on that socket > before this problem occurs. The loop looks (more-or-less) like this: > > loop() -> > {OK, Closed, Error} = Transport:messages(), % Transport == ranch_ssl > Transport:setopts(Socket, [{active, once}]), > receive > {OK, Socket, Data} -> start_request(Data), > % etc. > end > end. > > start_request(Data) -> > % message has a size prefix, and might span more than one packet. > case needs_more(Data) of > true -> get_more_with_recv(), > % if you see what I mean > end. > > So, it appears to be in the transition from {active, once} using > receive to {active, false} -- which should be automatic -- using recv. > But it only seems to happen when a single message from the client > spans more than one packet. And it's only happening from this one > particular client. I don't have any metrics for how often messages > span more than one packet, unfortunately. > > If I test it locally using ssl:send, then it's correct. > > Wireshark shows nothing odd about the packets, as far as I can tell, > it's just that the first one arrives as a message to the controlling > process, as does the second one (but shouldn't), and the third one > appears in recv. > > We're using R16 -- is this a bug in ssl that's fixed in 17? Is there > something wrong with the way I'm using it? Should I implement some > kind of workaround? > > The reason that I'm using this active/passive thing is that certain > kinds of outbound messages are sent as messages to the process from > elsewhere in the application, and we don't want to send them while > it's processing an inbound message (because they'll get ahead of the > response to the inbound message), and since an inbound message can be > split across multiple packets, we opted to do it this way. > > I would say that from an OTP view of things, the normal way to handle socket messages is with active once. >From a supervisor and code upgrading perspective you do usually not want black a process in a recv call. Regards Ingela Erlang/OTP team - Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From H.Li@REDACTED Tue Sep 9 10:09:16 2014 From: H.Li@REDACTED (H.Li) Date: Tue, 9 Sep 2014 09:09:16 +0100 Subject: [erlang-questions] processes stuck in erlang:bif_return_trap/1 In-Reply-To: References: <1fd2497815c1319d242043ce8f42b3f7.squirrel@webmail.cs.kent.ac.uk> Message-ID: <20140909080916.GA26038@banyan.kent.ac.uk> Many thanks! That's helpful. Regards, Huiqing On Tue, Sep 09, 2014 at 12:06:54AM +0200, Jesper Louis Andersen wrote: > On Mon, Sep 8, 2014 at 10:30 PM, wrote: > > > I don't quite understand what erlang:bif_return_trap/1 does, and am > > confused why so many processes got stuck in this function. > > > > It is a Red Herring(tm)[0]. > > Suppose the VM has to execute some BIF, link for example, for a process P. > In the course of linking P to another process, the VM will figure out it > needs to send a distributed message and that the distribution channel is > clogged up. So the VM suspends P until the link message has succeeded. The > problem now, is that we can't return to P, since it is suspended, and we > have a result from the link function we wish to tell P once it is > unsuspended. We can bump the reduction counter to 0, but this will still > have P execute until the next reduction counter bump, which is not what we > want. > > This is where bif_return_trap/1 comes in. We mangle the state of P such > that when it unsuspends, it is executing bif_return_trap/1 and the value is > ready for returning. That is, we arrange it such that bif_return_trap/1 is > an immediate continuation of the program. > > This is why it is a red herring. Some other function has executed and > mangled your current state. If distribution/ports are in play, consider > using erlang:system_monitor/2 to watch over the errornous state. There are > some old posts by Paul Davis and Scott L. Fritchie which could be somewhat > related: > > http://erlang.org/pipermail/erlang-questions/2012-December/071157.html > > [0] Ni! Ni! Ni! Ni! Ni! > > > -- > J. From roger@REDACTED Tue Sep 9 10:24:06 2014 From: roger@REDACTED (Roger Lipscombe) Date: Tue, 9 Sep 2014 09:24:06 +0100 Subject: [erlang-questions] Using ssl (via ranch) {active, once} behaving like {active, twice} In-Reply-To: References: <540DFBFE.2000803@ninenines.eu> Message-ID: On 9 September 2014 08:54, Ingela Andin wrote: > If Ranch uses a pool of accept sockets it is possible that you > you are seeing an inheritance bug in ssl that is fixed in 17. We are using a pool of accept sockets; but I would have thought that the fact that we've been around the loop and set {active, once} before this happens means that we're not getting bitten by this? Do you have a link to the fix in 17? > I would say that from an OTP view of things, the normal way to handle socket > messages is with active once. We are using {active, once}; can you point to an example of an OTP application that handles messages across multiple packets? I'm pretty sure that I can sort this out by using {active, true} and using selective receive, but I'd like to see an example with {active, once}. > From a supervisor and code upgrading perspective you do usually not want > black a process in a recv call. Surely that's {active, true}, or am I missing something? Regards, Roger. From ingela.andin@REDACTED Tue Sep 9 11:57:33 2014 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 9 Sep 2014 11:57:33 +0200 Subject: [erlang-questions] Using ssl (via ranch) {active, once} behaving like {active, twice} In-Reply-To: References: <540DFBFE.2000803@ninenines.eu> Message-ID: Hi! 2014-09-09 10:24 GMT+02:00 Roger Lipscombe : > On 9 September 2014 08:54, Ingela Andin wrote: > > If Ranch uses a pool of accept sockets it is possible that you > > you are seeing an inheritance bug in ssl that is fixed in 17. > > We are using a pool of accept sockets; but I would have thought that > the fact that we've been around the loop and set {active, once} before > this happens means that we're not getting bitten by this? > > Do you have a link to the fix in 17? > Fixed by commit: c80735b0c82ec0bbaa2e2bda4f1598210b193556 > > > I would say that from an OTP view of things, the normal way to handle > socket > > messages is with active once. > > We are using {active, once}; can you point to an example of an OTP > application that handles messages across multiple packets? I'm pretty > sure that I can sort this out by using {active, true} and using > selective receive, but I'd like to see an example with {active, once}. > The ssl application uses active once internally (the active once on an sslsocket is emulated), so does the ssh application and inets HTTP-server to name a few. > > > From a supervisor and code upgrading perspective you do usually not want > > black a process in a recv call. > > Surely that's {active, true}, or am I missing something? > Active once (or nowdays even active N) can be used to get some flow control as opposed to active mode. Regards Ingela Erlang/OTP Team - Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Tue Sep 9 12:53:47 2014 From: ivan@REDACTED (Ivan Uemlianin) Date: Tue, 09 Sep 2014 11:53:47 +0100 Subject: [erlang-questions] Feedback on inaka/erlang_standards In-Reply-To: <540E23E8.1060500@ninenines.eu> References: <540E23E8.1060500@ninenines.eu> Message-ID: <540EDC3B.7050604@llaisdy.com> I like the idea of a configurable coding standard checker, if that's what elvis is, and as long as team members read and discuss (and agree on) the team's "elvis.config". Discussing and agreeing on standards is as important as following them. Each team will have their own conventions: their own boats to float. As for particular points: - I tend to avoid if, and macros as far as possible: two fewer bits of syntax to remember. I notice I'm starting to avoid case too. > Module comments go with %%%, function comments with %%, and code comments with %. This is perhaps following emacs erlang-mode behaviour (which auto-intends comments differently depending on how many %s start the line). Discussing coding style is worthwhile. I think a monolithic One True Style is less useful, especially in a programming culture as articulate and reasoned (and occasionally stiff-necked) as erlang's. Best wishes Ivan On 08/09/2014 22:47, Lo?c Hoguin wrote: > Hello, > > I want to initiate a larger discussion that Twitter allows for the > inaka/erlang_standards repository. I hope you don't mind that I do it > here, Github issues is quite terrible for discussions and I feel that > talking about these things with a larger number of people can bring > better feedback. > > I will take all points listed in the README at > https://github.com/inaka/erlang_standards and comment if I have > something (hopefully) interesting to say about it. > > I know they are "Inaka's coding standards" but they are also used for > the Elvis coding standard checker project which would be interesting > to try and integrate as part of more people's workflow (regardless of > projects using the same standards or not, simply checking that > contributions respect it is already quite nice to have). > > An example output can be seen there: > https://gist.github.com/elbrujohalcon/332db8bc343f81235462 > > So let me start now. > > > Spaces over tabs, 2 space indentation. > > Whatever floats your boat. I just want to point out here that 2-4 > space vs tab vs mixed never lead anywhere. My problem with it though > is that the Elvis tool has an option "no_tabs". That's very > opinionated, instead you should have an option that allows you to > define indentation. For example I want to know if people use spaces > instead of tabs in my projects. > > > Surround operators and commas with spaces. > > As much as I like having these with spaces around, I have an issue > with that. I feel that the default should be "whatever Erlang is > doing". For example maps, it does "#{a => b,c => d}". The default your > tool should do is exactly what Erlang does, because you can then just > copy paste everything and not have to worry. We could extend that > logic to doing what erl_pp does but people really like their > tabs/spaces/goats so that'd probably be taking it a bit too far. > > > Try to always separate PRIVATE and PUBLIC functions in groups, with > the public ones first, unless it helps readability and code discovery. > > I have to disagree with that. This is what I have done for a *long > time* but I experimented and changed my mind recently. > > What I do now is for *each* exported function (nice OOP language btw), > start with the exported function, then the internal functions it > calls, and then the unit tests (and performance tests). Then do the > same for the next exported function and so on. It doesn't fit all > kinds of modules (of course!) but for modules that aren't doing > processes stuff (like gen_servers) it's very nice. > > For example: > https://github.com/ninenines/cowlib/blob/master/src/cow_qs.erl - if > that module was doing the old public, private, tests, it would be > pretty much unreadable. Having the unit tests right there with the > code is very nice because you can easily see what the code takes and > returns. > > > Don't design your system using god modules (modules that have a huge > number of functions and/or deal with very unrelated things) > > Well that's fine if your problem domain is simple, but not so much > otherwise. The problem with your approach comes from being able to set > a maximum number of functions project-wide. In Cowboy for example most > modules are small, and cowboy_req is the exception. I know cowboy_req > has a lot of exported functions, I don't want a tool to tell me that. > It's meant to be this way. > > > Don't write the same code in many places, use functions and > variables for that > > I have for a long time been a proponent of applying DRY only when this > code exists more than 2 times, and the code is significant enough. I > care little about repeated code that's 2 lines long for example. I > don't have hard limits there but it would be nice to analyze a few > projects with different settings and find what's the most useful. > > > When having lots of modules, use subdirectories for them, named with > a nice descriptive name for what that "package" does. > > Sounds like you need to separate into different applications. > > > Don't write spaghetti code (A list comprehension with a case inside, > or blocks with begin/end, and nested stuff) > > Not what I call spaghetti code? I'm not even sure what this is about. > List comprehensions with a case or begin/end inside can be immensely > useful. Especially the begin/end ones. > > > Don't use if . > > I would say the opposite. ROK has already argued on that, no need to > duplicate. :-) > > > Macros should be named in ALL_UPPER_CASE: > > I would say "Don't use macros" instead. I would love the tool to tell > me when it sees macros that aren't ?MODULE (and a way to whitelist a > few small ones). > > > Write the -spec's for your public fun's, and for private fun's when > it adds real value for documentation purposes. Define as many types as > needed. > > Honestly I'd remove the comment about internal functions (wow OOP > man), they're generally not very useful, unless Dialyzer complains. > > > Module comments go with %%%, function comments with %%, and code > comments with %. > > Or you could stop worrying and enjoy life. Seriously that's some > nitpicking you got there. :-) > > Instead of saying that you should probably worry that people > *actually* comment their code. > > > Try to write functions with a small number of lines. 12 lines per > function except for integration tests is a good measure. > > 12, seriously? If I'm writing a game the init function or main loop > function will hardly be less than 20 lines. > > > Help dialyzer and xref as much as you can, so that they can work > for you > > Like with most things in life, there's an optimal middle ground. Don't > go out of your way to make tools happy. Tools are supposed to help you > not hinder you. > > > When having many nested "include files", use > -ifndef(HEADER_FILE_HRL) .... -endif so they can be included in any > order without conflicts. > > I'll need an option that complains when -include is used (again, with > whitelist, probably never warn about file.hrl either). :-) > > That's it. Hope to spark some (not too pointless) discussion. > > Enjoy! > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From elbrujohalcon@REDACTED Tue Sep 9 15:19:16 2014 From: elbrujohalcon@REDACTED (Brujo Benavides @ Inaka) Date: Tue, 9 Sep 2014 10:19:16 -0300 Subject: [erlang-questions] Feedback on inaka/erlang_standards In-Reply-To: <540EDC3B.7050604@llaisdy.com> References: <540E23E8.1060500@ninenines.eu> <540EDC3B.7050604@llaisdy.com> Message-ID: <8C1B9C56-7267-42D3-B76F-618049E21841@inaka.net> Hi all, Long mail ahead, fasten your seatbelt ;) Before addressing the individual points in Lo?c?s mail, I want to get a couple of facts about elvis[1] and erlang_standards[2] straight, to eliminate (or alleviate) some doubts about them: First of all, a note on their purposes: 1. Elvis is an open-source configurable and extensible style-checker, intended to be used by any Erlang programmer in the world, therefore its ?standards? (a.k.a. rules) are as flexible as they may be (e.g. the rule is defined as ?nesting_level, [MaxNestingLevel]? so that each dev can pick their favourite MaxNestingLevel) 2. erlang_standards is the set of standards that we (at Inaka) enforce in our Erlang code. Since they reflect _our_ take on style rules, they?re not as flexible as elvis? rules (e.g. the rule says "Try not to nest more than 3 levels deep.?, i.e. MaxNestingLevel = 3 for us). Now, one of our goals when building elvis was for it to be as flexible as possible. To achieve that goal, we provided a couple of ways in which devs can configure it, namely: - if you use elvis as a command line tool, you MUST provide a configuration file (there is an example on the elvis repo [3], but there are NO default values). That way you MUST decide which rules you want to enforce explicitly. - those rules in elvis.config, are just MFAs that all happen to be in the elvis_style module, but can actually be in any module you want (granted, if you want to apply your own rules you need to run elvis in an Erlang shell with access to your modules, but I already opened an issue there to see if we can find a way to handle the same behaviour in the escript version) - if you use elvis as a server integration with github [4] there is a default set of rules, BUT you can override it by simply placing an elvis.config file in the root folder of your repo. Granted: you can?t run rules defined in your own modules there, sorry about that :) You can submit them as pull requests to elvis and, once they?re merged, they?ll soon be available at the service ;) So, in a nutshell: Elvis should fit your needs and your rules, not the ones at Inaka. If you think that?s not the case, and it needs further integration/configuration/simplicity? please open an issue on github or better send us a PR. Now, moving on to erlang_standards. As much as I would love that repo to become a centralised list of standards for the whole erlang community, I would argue that if that?s going to be the case, it shouldn?t be in Inaka?s github, but rather on erlang?s github (https://github.com/erlang/erlang_standards) or something similar (BTW: I?m totally fine giving you ownership of the repo, OTP guys? I would then fork it at Inaka and adapt the rules that we, at the company, want to enforce but are not massively accepted by the community). In the meantime, as I said, it will only reflect the ideas of the ~8 erlangers that were/are working at Inaka. We accept contributions because we love good and constructive debate, but we don?t expect anybody outside our company (well? we don?t even expect everybody INSIDE our company) to like them :) With all that said, and taking off my Inaka?s CTO hat, I?ll address Lo?c?s points with my very personal opinions that in no way are supposed to represent Inaka?s vision (should that even exist): > Spaces over tabs, 2 space indentation. I agree: whatever floats your boat. Elvis has just the no_tabs rule because writing a proper "indentation, [spaces, 2]? rule proved way harder than we originally thought. If somebody out there is talented or resourceful enough to write it, we will LOVE to have it added to elvis. > Surround operators and commas with spaces. Again, I totally agree. But, again, having such a rule as you describe would be awesome? we just wrote a simpler one that is *just good enough*, but can use lots of improvement, for sure. > Try to always separate PRIVATE and PUBLIC functions in groups, with the public ones first, unless it helps readability and code discovery. I agree on the OOP'ism (facepalm). I?ll open an issue to change that :P I also agree that many modules are better structured as you describe, that?s why the "unless it helps readability and code discovery? is there. This rule, as all other very subjective and in-the-eye-of-the-code-reviewer rules, will never reach elvis for that reason exactly. > Don't design your system using god modules (modules that have a huge number of functions and/or deal with very unrelated things) I agree that some modules are the exception to this rule and I reckon that just checking the number of exported functions on them is not exactly a good measure of the fact that they?re god modules or not. For instance, you can have a god module that?s just a gen server with a few exported functions but dozens of handle_call and handle_cast clauses: it will be undetected by elvis, but it should be detected by the code reviewer. So, yeah? the god_modules rule is not actually detecting the existence of god modules, because probably? no rule can. It just detects one kind of big modules and warns you about them so that you can check yourself if it?s in fact a god module or not. In cowboy?s case, you should just remove it from your elvis.config ;) > Don't write the same code in many places, use functions and variables for that I agree completely with what you said. That?s why this is another one of those VERY subjective rules and, as the README states it?s here (instead of being treated as "just a good idea? to let reviewers reject PRs that include the same code several times or PRs that re-implement something that they know it's already done somewhere else > When having lots of modules, use subdirectories for them, named with a nice descriptive name for what that "package" does. I also agree, sometimes those things are better off as applications, but we have apps with cowboy handlers and sumo models that are better placed in ?handlers" and ?models? folders. But yeah? I would add the part about considering moving stuff to different applications to the rule. [5] > Don't write spaghetti code (A list comprehension with a case inside, or blocks with begin/end, and nested stuff) I used to be a big fan of list comprehensions with things inside them as the example [6], which incidentally is based on a code written by me, will show soon, until I had to explain parts of my old code to new devs here. They stared at me for hours while I stared at my old code for the same time :P Marcelo Gornstein (hopefully he?s reading this as well) was one of the victims and, in the process of understanding what I had written, he taught me how to write it more clearly and manageably. That?s where this rule came from. And again? it may never be checked by elvis, because it?s kinda subjective. > Don't use if . I?m a very anti-if kinda guy, and it comes from my years working on OOP languages like SmallTalk. But I also know how this subject may spark lots of flame-wars, so I refrain from discussing it. The three or four last times when I enrolled in a debate about the subject it all boiled down to me dissecting other people?s ifs into functions or cases or other constructs that (at least to my eyes) looked better and more manageable, but I don?t expect anybody to abide to that rule except for Inaka employees in Inaka projects, of course ;) > Macros should be named in ALL_UPPER_CASE I?m with you 100%? I HATE macros, except for ?MODULE, ?MODULE_STRING, ?LINE, ?FILE, etc. But I couldn?t get that far for Inaka?s standards so the compromise was to, at least, write them in all upper case :P > Write the -spec's for your public fun's, and for private fun's when it adds real value for documentation purposes. Define as many types as needed. I agree with you on this one as well, but Inaka?s consensus was to require devs to write specs for functions with too generic names like get_events(Account, Day, Order). ?\_(?)_/? > Module comments go with %%%, function comments with %%, and code comments with %. Yeah, as Ivan said: this rule is here to cope with Emacs users :P (Yeah? I?m not one of them, I?m a happy sublimer :P) Keep in mind that this rule it?s not in the list of actual (i.e. strict) rules, but in the list of ?Suggestions & Great Ideas? which is the list of "nice things to keep in mind but nobody will complain if you don?t?. > Try to write functions with a small number of lines. 12 lines per function except for integration tests is a good measure. If you?re writing an init function or main loop and it has more than 12 lines (that should be ?expressions", BTW [7]), then you probably can find a way to divide that function into a couple of different ones, each one doing a smaller (and more manageable) part of the work. In the same sense as the if?s rule? I wouldn't expect everybody to agree on this one. I know it can spark flame-wars as well, and (in my case, and I was not the only one advocating for this) it also comes from my background as a SmallTalk programmer. This rule is also out of the stricter part, like the one above. > Help dialyzer and xref as much as you can, so that they can work for you Yeah, this one is as vague as it can be. It may be removed from the list moving forward because it?s impossible (even for human reviewers, and not just elvis) to check. That?s why, also, it?s not in the list of actual rules, but in the list of ?Suggestions & Great Ideas?. > When having many nested "include files", use -ifndef(HEADER_FILE_HRL) .... -endif so they can be included in any order without conflicts. ?aaaaaand I agree with you on this one, too. And I think your suggestion is really valuable, please add it as an issue for elvis and, if you like, as a rule suggestion [8] for erlang_standards. So, to sum up: I basically agree with you on almost everything but two very-subjective very-flame-war-inducing rules which I don?t really want to discuss (one of them it?s not even an actual strict rule). I also want everybody to notice how elvis is configurable and how erlang_standards is Inaka?s list and I would LOVE to have a similar one at [2]. All that said, thank you for reading up to this point, I know it was very long email and therefore I leave you with the best animated gif in the history of animated gifs: http://www.reactiongifs.com/r/best-gif-ever1.gif Enjoy! [1] https://github.com/inaka/elvis [2] https://github.com/inaka/erlang_standards [3] https://github.com/inaka/elvis/blob/master/config/elvis.config [4] https://elvis.inakalabs.com [5] https://github.com/inaka/erlang_standards/issues/16 [6] https://github.com/inaka/erlang_standards/blob/elbrujohalcon.13.provide_examples_in__erl_files/src/spaghetti.erl [7] https://github.com/inaka/erlang_standards/issues/17 [8] https://github.com/inaka/erlang_standards/blob/master/CONTRIBUTING.md > On Sep 9, 2014, at 07:53, Ivan Uemlianin wrote: > > I like the idea of a configurable coding standard checker, if that's what elvis is, and as long as team members read and discuss (and agree on) the team's "elvis.config". Discussing and agreeing on standards is as important as following them. Each team will have their own conventions: their own boats to float. > > As for particular points: > > - I tend to avoid if, and macros as far as possible: two fewer bits of syntax to remember. I notice I'm starting to avoid case too. > > > Module comments go with %%%, function comments with %%, and code comments with %. > > This is perhaps following emacs erlang-mode behaviour (which auto-intends comments differently depending on how many %s start the line). > > Discussing coding style is worthwhile. I think a monolithic One True Style is less useful, especially in a programming culture as articulate and reasoned (and occasionally stiff-necked) as erlang's. > > Best wishes > > Ivan > > > On 08/09/2014 22:47, Lo?c Hoguin wrote: >> Hello, >> >> I want to initiate a larger discussion that Twitter allows for the inaka/erlang_standards repository. I hope you don't mind that I do it here, Github issues is quite terrible for discussions and I feel that talking about these things with a larger number of people can bring better feedback. >> >> I will take all points listed in the README at https://github.com/inaka/erlang_standards and comment if I have something (hopefully) interesting to say about it. >> >> I know they are "Inaka's coding standards" but they are also used for the Elvis coding standard checker project which would be interesting to try and integrate as part of more people's workflow (regardless of projects using the same standards or not, simply checking that contributions respect it is already quite nice to have). >> >> An example output can be seen there: https://gist.github.com/elbrujohalcon/332db8bc343f81235462 >> >> So let me start now. >> >> > Spaces over tabs, 2 space indentation. >> >> Whatever floats your boat. I just want to point out here that 2-4 space vs tab vs mixed never lead anywhere. My problem with it though is that the Elvis tool has an option "no_tabs". That's very opinionated, instead you should have an option that allows you to define indentation. For example I want to know if people use spaces instead of tabs in my projects. >> >> > Surround operators and commas with spaces. >> >> As much as I like having these with spaces around, I have an issue with that. I feel that the default should be "whatever Erlang is doing". For example maps, it does "#{a => b,c => d}". The default your tool should do is exactly what Erlang does, because you can then just copy paste everything and not have to worry. We could extend that logic to doing what erl_pp does but people really like their tabs/spaces/goats so that'd probably be taking it a bit too far. >> >> > Try to always separate PRIVATE and PUBLIC functions in groups, with the public ones first, unless it helps readability and code discovery. >> >> I have to disagree with that. This is what I have done for a *long time* but I experimented and changed my mind recently. >> >> What I do now is for *each* exported function (nice OOP language btw), start with the exported function, then the internal functions it calls, and then the unit tests (and performance tests). Then do the same for the next exported function and so on. It doesn't fit all kinds of modules (of course!) but for modules that aren't doing processes stuff (like gen_servers) it's very nice. >> >> For example: https://github.com/ninenines/cowlib/blob/master/src/cow_qs.erl - if that module was doing the old public, private, tests, it would be pretty much unreadable. Having the unit tests right there with the code is very nice because you can easily see what the code takes and returns. >> >> > Don't design your system using god modules (modules that have a huge number of functions and/or deal with very unrelated things) >> >> Well that's fine if your problem domain is simple, but not so much otherwise. The problem with your approach comes from being able to set a maximum number of functions project-wide. In Cowboy for example most modules are small, and cowboy_req is the exception. I know cowboy_req has a lot of exported functions, I don't want a tool to tell me that. It's meant to be this way. >> >> > Don't write the same code in many places, use functions and variables for that >> >> I have for a long time been a proponent of applying DRY only when this code exists more than 2 times, and the code is significant enough. I care little about repeated code that's 2 lines long for example. I don't have hard limits there but it would be nice to analyze a few projects with different settings and find what's the most useful. >> >> > When having lots of modules, use subdirectories for them, named with a nice descriptive name for what that "package" does. >> >> Sounds like you need to separate into different applications. >> >> > Don't write spaghetti code (A list comprehension with a case inside, or blocks with begin/end, and nested stuff) >> >> Not what I call spaghetti code? I'm not even sure what this is about. List comprehensions with a case or begin/end inside can be immensely useful. Especially the begin/end ones. >> >> > Don't use if . >> >> I would say the opposite. ROK has already argued on that, no need to duplicate. :-) >> >> > Macros should be named in ALL_UPPER_CASE: >> >> I would say "Don't use macros" instead. I would love the tool to tell me when it sees macros that aren't ?MODULE (and a way to whitelist a few small ones). >> >> > Write the -spec's for your public fun's, and for private fun's when it adds real value for documentation purposes. Define as many types as needed. >> >> Honestly I'd remove the comment about internal functions (wow OOP man), they're generally not very useful, unless Dialyzer complains. >> >> > Module comments go with %%%, function comments with %%, and code comments with %. >> >> Or you could stop worrying and enjoy life. Seriously that's some nitpicking you got there. :-) >> >> Instead of saying that you should probably worry that people *actually* comment their code. >> >> > Try to write functions with a small number of lines. 12 lines per function except for integration tests is a good measure. >> >> 12, seriously? If I'm writing a game the init function or main loop function will hardly be less than 20 lines. >> >> > Help dialyzer and xref as much as you can, so that they can work for you >> >> Like with most things in life, there's an optimal middle ground. Don't go out of your way to make tools happy. Tools are supposed to help you not hinder you. >> >> > When having many nested "include files", use -ifndef(HEADER_FILE_HRL) .... -endif so they can be included in any order without conflicts. >> >> I'll need an option that complains when -include is used (again, with whitelist, probably never warn about file.hrl either). :-) >> >> That's it. Hope to spark some (not too pointless) discussion. >> >> Enjoy! >> > > -- > ============================================================ > Ivan A. Uemlianin PhD > Llaisdy > Speech Technology Research and Development > > ivan@REDACTED > www.llaisdy.com > llaisdy.wordpress.com > github.com/llaisdy > www.linkedin.com/in/ivanuemlianin > > festina lente > ============================================================ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Tue Sep 9 17:41:18 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 9 Sep 2014 19:41:18 +0400 Subject: [erlang-questions] erl_driver: is char *buf from control safe? Message-ID: I send very big binary to driver: erlang:port_control(Port, 15, YUV) Can I send it's pointer buf to async thread? Is it safe? -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Tue Sep 9 17:49:13 2014 From: garazdawi@REDACTED (Lukas Larsson) Date: Tue, 9 Sep 2014 17:49:13 +0200 Subject: [erlang-questions] erl_driver: is char *buf from control safe? In-Reply-To: References: Message-ID: On Tue, Sep 9, 2014 at 5:41 PM, Max Lapshin wrote: > I send very big binary to driver: > > erlang:port_control(Port, 15, YUV) > > Can I send it's pointer buf to async thread? Is it safe? > No, you want to use port_command together with driver_enq/deq+pdl to do that. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Sep 9 18:25:27 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 9 Sep 2014 20:25:27 +0400 Subject: [erlang-questions] erl_driver: is char *buf from control safe? In-Reply-To: References: Message-ID: will driver_enq copy data? I need to avoid copying at any cost. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Sep 9 18:40:58 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 9 Sep 2014 20:40:58 +0400 Subject: [erlang-questions] erl_driver: is char *buf from control safe? In-Reply-To: References: Message-ID: I need to send binary to driver, do some VERY cpu consumptive job with it (transcoding video) and return back. I was trying to use dirty NIFs, but it seems to be very unstable: after several hours of work whole erlang machine becomes very, very slowly without any CPU usage. Now I try to migrate to erl drivers, but I've got new problem: I don't understand how to get zerocopy from erlang level to C level. Zerocopy of binaries is a must-have requirement, because copying raw video can take up to several gigabits per second inside single node. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Tue Sep 9 18:49:04 2014 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 9 Sep 2014 12:49:04 -0400 Subject: [erlang-questions] erl_driver: is char *buf from control safe? In-Reply-To: References: Message-ID: On Tue, Sep 9, 2014 at 12:40 PM, Max Lapshin wrote: > I need to send binary to driver, do some VERY cpu consumptive job with it > (transcoding video) and return back. > > > I was trying to use dirty NIFs, but it seems to be very unstable: after > several hours of work whole erlang machine becomes very, very slowly > without any CPU usage. > I'm very interested in seeing an example of this. Now I try to migrate to erl drivers, but I've got new problem: I don't > understand how to get zerocopy from erlang level to C level. > Pass your data to the driver via port_command, as Lukas suggested. Implement the outputv callback and call driver_enqv to enqueue the incoming ErlIOVec argument into your driver queue. This won't copy. Also, since async threads are involved for doing the work, use the port data lock to protect the queue, as Lukas suggested. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Sep 9 18:57:25 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 9 Sep 2014 20:57:25 +0400 Subject: [erlang-questions] erl_driver: is char *buf from control safe? In-Reply-To: References: Message-ID: Steve, I can give access for you to one server with problems that I describe. -------------- next part -------------- An HTML attachment was scrubbed... URL: From schneider@REDACTED Tue Sep 9 19:28:27 2014 From: schneider@REDACTED (Frans Schneider) Date: Tue, 09 Sep 2014 19:28:27 +0200 Subject: [erlang-questions] wx_object and OTP design pattern Message-ID: <540F38BB.8010006@xs4all.nl> Hi list, I want to incorporate wx in my application and was wondering how the wx_object behaviour fits into the OTP application structure. Any design patterns for this? All examples or applications using wxErlang I found do not follow the OTP prinicples. Thanks, Frans From max.lapshin@REDACTED Tue Sep 9 21:41:51 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 9 Sep 2014 23:41:51 +0400 Subject: [erlang-questions] erl_driver: is char *buf from control safe? In-Reply-To: References: Message-ID: I'm not sure that I properly understand description or driver_enqv API. I don't need iovector, I need exactly the same binary (same pointer) that I get from decoder. Can driver_deq API guarantee that output binary will not be copied? -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Tue Sep 9 22:33:59 2014 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 9 Sep 2014 22:33:59 +0200 Subject: [erlang-questions] erl_driver: is char *buf from control safe? In-Reply-To: References: Message-ID: On Tue, Sep 9, 2014 at 9:41 PM, Max Lapshin wrote: > Can driver_deq API guarantee that output binary will not be copied? > If the binary is an offheap binary (which I guess your binaries are), then it will most likely never be copied. The function outputv, driver_enqv, driver_peekqv and driver_deq are built specifically to do as little copying as is possible. I can see in the code that there are cases when binaries are copied, but I don't know when and why they might be triggered, I don't think they will be for your usecase. If you have problems figuring out how to use the API, file write in the efile driver is the primary example to look at. Or maybe the test code? https://github.com/erlang/otp/blob/maint/erts/emulator/test/driver_SUITE_data/queue_drv.c -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed Sep 10 04:22:59 2014 From: zxq9@REDACTED (zxq9) Date: Wed, 10 Sep 2014 11:22:59 +0900 Subject: [erlang-questions] wx_object and OTP design pattern In-Reply-To: <540F38BB.8010006@xs4all.nl> References: <540F38BB.8010006@xs4all.nl> Message-ID: <4068834.Syb7rLft49@burrito> On Tuesday 09 September 2014 19:28:27 Frans Schneider wrote: > > I want to incorporate wx in my application and was wondering how the > wx_object behaviour fits into the OTP application structure. Any design > patterns for this? All examples or applications using wxErlang I found > do not follow the OTP prinicples. I've been experimenting with wx quite a bit myself lately. I noticed the same thing looking for open source examples of wx use -- almost no utilization of wx_object. Then I started reading the code that appears when you run wx:demo/0 more carefully. Scroll up to the header (which is not where the module code window starts at by default), and you'll notice the windows (note: be careful what "frame" and "window" mean in wx terminology) in demo all use wx_object. Inspired by a combination of that and the Android-esque idea that one activity equals one frame, I have found wx_object to be a helpful thing. If I can provide an interface to a single user task in a single window, then wx_object is my wx-augmented gen_server that represents that activity. There is a bit of a semantic stew up above ("frame", "task", "window", "activity", etc.) but the basic idea is that wherever you have a chunk of user functionality that is best represented as a process of its own (nearly always the case), it is easier to integrate that using a wx_object than to write a home-made wx process that talks to a GUI handling gen_server that then talks to whatever your logic bits are. It is cumbersome, on the other hand, to start gen_serverizing every aspect of the GUI with wx_objects everywhere. As always, you just have to play around with this a bit on your own to decide what your "correct" level of granularity is there. I'm still toying with this and haven't decided, but Android folks define "activity" and "task" have provided me with reasonable starting points. I am *intensely* interested in anyone else's opinion on this. Using Erlang in this way has vastly simplified my reasoning about GUIs, but it still seems that there are not many examples of wxErlang GUI applications in the wild so I feel like I'm blazing my own trail every day. -Craig From max.lapshin@REDACTED Wed Sep 10 07:01:16 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 10 Sep 2014 09:01:16 +0400 Subject: [erlang-questions] erl_driver: is char *buf from control safe? In-Reply-To: References: Message-ID: https://github.com/erlang/otp/blob/maint/erts/emulator/test/driver_SUITE_data/queue_drv.c#L189 So, API here does not 100% guarantee that I will get the same binary, but promise that perhaps will? Well, I tried the third option: nif with separate thread per each nif. It works, need to check it on production. Thank you again for help. Steve: if you want to see at stuck VM with dirty nifs, write me an private email, please, I'll give you access. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Wed Sep 10 08:24:01 2014 From: dangud@REDACTED (Dan Gudmundsson) Date: Wed, 10 Sep 2014 08:24:01 +0200 Subject: [erlang-questions] wx_object and OTP design pattern In-Reply-To: <4068834.Syb7rLft49@burrito> References: <540F38BB.8010006@xs4all.nl> <4068834.Syb7rLft49@burrito> Message-ID: On Wed, Sep 10, 2014 at 4:22 AM, zxq9 wrote: > On Tuesday 09 September 2014 19:28:27 Frans Schneider wrote: > > > > I want to incorporate wx in my application and was wondering how the > > wx_object behaviour fits into the OTP application structure. Any design > > patterns for this? All examples or applications using wxErlang I found > > do not follow the OTP prinicples. > > The top level window could be an wx_object and incorporated in OTP-application. But generally a terminate signal will be sent when the underlying object is destroyed, so wx/wxWidgets handles this and it would only be troublesmoe to start several wx_objects from supervisor. You typically do not want to restart wx_object from a supervisor if it crashes, it is hard enough to write stable wx applications :-) > I've been experimenting with wx quite a bit myself lately. I noticed the > same > thing looking for open source examples of wx use -- almost no utilization > of > wx_object. Then I started reading the code that appears when you run > wx:demo/0 > more carefully. Scroll up to the header (which is not where the module code > window starts at by default), and you'll notice the windows (note: be > careful > what "frame" and "window" mean in wx terminology) in demo all use > wx_object. > > Inspired by a combination of that and the Android-esque idea that one > activity > equals one frame, I have found wx_object to be a helpful thing. If I can > provide an interface to a single user task in a single window, then > wx_object > is my wx-augmented gen_server that represents that activity. > > There is a bit of a semantic stew up above ("frame", "task", "window", > "activity", etc.) but the basic idea is that wherever you have a chunk of > user > functionality that is best represented as a process of its own (nearly > always > the case), it is easier to integrate that using a wx_object than to write a > home-made wx process that talks to a GUI handling gen_server that then > talks > to whatever your logic bits are. > > It is cumbersome, on the other hand, to start gen_serverizing every aspect > of > the GUI with wx_objects everywhere. As always, you just have to play around > with this a bit on your own to decide what your "correct" level of > granularity > is there. I'm still toying with this and haven't decided, but Android folks > define "activity" and "task" have provided me with reasonable starting > points. > > I am *intensely* interested in anyone else's opinion on this. Using Erlang > in > this way has vastly simplified my reasoning about GUIs, but it still seems > that > there are not many examples of wxErlang GUI applications in the wild so I > feel > like I'm blazing my own trail every day. > > I have also used it to create my own widgets, an example: https://github.com/dgud/wings/blob/dgud/wx-backend/src/ww_color_ctrl.erl and ww_color_slider.erl > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Wed Sep 10 08:57:58 2014 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 09 Sep 2014 23:57:58 -0700 Subject: [erlang-questions] erl_driver: is char *buf from control safe? In-Reply-To: References: Message-ID: <540FF676.3090906@gmail.com> I am pretty sure GEPD has zero-copy working for binaries the way you want it: https://github.com/okeuday/GEPD/ It would probably be similar to the pchar_test1. On 09/09/2014 09:49 AM, Steve Vinoski wrote: > > > > On Tue, Sep 9, 2014 at 12:40 PM, Max Lapshin > wrote: > > I need to send binary to driver, do some VERY cpu consumptive job with it (transcoding video) and return back. > > > I was trying to use dirty NIFs, but it seems to be very unstable: after several hours of work whole erlang machine becomes very, very slowly without any CPU usage. > > > I'm very interested in seeing an example of this. > > Now I try to migrate to erl drivers, but I've got new problem: I don't understand how to get zerocopy from erlang level to C level. > > > Pass your data to the driver via port_command, as Lukas suggested. Implement the outputv callback and call driver_enqv to enqueue the incoming ErlIOVec argument into your driver queue. This won't copy. Also, since async threads are involved for doing the work, use the port data lock to protect the queue, as Lukas suggested. > > --steve > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.shturm@REDACTED Wed Sep 10 08:18:09 2014 From: alex.shturm@REDACTED (Alex Shturm) Date: Tue, 9 Sep 2014 23:18:09 -0700 Subject: [erlang-questions] Problems with prim_file and ports Message-ID: Hello everybody. I'm having a problem with the way prim_file deals with ports; hope you can suggest something. I have a gen_server that handles requests to read files. The volume of requests is quite high, so I try to optimize everything I can. In particular, in the beginning I open a port by calling prim_file:open/1, and then use this port for all subsequent file operations. I found that calling prim_file:read_file/2 (to read the entire file) causes no problems. However calling prim_file:open/3 + prim_file:read/2 + prim_file:close/1 (in order to read a portion of a file) causes the port to close. This can be reproduced like this: Eshell V5.10.4 (abort with ^G) 1> 1> Filename = "/tmp/testfile". "/tmp/testfile" 2> os:cmd("touch /tmp/testfile"). [] 3> {ok, Port} = prim_file:open([binary]). {ok,#Port<0.611>} 4> prim_file:read_file(Port, Filename). {ok,<<>>} 5> prim_file:read_file(Port, Filename). {ok,<<>>} 6> {ok, Fd} = prim_file:open(Port, Filename, [read, binary]). {ok,{file_descriptor,prim_file,{#Port<0.611>,11}}} 7> prim_file:read(Fd, 100). eof 8> prim_file:close(Fd). ok 9> prim_file:read_file(Port, Filename). {error,einval} 10> The reason for this behavior is prim_file:close/1 which in addition to issuing a driver command "file close" also closes the port. This looks counterintuitive to me: prim_file:open/3 did not open the port, so why prim_file:close/1 should close it? Can somebody provide some insight into this issue, and/or a workaround? Thank you, Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.meadows.jonsson@REDACTED Wed Sep 10 10:16:12 2014 From: eric.meadows.jonsson@REDACTED (=?UTF-8?Q?Eric_Meadows=2DJ=C3=B6nsson?=) Date: Wed, 10 Sep 2014 10:16:12 +0200 Subject: [erlang-questions] SSL peer verification in httpc with Mozilla's certificate store Message-ID: We are using httpc with the `{verify, verify_peer}` option for SSL connections. We also provide CA certificates through the `cacertfile` option. The certificate store we are using is from Mozilla [1] where we extract all certificates that been set as trusted for issuing new certificates. Using this set of certificates, when accessing https://s3.amazonaws.com, gives us the following error: 17:03:17.397 [error] SSL: :certify: ssl_handshake.erl:1389:Fatal error: unknown ca Using the same certificate file with curl, python's built-in http client or ruby's http client produces no error and the connection is successful. I believe this happens because the root certificate in amazon's certificate chain is not include the certificate file. The intermediate certificate is included though, so it is trusted. It seems erlang's SSL implementation does not handle this scenario even though most HTTP clients and browsers do. From what I can read about path validation it is recommended to stop validation when a trusted certificate is found in the chain and not continue to the root and check it as well. -- Eric Meadows-J?nsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From schneider@REDACTED Wed Sep 10 13:12:16 2014 From: schneider@REDACTED (Frans Schneider) Date: Wed, 10 Sep 2014 13:12:16 +0200 Subject: [erlang-questions] wx_object and OTP design pattern In-Reply-To: <540F38BB.8010006@xs4all.nl> References: <540F38BB.8010006@xs4all.nl> Message-ID: <54103210.2030900@xs4all.nl> What I have done so far is this: I have an OTP application xxx with the xxx application behaviour, a supervisor and some gen_servers. Based on the code from observer, I created a wx application with a hierarchy of wx_objects in line with functional parts of the 'view' I want to create. @Craig, I fully agree with you that the whole wx_object concept is indeed very nice, a specially since I would like to follow with the MVC design pattern. From xxx_app.erl, I start the wx application before starting xxx's supervisor. The wx application has a application:stop(xxx) in it's terminate callback to stop xxx when the top frame window is closed. To communicate with the 'view' from the rest of the xxx application (model and controller), I guess I should use some kind of pubsub protocol with named processes. Something in line with pureMVC (http://puremvc.org/) without the OOP stuff of course.. Any suggestions or comments are highly appreciated. Frans On 09/09/2014 07:28 PM, Frans Schneider wrote: > Hi list, > > I want to incorporate wx in my application and was wondering how the > wx_object behaviour fits into the OTP application structure. Any > design patterns for this? All examples or applications using wxErlang > I found do not follow the OTP prinicples. > > Thanks, > > Frans From raould@REDACTED Wed Sep 10 23:58:37 2014 From: raould@REDACTED (Raoul Duke) Date: Wed, 10 Sep 2014 14:58:37 -0700 Subject: [erlang-questions] sf bay area usa n2o tutoring? Message-ID: anybody feel like teaching me and whoever else is interested about n2o? :-) From gvidal@REDACTED Thu Sep 11 13:21:39 2014 From: gvidal@REDACTED (German Vidal) Date: Thu, 11 Sep 2014 12:21:39 +0100 Subject: [erlang-questions] PhD position - Computer assisted validation in Erlang References: Message-ID: -------------------------------------------------------------------------- PhD scholarship at MiST, DSIC (Universitat Politecnica de Valencia, Spain) http://users.dsic.upv.es/~gvidal/german/mist Computer assisted validation in Erlang A fully-funded PhD position to start in early 2015. ** DEADLINE FOR APPLICATIONS: SEPTEMBER 24, 2014 ** -------------------------------------------------------------------------- The common goal of the MiST research group is the development of techniques for assisting the construction of reliable software through all phases of its life cycle. We aim at providing a solid formal basis for our developments which would allow us to formally prove the correctness and effectiveness of the techniques. The group keeps a balance between theoretical and practical developments. We have a new doctoral position available to start in early 2015. Candidates should be motivated to successfully complete a full Doctoral degree program. The PhD scholarship will cover a four-year period and is linked to a research project (see below) funded within the National Plan for Research and Development. The successful applicant will receive high level doctoral training in close interaction with her/his direct supervisor and other project researchers. Each scholarship includes a PhD fees exemption, an annual salary of EUR 16422 (divided in 14 payments) and social security entitlements. The possibility to apply for additional funding to cover the cost of international stays of up to three months per year is also associated with each PhD scholarship. Furthermore, travelling costs for attending conferences and schools, computer equipment, etc., is also covered by the group. -------------------------------------------------------------------------- CAVI project We live in a digital society in which the software reliability has become a crucial aspect, both because the risks for human lives that a software bug may cause and also for its associated economic cost. Most current software products use heterogenous technologies and different programming languages that are glued together by means of complex mechanisms. This complexity, together with the fact that most software components are made by third-party developers, makes the software validation tasks a true challenge for software development companies in terms of cost. However, validation tasks are crucial for guaranteeing the quality of the software products. In this project, we aim at advancing the knowledge and technology within the area of software validation at different levels: - Software testing. This is the technology most commonly used by the software industry to prevent errors during the software development. The controlled execution of a number of tests allows one to validate, to some extent, the correctness of the software developed. - Formal verification. The main drawback of the previous technique is that, despite the fact that they can be very helpful to locate the bugs of a program, they cannot be used to ensure that a program is error-free. Therefore, we will also consider formal verification techniques that finitely approximate (by means of static analysis, abstract interpretation, constraints, etc.) the possible executions of a program. Thus, we may obtain a total assurance of the correctness of a program, which makes these techniques specially adequate for validating critical software. - Program debugging. Complementary to the previous methods, we can also find a number of debugging techniques that help the programmer to detect and locate the most common program errors. We plan to apply most of our developments to the functional and concurrent programming language Erlang; see, e.g., http://www.erlang.org/ http://en.wikipedia.org/wiki/Erlang_(programming_language) -------------------------------------------------------------------------- Applications and evaluation criteria Formal applications for the PhD scholarships must be made online, through the framework of the Spanish National Program for Training of Researchers (FPI program): http://tinyurl.com/nmx6auf In particular, in the above URL you can find: - General requirements (basically, you should be already accepted and/or enrolled in a PhD program) - Frequently asked questions - The online application site (you need to sign up first and then add mainly the following documents: copy of your passport-in case you're not spanish-, updated CV, academic certifications of your graduate and master degrees, copy of the acceptance/registration to a PhD programme). Unfortunately, all the available information is only in Spanish. Nevertheless, we will happy to help you with the process, so don't hesitate to ask us for clarifications and help with the online application in case you are interested. Evaluation criteria: - Academic performance and achievements of the candidate - Knowledge on functional and/or logic programming paradigms (including Erlang, Haskell and/or Prolog programming languages). - Willingness to undertake research visits of up to three months in other countries. - Ability to write and interact in English. -------------------------------------------------------------------------- Interested candidates are invited to contact German Vidal via email (gvidal@REDACTED). Please note that the deadline for the application is approaching soon! -------------------------------------------------------------------------- From davidnwelton@REDACTED Thu Sep 11 15:35:04 2014 From: davidnwelton@REDACTED (David Welton) Date: Thu, 11 Sep 2014 15:35:04 +0200 Subject: [erlang-questions] Architectural quandaries Message-ID: Hi, I'm still struggling a bit with some doubts about how to best architect a complex, real-world Erlang application. Sorry for the long and somewhat rambling post, but it helps a bit just to write down what's going on. Dramatis person?: * A hardware system, which consists of some C++ programs managed by Erlang. Without these programs, the machine cannot do the job it was built for. * A tablet, which acts as the user interface, and talks with Erlang via the Erlang protocol and HTTP, where appropriate. * A Postgres database. * Various and sundry other subsystems, such as one for generating reports. * A service application (in the Erlang sense) that can be used to diagnose problems, run scripts on the underlying Linux system, and so on. Anyone who's been around Erlang any length of time knows that the idea is to "let it crash!", though it may strike fear into the hearts of managers listening in on the conversation. Clearly though, the answer has to be more complex than that. For instance, in our case, if the hardware has gone haywire, one approach would be to keep trying to kick off the programs managing it, and as that repeatedly fails, propagate the error up the supervision tree, until the whole node falls over and is restarted by the heartbeat script. By the way: we don't care too much about having tons of uptime or high availability or that sort of thing. Perhaps, when we let it crash, if the code to communicate with the tablet hasn't already been torn down, the system will manage to leave a hurried, scribbled note to the tablet explaining that it's in the process of crashing and will restart. At this point though, the operator at the tablet is going to keep getting these "CRASHING, SORRY, BYE" messages and only interact with the system for a limited time before it all falls down again (which it is likely to, if the problem persists), which sounds like something bound to be frustrating. So, clearly the hardware system should be isolated, so that its thrashing around with an unforeseen problem will at least leave the system available to do things like print reports for data already generated, and interact with the database to retrieve stored data, even if in its degraded state, it can no longer use the hardware to acquire new data. Importantly, the end user in front of the tablet might be able to push some buttons to run diagnostics in order to help determine the problem. I've read some of "JLOUIS' Ramblings" on error kernels: http://jlouisramblings.blogspot.it/2010/11/on-erlang-state-and-crashes.html - a good read, but a bit on the philosophical side. I need to translate the ideas into working Erlang code. Several things that come to mind: * The hardware control Erlang application, which is currently 'permanent', is switched to 'temporary', and some *other* code tries to restart it when it crashes, and reports on that to the tablet interface. It would be necessary to write this code, because it's not something provided by OTP. I get a little bit of a feeling, considering this idea, that "if it's not in OTP, maybe it's not a good idea". * Somewhere, in the hardware code's supervision tree, we put a supervisor that has MaxR and MaxT values like 100000000, 1 (speaking of which, wouldn't it be nicer to just have an atom that like always_restart_never_crash), and the interface code keeps tabs of what's happening and can act as a consequence. * We could employ, in some way, another node. I'm less enthusiastic about this, because we don't need any kind of high availability, and because it would most likely complicate things more than we need. One of the above solutions seems like it ought to be workable. To anyone who's made it to the end: a free beer if you ever happen by Padova, Italy. Thanks! -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From jay@REDACTED Thu Sep 11 16:28:52 2014 From: jay@REDACTED (Jay Nelson) Date: Thu, 11 Sep 2014 07:28:52 -0700 Subject: [erlang-questions] Architectural quandaries Message-ID: <3BB1DF26-F5CA-4F10-BA07-5C8C9766CE35@duomark.com> David Welton wrote: > I'm still struggling a bit with some doubts about how to best > architect a complex, real-world Erlang application. Let it crash serves a purpose when you have complex code, but somewhere at the base you want some sense of stability, especially in a loosely coupled environment. I would focus on a simple core that starts up and connects things. This core should have a very reliable low level signal between server and tablet that it is in one of a few states: started, connecting, disconnecting, running, etc. When all else is going haywire, this core needs to at least indicate to the tablet so you can give feedback to the user. For continuity, some data should be cached on the tablet after connecting / running, so that intermittent failures can be smoothed over in the user?s eyes. Just because the components may be currently unstable or reconnecting doesn?t mean the user experience has to be. Recently I have been confronted with the excess complexity of our systems and have been sketching out new approaches to bring up and down systems in a more controlled manner (and dealing with multi-minute startups in a better way). You should consider sketching out a supervisor hierarchy along the lines of: 1) Rest_for_one root children - this guarantees startup ordering - allows later children to make assumptions 2) Start fast core things early - included base libraries with no real initialization - required ets tables (you might want to cache send to / recv from tablet events) 3) Start an FSM to control startup - use messages to this to start / stop children of other supervisors 4) Rest of the supervisors with no children - if possible one_for_one under a single last rest_for_one supervisor - add start_child / stop_child functions and call these from the FSM Root (rest_for_one) - libs - pre-allocated resources - FSM - Services (one_for_one) - Service 1..N 5) Use a gen_event style mechanism to trigger the #3 FSM Spawning deep supervisor trees and using start_phases makes for a slow startup and worse situation on recovery (start_phases won?t run on recovery). Put all start coordination in the FSM and allow for various startup orderings by allowing service peers rather than strict hierarchies. That way one service can go offline without degrading other services. Use a protocol like UBF between server and tablet so you have a way of wringing out all the protocol issues. Focus on just getting the server rock steady with admin control (up/down services independently) and robust pinging of the tablet with buffered events on both sides. Look at SSE-style data exchange for lightweight continuous comms that startup and shutdown quickly, and use this as a base prior to and below the UBF (which is for more complicated transactions of the application). The base comms should be one-way broadcast in both directions because you can?t know if the other side is listening at all times. Simulate hollow services with receive after Estimated_Compute_Time end and focus on the connection behaviour before you write any application logic. Sounds like fun! jay From eliezer@REDACTED Thu Sep 11 18:07:49 2014 From: eliezer@REDACTED (Eliezer Croitoru) Date: Thu, 11 Sep 2014 19:07:49 +0300 Subject: [erlang-questions] I need a little newcomer help with basic STDIN\OUT code. Message-ID: <5411C8D5.6090402@ngtech.co.il> Hello, My name is Eliezer and I have never used erlang before. I am writing couple helpers for squid proxy server as a practice task. I understood that erlang can provide concurrency options and was curios about writing a small acl helper using it. I have written example helpers in python, ruby, perl and couple other languages and I want to write one in erlang too. Since I am not familiar with the syntax I would like to get a recommendation about a brief tutorial that can help me get this specific task done. The program task is: read each line from STDIN and response accordingly to STDOUT by a procedure\function\method. In ruby, perl and python I have a method to fetch only one line from STDIN which allows me to run a couple things on this line only while the STDIN buffer receives more content. I would like to do the same using erlang. Any help will be appreciated. Thanks, Eliezer From daniel.goertzen@REDACTED Thu Sep 11 20:41:33 2014 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Thu, 11 Sep 2014 13:41:33 -0500 Subject: [erlang-questions] I need a little newcomer help with basic STDIN\OUT code. In-Reply-To: <5411C8D5.6090402@ngtech.co.il> References: <5411C8D5.6090402@ngtech.co.il> Message-ID: The program below will read and then write one line at a time; hopefully it will help you get started. Keep in mind that the task you describe isn't going to touch on concurrency or any of the other great parts of Erlang. Try writing a little proxy server, then you will have fun. :) #!/usr/bin/env escript main(_Params) -> handle(). handle() -> handle(io:get_line([])). handle(eof) -> halt(0); handle(Data) -> io:format("~s", [Data]), handle(). On Thu, Sep 11, 2014 at 11:07 AM, Eliezer Croitoru wrote: > Hello, > > My name is Eliezer and I have never used erlang before. > I am writing couple helpers for squid proxy server as a practice task. > I understood that erlang can provide concurrency options and was curios > about writing a small acl helper using it. > I have written example helpers in python, ruby, perl and couple other > languages and I want to write one in erlang too. > > Since I am not familiar with the syntax I would like to get a > recommendation about a brief tutorial that can help me get this specific > task done. > The program task is: > read each line from STDIN and response accordingly to STDOUT by a > procedure\function\method. > > In ruby, perl and python I have a method to fetch only one line from STDIN > which allows me to run a couple things on this line only while the STDIN > buffer receives more content. > > I would like to do the same using erlang. > > Any help will be appreciated. > > Thanks, > Eliezer > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Fri Sep 12 10:28:03 2014 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 12 Sep 2014 10:28:03 +0200 Subject: [erlang-questions] SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: References: Message-ID: Hi! When it comes to when to stop the path validation there is no clear answer that will always cut it and TLS, X509 RFC are not always in total agreement. Even the same RFC is sometimes contradicting itself. The issue you are describing has been addressed in the latest version of the ssl application. See commit 1c9e0651c4917b63f49d8505dba7e820da8e32d2, where I added a new option partial_chain that lets the user decide which certificate in the chain that shall be considered the trusted anchor if the whole chain can not be validated. Regards Ingela Erlang/OTP team - Ericsson AB 2014-09-10 10:16 GMT+02:00 Eric Meadows-J?nsson < eric.meadows.jonsson@REDACTED>: > We are using httpc with the `{verify, verify_peer}` option for SSL > connections. We also provide CA certificates through the `cacertfile` > option. The certificate store we are using is from Mozilla [1] where we > extract all certificates that been set as trusted for issuing new > certificates. > > Using this set of certificates, when accessing https://s3.amazonaws.com, > gives us the following error: > > 17:03:17.397 [error] SSL: :certify: ssl_handshake.erl:1389:Fatal > error: unknown ca > > Using the same certificate file with curl, python's built-in http client > or ruby's http client produces no error and the connection is successful. I > believe this happens because the root certificate in amazon's certificate > chain is not include the certificate file. The intermediate certificate is > included though, so it is trusted. It seems erlang's SSL implementation > does not handle this scenario even though most HTTP clients and browsers > do. From what I can read about path validation it is recommended to stop > validation when a trusted certificate is found in the chain and not > continue to the root and check it as well. > > -- > Eric Meadows-J?nsson > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Fri Sep 12 11:45:25 2014 From: sdl.web@REDACTED (Leo Liu) Date: Fri, 12 Sep 2014 17:45:25 +0800 Subject: [erlang-questions] why doesn't this dbg example work? Message-ID: Hi there, In dbg's reference manual, there is this example, which I cannot replicate: (x@REDACTED)4> dbg:tp(lists,sort,cx). {ok,[{matched,nonode@REDACTED,2},{saved,cx}]} (x@REDACTED)4> lists:sort([2,1]). (<0.32.0>) call lists:sort([2,1]) ({erl_eval,do_apply,5}) (<0.32.0>) returned from lists:sort/1 -> [1,2] [1,2] My shell session: (devel@REDACTED)1> dbg:tracer(). {ok,<0.2100.0>} (devel@REDACTED)2> dbg:tp(lists, sort, cx). {ok,[{matched,devel@REDACTED,2},{saved,cx}]} (devel@REDACTED)3> lists:sort([2,1]). [1,2] Note there are no trace outputs in my shell session. Ideas? Thanks, Leo From vladdu55@REDACTED Fri Sep 12 11:52:40 2014 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 12 Sep 2014 11:52:40 +0200 Subject: [erlang-questions] why doesn't this dbg example work? In-Reply-To: References: Message-ID: Hi! You have to specify first which processes are going to be traced, and how. dbg:p(self(), [c]), fpr example. best regards, Vlad On Fri, Sep 12, 2014 at 11:45 AM, Leo Liu wrote: > Hi there, > > In dbg's reference manual, there is this example, which I cannot > replicate: > > (x@REDACTED)4> dbg:tp(lists,sort,cx). > {ok,[{matched,nonode@REDACTED,2},{saved,cx}]} > (x@REDACTED)4> lists:sort([2,1]). > (<0.32.0>) call lists:sort([2,1]) ({erl_eval,do_apply,5}) > (<0.32.0>) returned from lists:sort/1 -> [1,2] > [1,2] > > My shell session: > > (devel@REDACTED)1> dbg:tracer(). > {ok,<0.2100.0>} > (devel@REDACTED)2> dbg:tp(lists, sort, cx). > {ok,[{matched,devel@REDACTED,2},{saved,cx}]} > (devel@REDACTED)3> lists:sort([2,1]). > [1,2] > > Note there are no trace outputs in my shell session. Ideas? > > Thanks, > Leo > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.meadows.jonsson@REDACTED Fri Sep 12 15:53:00 2014 From: eric.meadows.jonsson@REDACTED (=?UTF-8?Q?Eric_Meadows=2DJ=C3=B6nsson?=) Date: Fri, 12 Sep 2014 15:53:00 +0200 Subject: [erlang-questions] SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: References: Message-ID: Great, that looks like exactly what we need! Thanks Ingela. I found the commit on the OTP repo, but could not find the branch it belonged to. Do you know when when it will be available and which release it will be part of? On Fri, Sep 12, 2014 at 10:28 AM, Ingela Andin wrote: > Hi! > > When it comes to when to stop the path validation there is no clear answer > that will always cut it and TLS, X509 RFC are not always in total > agreement. Even the same > RFC is sometimes contradicting itself. The issue you are describing has > been addressed in the latest version of the ssl application. > > See commit 1c9e0651c4917b63f49d8505dba7e820da8e32d2, where I added a new > option partial_chain that lets the user decide which certificate in the > chain that shall be > considered the trusted anchor if the whole chain can not be validated. > > Regards Ingela Erlang/OTP team - Ericsson AB > > > 2014-09-10 10:16 GMT+02:00 Eric Meadows-J?nsson < > eric.meadows.jonsson@REDACTED>: > >> We are using httpc with the `{verify, verify_peer}` option for SSL >> connections. We also provide CA certificates through the `cacertfile` >> option. The certificate store we are using is from Mozilla [1] where we >> extract all certificates that been set as trusted for issuing new >> certificates. >> >> Using this set of certificates, when accessing https://s3.amazonaws.com, >> gives us the following error: >> >> 17:03:17.397 [error] SSL: :certify: ssl_handshake.erl:1389:Fatal >> error: unknown ca >> >> Using the same certificate file with curl, python's built-in http client >> or ruby's http client produces no error and the connection is successful. I >> believe this happens because the root certificate in amazon's certificate >> chain is not include the certificate file. The intermediate certificate is >> included though, so it is trusted. It seems erlang's SSL implementation >> does not handle this scenario even though most HTTP clients and browsers >> do. From what I can read about path validation it is recommended to stop >> validation when a trusted certificate is found in the chain and not >> continue to the root and check it as well. >> >> -- >> Eric Meadows-J?nsson >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- Eric Meadows-J?nsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From torbjorn.lager@REDACTED Fri Sep 12 15:22:24 2014 From: torbjorn.lager@REDACTED (=?UTF-8?Q?Torbj=C3=B6rn_Lager?=) Date: Fri, 12 Sep 2014 15:22:24 +0200 Subject: [erlang-questions] The implementation of link/1 and monitor/2 Message-ID: Hi there, I have read about links and monitors in Erlang and think I'm beginning to understand how they work. I'm a bit curious how they are implemented though. 1. It seems to me that for a link between two processes P1 and P2, two OTHER processes are needed, one watching over P1 and one watching over P2. Is this how Erlang implements links? 2. It seems to me that a monitor is also a kind of process. Is it a regular Erlang process of the kind you can spawn yourself? 3. Related: If Erlang didn't have link/1 and monitor/2 as BIFs, could they be implemented in Erlang, or are they true language primitives? 4. Related: If Erlang had only monitor/2, could link/1 somehow be implemented in terms of it? Thanks, Torbj?rn -------------- next part -------------- An HTML attachment was scrubbed... URL: From eliezer@REDACTED Fri Sep 12 17:16:24 2014 From: eliezer@REDACTED (Eliezer Croitoru) Date: Fri, 12 Sep 2014 18:16:24 +0300 Subject: [erlang-questions] I need a little newcomer help with basic STDIN\OUT code. In-Reply-To: References: <5411C8D5.6090402@ngtech.co.il> Message-ID: <54130E48.3030001@ngtech.co.il> Thanks Daniel, I have tried to write a tiny proxy but in ruby and I did managed to write something but not as "concurrent" as I want. I am unable to understand what each step does and can I add functionality. I will try a basic tutorial to understand it first. Thanks, Eliezer On 09/11/2014 09:41 PM, Daniel Goertzen wrote: > The program below will read and then write one line at a time; hopefully > it will help you get started. Keep in mind that the task you describe > isn't going to touch on concurrency or any of the other great parts of > Erlang. Try writing a little proxy server, then you will have fun. :) > > > #!/usr/bin/env escript > > main(_Params) -> > handle(). > > handle() -> > handle(io:get_line([])). > > handle(eof) -> > halt(0); > > handle(Data) -> > io:format("~s", [Data]), > handle(). > > > On Thu, Sep 11, 2014 at 11:07 AM, Eliezer Croitoru > wrote: > > Hello, > > My name is Eliezer and I have never used erlang before. > I am writing couple helpers for squid proxy server as a practice task. > I understood that erlang can provide concurrency options and was > curios about writing a small acl helper using it. > I have written example helpers in python, ruby, perl and couple > other languages and I want to write one in erlang too. > > Since I am not familiar with the syntax I would like to get a > recommendation about a brief tutorial that can help me get this > specific task done. > The program task is: > read each line from STDIN and response accordingly to STDOUT by a > procedure\function\method. > > In ruby, perl and python I have a method to fetch only one line from > STDIN which allows me to run a couple things on this line only while > the STDIN buffer receives more content. > > I would like to do the same using erlang. > > Any help will be appreciated. > > Thanks, > Eliezer > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/__listinfo/erlang-questions > > > From rvirding@REDACTED Sat Sep 13 02:02:46 2014 From: rvirding@REDACTED (Robert Virding) Date: Sat, 13 Sep 2014 02:02:46 +0200 Subject: [erlang-questions] Erlounge in San Diego Message-ID: I will be in San Diego 21 - 28 sept. Anybody interested in an informal erlounge during the week? Robert -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Sat Sep 13 02:14:42 2014 From: sdl.web@REDACTED (Leo Liu) Date: Sat, 13 Sep 2014 08:14:42 +0800 Subject: [erlang-questions] why doesn't this dbg example work? In-Reply-To: (Vlad Dumitrescu's message of "Fri, 12 Sep 2014 11:52:40 +0200") References: Message-ID: On 2014-09-12 11:52 +0200, Vlad Dumitrescu wrote: > You have to specify first which processes are going to be traced, and how. > dbg:p(self(), [c]), fpr example. > > best regards, > Vlad Thanks for the info. Leo From barcojie@REDACTED Sat Sep 13 04:19:14 2014 From: barcojie@REDACTED (Barco You) Date: Sat, 13 Sep 2014 10:19:14 +0800 Subject: [erlang-questions] Erlounge in San Diego In-Reply-To: References: Message-ID: If anyone in Shenzhen please let me know:) On Sep 13, 2014 8:03 AM, "Robert Virding" wrote: > I will be in San Diego 21 - 28 sept. Anybody interested in an informal > erlounge during the week? > > Robert > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Sat Sep 13 11:09:10 2014 From: ingela.andin@REDACTED (Ingela Andin) Date: Sat, 13 Sep 2014 11:09:10 +0200 Subject: [erlang-questions] Fwd: SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: Ingela Andin Date: 2014-09-12 16:17 GMT+02:00 Subject: Re: [erlang-questions] SSL peer verification in httpc with Mozilla's certificate store To: Eric Meadows-J?nsson Hi! It will be in 17.3, that most likely will be released next week on wensday. Regards Ingela Erlang/OTP team - Ericsson AB 2014-09-12 15:53 GMT+02:00 Eric Meadows-J?nsson < eric.meadows.jonsson@REDACTED>: > Great, that looks like exactly what we need! Thanks Ingela. > > I found the commit on the OTP repo, but could not find the branch it > belonged to. Do you know when when it will be available and which release > it will be part of? > > On Fri, Sep 12, 2014 at 10:28 AM, Ingela Andin > wrote: > >> Hi! >> >> When it comes to when to stop the path validation there is no clear >> answer that will always cut it and TLS, X509 RFC are not always in total >> agreement. Even the same >> RFC is sometimes contradicting itself. The issue you are describing has >> been addressed in the latest version of the ssl application. >> >> See commit 1c9e0651c4917b63f49d8505dba7e820da8e32d2, where I added a new >> option partial_chain that lets the user decide which certificate in the >> chain that shall be >> considered the trusted anchor if the whole chain can not be validated. >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> >> 2014-09-10 10:16 GMT+02:00 Eric Meadows-J?nsson < >> eric.meadows.jonsson@REDACTED>: >> >>> We are using httpc with the `{verify, verify_peer}` option for SSL >>> connections. We also provide CA certificates through the `cacertfile` >>> option. The certificate store we are using is from Mozilla [1] where we >>> extract all certificates that been set as trusted for issuing new >>> certificates. >>> >>> Using this set of certificates, when accessing https://s3.amazonaws.com, >>> gives us the following error: >>> >>> 17:03:17.397 [error] SSL: :certify: ssl_handshake.erl:1389:Fatal >>> error: unknown ca >>> >>> Using the same certificate file with curl, python's built-in http client >>> or ruby's http client produces no error and the connection is successful. I >>> believe this happens because the root certificate in amazon's certificate >>> chain is not include the certificate file. The intermediate certificate is >>> included though, so it is trusted. It seems erlang's SSL implementation >>> does not handle this scenario even though most HTTP clients and browsers >>> do. From what I can read about path validation it is recommended to stop >>> validation when a trusted certificate is found in the chain and not >>> continue to the root and check it as well. >>> >>> -- >>> Eric Meadows-J?nsson >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > > -- > Eric Meadows-J?nsson > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Sat Sep 13 17:56:42 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 13 Sep 2014 08:56:42 -0700 Subject: [erlang-questions] Fwd: SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: References: Message-ID: <20140913155641.GA1653@ferdair.local> On 09/13, Ingela Andin wrote: > It will be in 17.3, that most likely will be released next week on wensday. > > Regards Ingela Erlang/OTP team - Ericsson AB > Did the Erlang/OTP team forget to officially announce 17.2 or is that a typo? From tuncer.ayaz@REDACTED Sat Sep 13 18:47:00 2014 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 13 Sep 2014 18:47:00 +0200 Subject: [erlang-questions] Fwd: SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: <20140913155641.GA1653@ferdair.local> References: <20140913155641.GA1653@ferdair.local> Message-ID: On Sat, Sep 13, 2014 at 5:56 PM, Fred Hebert wrote: > On 09/13, Ingela Andin wrote: > > It will be in 17.3, that most likely will be released next week on > > wensday. > > > > Regards Ingela Erlang/OTP team - Ericsson AB > > > > Did the Erlang/OTP team forget to officially announce 17.2 or is > that a typo? It's not a typo and we're at 17.2.2 right now. Maybe summer vacation was the reason for the missing 17.2 release notes. From michael.s.klishin@REDACTED Sat Sep 13 20:09:52 2014 From: michael.s.klishin@REDACTED (Michael Klishin) Date: Sat, 13 Sep 2014 22:09:52 +0400 Subject: [erlang-questions] Fwd: SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: References: <20140913155641.GA1653@ferdair.local> Message-ID: On 13 September 2014 at 20:47:53, Tuncer Ayaz (tuncer.ayaz@REDACTED) wrote: > It's not a typo and we're at 17.2.2 right now. Maybe summer vacation > was the reason for the missing 17.2 release notes. 17.2 is also missing from the downloads page: http://www.erlang.org/download.html? -- @michaelklishin, github.com/michaelklishin From andreas@REDACTED Sat Sep 13 20:46:20 2014 From: andreas@REDACTED (Andreas Schumacher) Date: Sat, 13 Sep 2014 20:46:20 +0200 Subject: [erlang-questions] Fwd: Fwd: SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: <314D10C4-C1B3-4DE0-A195-8B659B4065A9@ericsson.com> References: <314D10C4-C1B3-4DE0-A195-8B659B4065A9@ericsson.com> Message-ID: OTP 17.2, as well as 17.0.1, 17.0.2, 17.1.1, 17.1.2, 17.2.1, and 17.2.2 have been internal patches; and thus, they have not been delivered as open source service packages, which we release roughly once a quarter. The description of the contents of those patches is included in the README of the subsequent service package or major release, respectively. In releases prior to OTP 17, those internal patches had internal identifiers, which were not visible in the public Github repository. That was changed with the adaptation of the new versioning scheme. The reason for neither announcing nor making those patches publicly available on erlang.org is mainly to avoid administrative overhead; although, we may reconsider that in the future. Andreas Schumacher, Erlang/OTP, Ericsson AB *From: *Michael Klishin *Subject: * *Re: [erlang-questions] Fwd: SSL peer verification in httpc with Mozilla's certificate store* *Date: *13 Sep 2014 20:09:52 GMT+2 *To: *Tuncer Ayaz , Fred Hebert *Cc: *Erlang-Questions Questions On 13 September 2014 at 20:47:53, Tuncer Ayaz (tuncer.ayaz@REDACTED) wrote: It's not a typo and we're at 17.2.2 right now. Maybe summer vacation was the reason for the missing 17.2 release notes. 17.2 is also missing from the downloads page: http://www.erlang.org/download.html -- @michaelklishin, github.com/michaelklishin _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Sun Sep 14 00:02:06 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 13 Sep 2014 18:02:06 -0400 Subject: [erlang-questions] Fwd: Fwd: SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: References: <314D10C4-C1B3-4DE0-A195-8B659B4065A9@ericsson.com> Message-ID: <20140913220205.GC1653@ferdair.local> On 09/13, Andreas Schumacher wrote: > OTP 17.2, as well as 17.0.1, 17.0.2, 17.1.1, 17.1.2, 17.2.1, and 17.2.2 > have been internal patches; and thus, they have not been delivered as open > source service packages, which we release roughly once a quarter. The > description of the contents of those patches is included in the README of > the subsequent service package or major release, respectively. > > In releases prior to OTP 17, those internal patches had internal > identifiers, which were not visible in the public Github repository. That > was changed with the adaptation of the new versioning scheme. The reason > for neither announcing nor making those patches publicly available on > erlang.org is mainly to avoid administrative overhead; although, we may > reconsider that in the future. Ah but according to prior documentation: http://erlang.org/pipermail/erlang-questions/2014-February/077037.html > The current plan is to release 17.0 in March and up > to three additional OTP 17-based minor releases this year. When OTP 18.0 > will be released has not been decided, yet. And then: http://www.erlang.org/doc/system_principles/versions.html > In the normal case, a version will be constructed as > .. where is the most significant part. > However, more dot separated parts than this may exist. So is this to say that the 3 minor releases could as well be 17.1, 17.3, 17.199 ? Is there any regularity we can expect in version numbers, or we just won't really be able to know? That would be interesting to be aware of when explaining to people how Erlang switched from RXXBXX to a semver-looking-but-not-semver release system that also skips apparent version numbers for the general public, or less sarcastically, when planning to do things such as validate what release numbers exist or not. Regards, Fred. From t@REDACTED Sat Sep 13 21:32:49 2014 From: t@REDACTED (Tristan Sloughter) Date: Sat, 13 Sep 2014 14:32:49 -0500 Subject: [erlang-questions] Fwd: Fwd: SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: References: <314D10C4-C1B3-4DE0-A195-8B659B4065A9@ericsson.com> Message-ID: <1410636769.2265370.167128165.71538F05@webmail.messagingengine.com> Wait, so 17.2 is internal but 17.3 is external? -- Tristan Sloughter t@REDACTED On Sat, Sep 13, 2014, at 01:46 PM, Andreas Schumacher wrote: OTP 17.2, as well as 17.0.1, 17.0.2, 17.1.1, 17.1.2, 17.2.1, and 17.2.2 have been internal patches; and thus, they have not been delivered as open source service packages, which we release roughly once a quarter. The description of the contents of those patches is included in the README of the subsequent service package or major release, respectively. In releases prior to OTP 17, those internal patches had internal identifiers, which were not visible in the public Github repository. That was changed with the adaptation of the new versioning scheme. The reason for neither announcing nor making those patches publicly available on [1]erlang.org is mainly to avoid administrative overhead; although, we may reconsider that in the future. Andreas Schumacher, Erlang/OTP, Ericsson AB From: Michael Klishin <[2]michael.s.klishin@REDACTED> Subject: Re: [erlang-questions] Fwd: SSL peer verification in httpc with Mozilla's certificate store Date: 13 Sep 2014 20:09:52 GMT+2 To: Tuncer Ayaz <[3]tuncer.ayaz@REDACTED>, Fred Hebert <[4]mononcqc@REDACTED> Cc: Erlang-Questions Questions <[5]erlang-questions@REDACTED> On 13 September 2014 at 20:47:53, Tuncer Ayaz ([6]tuncer.ayaz@REDACTED) wrote: It's not a typo and we're at 17.2.2 right now. Maybe summer vacation was the reason for the missing 17.2 release notes. 17.2 is also missing from the downloads page: [7]http://www.erlang.org/download.html -- @michaelklishin, [8]github.com/michaelklishin _______________________________________________ erlang-questions mailing list [9]erlang-questions@REDACTED [10]http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list [11]erlang-questions@REDACTED [12]http://erlang.org/mailman/listinfo/erlang-questions References 1. http://erlang.org/ 2. mailto:michael.s.klishin@REDACTED 3. mailto:tuncer.ayaz@REDACTED 4. mailto:mononcqc@REDACTED 5. mailto:erlang-questions@REDACTED 6. mailto:tuncer.ayaz@REDACTED 7. http://www.erlang.org/download.html 8. http://github.com/michaelklishin 9. mailto:erlang-questions@REDACTED 10. http://erlang.org/mailman/listinfo/erlang-questions 11. mailto:erlang-questions@REDACTED 12. http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dgud@REDACTED Sun Sep 14 01:27:44 2014 From: dgud@REDACTED (Dan Gudmundsson) Date: Sun, 14 Sep 2014 01:27:44 +0200 Subject: [erlang-questions] Fwd: Fwd: SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: <1410636769.2265370.167128165.71538F05@webmail.messagingengine.com> References: <314D10C4-C1B3-4DE0-A195-8B659B4065A9@ericsson.com> <1410636769.2265370.167128165.71538F05@webmail.messagingengine.com> Message-ID: On Sat, Sep 13, 2014 at 9:32 PM, Tristan Sloughter wrote: > Wait, so 17.2 is internal but 17.3 is external? > On Sun, Sep 14, 2014 at 12:02 AM, Fred Hebert wrote: > > So is this to say that the 3 minor releases could as well be 17.1, 17.3, > 17.199 ? Is there any regularity we can expect in version numbers, or we > just won't really be able to know? > > That would be interesting to be aware of when explaining to people how > Erlang switched from RXXBXX to a semver-looking-but-not-semver release > system that also skips apparent version numbers for the general public, > or less sarcastically, when planning to do things such as validate what > release numbers exist or not. > Everything is pretty external and available to the general public, everything is pushed and tagged on github. It is just that we don't build a windows release and src tarball for every patch to erlang.org. Which is better than it was before. Patches are now available (to most of you) sooner than ever before. sigh.. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Sun Sep 14 03:29:18 2014 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Sun, 14 Sep 2014 03:29:18 +0200 Subject: [erlang-questions] Fwd: Fwd: SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: References: <314D10C4-C1B3-4DE0-A195-8B659B4065A9@ericsson.com> <1410636769.2265370.167128165.71538F05@webmail.messagingengine.com> Message-ID: 2014-09-14 1:27 GMT+02:00 Dan Gudmundsson : > On Sat, Sep 13, 2014 at 9:32 PM, Tristan Sloughter > wrote: > >> Wait, so 17.2 is internal but 17.3 is external? >> > > On Sun, Sep 14, 2014 at 12:02 AM, Fred Hebert wrote: > >> >> So is this to say that the 3 minor releases could as well be 17.1, 17.3, >> 17.199 ? Is there any regularity we can expect in version numbers, or we >> just won't really be able to know? >> > The error here was to use numbered services releases and also tie them to a system version. Actually there are two errors. System versions shouldn't so closely resemble semvers since they might cause confusion. The reasoning behind it has been laid out in previous mail discussions. I view a system version, i.e. 17.0, 17.2, 17.2.2, etc, as a set of application versions,much like versions of OS distributions with preinstalled programs. Any pattern in the system version naming should be viewed by you as purely coincidental .. Perhaps it would be better to name service releases like Erlang/OTP - 2014 September. Or just name the releases after random authors or book titles to keep you guessing =) -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Sun Sep 14 04:00:11 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 13 Sep 2014 22:00:11 -0400 Subject: [erlang-questions] Fwd: Fwd: SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: References: <314D10C4-C1B3-4DE0-A195-8B659B4065A9@ericsson.com> <1410636769.2265370.167128165.71538F05@webmail.messagingengine.com> Message-ID: <20140914020009.GD1653@ferdair.local> For me the surprise was partly that I expected only patch releases to be non-announced, and that feature releases would keep being quarterly -- i.e. what was changed was not the release frequency, but the versioning. What actually changed was the release frequency, which was augmented, and the announcement frequency remained the same (quarterly). Now the patch releases are certainly more useful for anyone who has a bug that needs fixing, I won't deny that. What surprised me is that I now need to be a lot more attentive to the github tags being applied if I want to keep being able to support repositories that tend to follow along with Erlang releases: erlang-history for one, or things like buildpacks for Heroku, which let people choose the runtime under which they deploy their applications. I'm guessing the choice is whether to support doing things only for quarterly anouncements, or also on patch-level releases. What does the OTP team feel should be done by external maintainers? If I'm maintaining things that depend on your release cycle, I don't want (nor need) to dictate how to do it, but I want to be able to adjust myself as best as possible with what was intended by the team. Regards, Fred. On 09/14, Bj?rn-Egil Dahlberg wrote: > 2014-09-14 1:27 GMT+02:00 Dan Gudmundsson : > > > On Sat, Sep 13, 2014 at 9:32 PM, Tristan Sloughter > > wrote: > > > >> Wait, so 17.2 is internal but 17.3 is external? > >> > > > > On Sun, Sep 14, 2014 at 12:02 AM, Fred Hebert wrote: > > > >> > >> So is this to say that the 3 minor releases could as well be 17.1, 17.3, > >> 17.199 ? Is there any regularity we can expect in version numbers, or we > >> just won't really be able to know? > >> > > > The error here was to use numbered services releases and also tie them to a > system version. > > Actually there are two errors. System versions shouldn't so closely > resemble semvers since they > might cause confusion. The reasoning behind it has been laid out in > previous mail discussions. > > I view a system version, i.e. 17.0, 17.2, 17.2.2, etc, as a set of > application versions,much like > versions of OS distributions with preinstalled programs. Any pattern > in the system version naming should be viewed by you as purely coincidental > .. > > Perhaps it would be better to name service releases like Erlang/OTP - 2014 > September. > > Or just name the releases after random authors or book titles to keep you > guessing =) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andreas@REDACTED Sun Sep 14 17:17:53 2014 From: andreas@REDACTED (Andreas Schumacher) Date: Sun, 14 Sep 2014 17:17:53 +0200 Subject: [erlang-questions] Fwd: FW: Fwd: Fwd: SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: References: <314D10C4-C1B3-4DE0-A195-8B659B4065A9@ericsson.com> <1410636769.2265370.167128165.71538F05@webmail.messagingengine.com> <20140914020009.GD1653@ferdair.local> Message-ID: External maintainers can ignore the versions that are not officially announced as a major release or service package, and for which neither source/binary packages nor documentation are published on erlang.org. Nevertheless, in order to increase transparency and to avoid similar confusion and discussions in the future, we could announce those patch packages on erlang-questions. Then external maintainers can decide themselves whether they want to include those packages into their products. In any case, the next quarterly service packages are OTP 17.3 on September 17 and another service package (probably OTP 17.4) on December 10. Andreas On 14/09/14 04:00, "Fred Hebert" wrote: >For me the surprise was partly that I expected only patch releases to be >non-announced, and that feature releases would keep being quarterly -- >i.e. what was changed was not the release frequency, but the versioning. > >What actually changed was the release frequency, which was augmented, >and the announcement frequency remained the same (quarterly). > >Now the patch releases are certainly more useful for anyone who has a >bug that needs fixing, I won't deny that. > >What surprised me is that I now need to be a lot more attentive to the >github tags being applied if I want to keep being able to support >repositories that tend to follow along with Erlang releases: >erlang-history for one, or things like buildpacks for Heroku, which let >people choose the runtime under which they deploy their applications. > >I'm guessing the choice is whether to support doing things only for >quarterly anouncements, or also on patch-level releases. > >What does the OTP team feel should be done by external maintainers? If >I'm maintaining things that depend on your release cycle, I don't want >(nor need) to dictate how to do it, but I want to be able to adjust >myself as best as possible with what was intended by the team. > >Regards, >Fred. > >On 09/14, Bj?rn-Egil Dahlberg wrote: >> 2014-09-14 1:27 GMT+02:00 Dan Gudmundsson : >> >> > On Sat, Sep 13, 2014 at 9:32 PM, Tristan Sloughter >> > wrote: >> > >> >> Wait, so 17.2 is internal but 17.3 is external? >> >> >> > >> > On Sun, Sep 14, 2014 at 12:02 AM, Fred Hebert >>wrote: >> > >> >> >> >> So is this to say that the 3 minor releases could as well be 17.1, >>17.3, >> >> 17.199 ? Is there any regularity we can expect in version numbers, >>or we >> >> just won't really be able to know? >> >> >> > >> The error here was to use numbered services releases and also tie them >>to a >> system version. >> >> Actually there are two errors. System versions shouldn't so closely >> resemble semvers since they >> might cause confusion. The reasoning behind it has been laid out in >> previous mail discussions. >> >> I view a system version, i.e. 17.0, 17.2, 17.2.2, etc, as a set of >> application versions,much like >> versions of OS distributions with preinstalled programs. Any pattern >> in the system version naming should be viewed by you as purely >>coincidental >> .. >> >> Perhaps it would be better to name service releases like Erlang/OTP - >>2014 >> September. >> >> Or just name the releases after random authors or book titles to keep >>you >> guessing =) > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Sun Sep 14 20:28:50 2014 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 14 Sep 2014 20:28:50 +0200 Subject: [erlang-questions] Fwd: FW: Fwd: Fwd: SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: References: <314D10C4-C1B3-4DE0-A195-8B659B4065A9@ericsson.com> <1410636769.2265370.167128165.71538F05@webmail.messagingengine.com> <20140914020009.GD1653@ferdair.local> Message-ID: On Sun, Sep 14, 2014 at 5:17 PM, Andreas Schumacher wrote: > External maintainers can ignore the versions that are not officially > announced as a major release or service package, and for which > neither source/binary packages nor documentation are published on > erlang.org. Nevertheless, in order to increase transparency and to > avoid similar confusion and discussions in the future, we could > announce those patch packages on erlang-questions. Then external > maintainers can decide themselves whether they want to include those > packages into their products. I don't mind tag-only releases, and actually _appreciate_ it very much, so thanks a lot! The confusion is, from what I can tell, based on the fact that 17.1 was announced but 17.2 was not. If commit messages and git history is maintained in a descriptive and consistent state, you could automate release notes with links to the relevant commits. But right now there are somtimes intermediate and other commits you'd have to filter from auto-generated release notes. For example, try 'git shortlog OTP-17.1.2...OTP-17.2.2', and you'll find 'wip' commits that ideally would have been amended or squashed. That said, does the OTP team backport fixes to old releases? If they do, such tags (without an official release) can be useful for various production nodes. From eranga.erl@REDACTED Mon Sep 15 09:44:18 2014 From: eranga.erl@REDACTED (Eranga Udesh) Date: Mon, 15 Sep 2014 13:14:18 +0530 Subject: [erlang-questions] [Patch] XMERL Attributes without whitespaces Message-ID: Hi, When parsing XML, xmerl throws error when there are no whitespace character between 2 attributes. A sample XML is as below. Highlighted areas have problems. Test XML However W3C (http://www.w3schools.com/xml/xml_validator.asp) validates above XLM without any errors. Therefore I guess xmerl better support that. Attached is a patch to fix that. It could help others too, if that patch is added to the xmerl package distributed in OTP distribution. - Eranga -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: xmerl_scan.erl.patch Type: application/octet-stream Size: 593 bytes Desc: not available URL: From desired.mta@REDACTED Mon Sep 15 10:22:09 2014 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Mon, 15 Sep 2014 10:22:09 +0200 Subject: [erlang-questions] [Patch] XMERL Attributes without whitespaces In-Reply-To: References: Message-ID: On Mon, Sep 15, 2014 at 9:44 AM, Eranga Udesh wrote: > Hi, > > However W3C (http://www.w3schools.com/xml/xml_validator.asp) validates above > XLM without any errors. Therefore I guess xmerl better support that. Hi, Udesh, according to XML 1-1 specification[1], your example does not adhere to the grammar. Relevant grammar sections: [3] S ::= (#x20 | #x9 | #xD | #xA)+ [40] STag ::= '<' Name (S Attribute)* S? '>' [41] Attribute ::= Name Eq AttValue Important bit: (S Attribute)*. As you can see, Attribute can never follow Attribute without a whitespace between them. > Attached is a patch to fix that. It could help others too, if that patch is > added to the xmerl package distributed in OTP distribution. Maybe that W3C parser supports non-spaced attributes is coincidental? Or maybe I am looking at the wrong grammar? -- Motiejus Jak?tys [1]: http://www.w3.org/TR/xml11/#sec-starttags From eranga.erl@REDACTED Mon Sep 15 13:22:25 2014 From: eranga.erl@REDACTED (Eranga Udesh) Date: Mon, 15 Sep 2014 16:52:25 +0530 Subject: [erlang-questions] [Patch] XMERL Attributes without whitespaces In-Reply-To: References: Message-ID: Hi Motiejus, It is not just W3C parser, but it seems PHP/Java XML parser(s) too support attributes without spaces. For one of our projects in Erlang, we had to customize xmerl, because their existing XML parsers are supporting the XML syntax without whitespaces in attributes. When validated the same XML in W3C, no errors shown, so being a new solution, we had to comply to their existing XML syntax. - Eranga On Mon, Sep 15, 2014 at 1:52 PM, Motiejus Jak?tys wrote: > On Mon, Sep 15, 2014 at 9:44 AM, Eranga Udesh > wrote: > > Hi, > > > > However W3C (http://www.w3schools.com/xml/xml_validator.asp) validates > above > > XLM without any errors. Therefore I guess xmerl better support that. > > Hi, Udesh, > > according to XML 1-1 specification[1], your example does not adhere to > the grammar. Relevant grammar sections: > > [3] S ::= (#x20 | #x9 | #xD | #xA)+ > [40] STag ::= '<' Name (S Attribute)* S? '>' > [41] Attribute ::= Name Eq AttValue > > Important bit: (S Attribute)*. As you can see, Attribute can never > follow Attribute without a whitespace between them. > > > Attached is a patch to fix that. It could help others too, if that patch > is > > added to the xmerl package distributed in OTP distribution. > > Maybe that W3C parser supports non-spaced attributes is coincidental? > Or maybe I am looking at the wrong grammar? > > -- > Motiejus Jak?tys > > [1]: http://www.w3.org/TR/xml11/#sec-starttags > -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidnwelton@REDACTED Mon Sep 15 17:22:00 2014 From: davidnwelton@REDACTED (David Welton) Date: Mon, 15 Sep 2014 17:22:00 +0200 Subject: [erlang-questions] Architectural quandaries In-Reply-To: <3BB1DF26-F5CA-4F10-BA07-5C8C9766CE35@duomark.com> References: <3BB1DF26-F5CA-4F10-BA07-5C8C9766CE35@duomark.com> Message-ID: Hi, On Thu, Sep 11, 2014 at 4:28 PM, Jay Nelson wrote: > David Welton wrote: >> I'm still struggling a bit with some doubts about how to best >> architect a complex, real-world Erlang application. > Let it crash serves a purpose when you have complex code, but somewhere > at the base you want some sense of stability, especially in a loosely coupled > environment. > > I would focus on a simple core that starts up and connects things. This core > should have a very reliable low level signal between server and tablet that > it is in one of a few states: started, connecting, disconnecting, running, etc. Yes, that sounds sensible. > When all else is going haywire, this core needs to at least indicate to the > tablet so you can give feedback to the user. For continuity, some data > should be cached on the tablet after connecting / running, so that intermittent > failures can be smoothed over in the user?s eyes. Just because the components > may be currently unstable or reconnecting doesn?t mean the user experience > has to be. > > Recently I have been confronted with the excess complexity of our systems > and have been sketching out new approaches to bring up and down systems > in a more controlled manner (and dealing with multi-minute startups in a better > way). You should consider sketching out a supervisor hierarchy along the lines > of: > > 1) Rest_for_one root children > - this guarantees startup ordering > - allows later children to make assumptions > > 2) Start fast core things early > - included base libraries with no real initialization > - required ets tables (you might want to cache send to / recv from tablet events) > > 3) Start an FSM to control startup > - use messages to this to start / stop children of other supervisors > > 4) Rest of the supervisors with no children > - if possible one_for_one under a single last rest_for_one supervisor > - add start_child / stop_child functions and call these from the FSM > > Root (rest_for_one) > - libs > - pre-allocated resources > - FSM > - Services (one_for_one) > - Service 1..N So you would advocate putting everything in one Erlang 'application' in order to take advantage of the restart capabilities such as rest_for_one? I was actually moving to break things up into separate applications with different git trees and everything, so that things could be developed in a more independent way: for instance, the report generation software gets its own application, separate from the hardware control software. I was starting to think along the lines of a centralized system for monitoring some of these applications... -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From ngreco@REDACTED Mon Sep 15 17:33:28 2014 From: ngreco@REDACTED (Nahuel Greco) Date: Mon, 15 Sep 2014 12:33:28 -0300 Subject: [erlang-questions] The implementation of link/1 and monitor/2 In-Reply-To: References: Message-ID: 1. No, links are managed natively by the scheduler/virtual machine, check beam/erl_monitors.h, they are NOT process. Think about them as "lines" connecting process. 2. No, same as (1). 3. They are true language primitives. I think the only way to re-implement them is to create some sort of supervisor process and poll using erlang:process_info/2 to check if the process is alive. But that will be a CPU waste. 4. I think yes but you will need a third process to manage the "link" between two process, to call exit(pid,kill) on the "linked" process not defined to trap exits, and must implement process_flag(trap_exit, true), spawn_link et al. Monitors were added later to the language, there is probably some sort of more fundamental structure to be factored out of monitors and links. Saludos, Nahuel Greco. On Fri, Sep 12, 2014 at 10:22 AM, Torbj?rn Lager wrote: > Hi there, > > I have read about links and monitors in Erlang and think I'm beginning to > understand how they work. I'm a bit curious how they are implemented > though. > > 1. It seems to me that for a link between two processes P1 and P2, two > OTHER processes are needed, one watching over P1 and one watching over P2. > Is this how Erlang implements links? > > 2. It seems to me that a monitor is also a kind of process. Is it a > regular Erlang process of the kind you can spawn yourself? > > 3. Related: If Erlang didn't have link/1 and monitor/2 as BIFs, could they > be implemented in Erlang, or are they true language primitives? > > 4. Related: If Erlang had only monitor/2, could link/1 somehow be > implemented in terms of it? > > Thanks, > Torbj?rn > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay@REDACTED Mon Sep 15 22:00:18 2014 From: jay@REDACTED (Jay Nelson) Date: Mon, 15 Sep 2014 13:00:18 -0700 Subject: [erlang-questions] Architectural quandaries Message-ID: <1F810C27-947B-4B26-96D5-0CEAB0B5EA81@duomark.com> David N. Welton wrote: > So you would advocate putting everything in one Erlang ?application' > in order to take advantage of the restart capabilities such as > rest_for_one? I was actually moving to break things up into separate > applications with different git trees and everything, so that things > could be developed in a more independent way: for instance, the report > generation software gets its own application, separate from the > hardware control software. Don?t confuse development with operations. If you like, you can make separate applications in separate github repos. That may make it easier to test each component. I would do that and have separate PropEr test suites run by common_test for each one (that?s my current style). It means managing separate repos, but if the components are generally useful, it makes it convenient for others. In operations I would have one application that uses included_applications and starts the root supervisor of each of your other components in the correct dependency and startup sequence. Especially if you are writing all the components, you will be intimately familiar with the dependencies and start up behaviour of each one. Of course, this overlord application is the real application you started talking about and would be a separate repo of its own. If you have several applications, rather than using included_applications, you will have the possibility of a component failure which is undetected and will not restart without manually restarting or writing your own code to monitor and manage them. > I was starting to think along the lines of a centralized system for > monitoring some of these applications... Hmm. I prefer to use the OTP tools that are present, and use them to my benefit to avoid such circumstances. Splitting into independent applications defeats all the restart facilities of OTP, unless you use heart and make them all permanent applications and are willing to wait for VM restarts when things start to go sideways? jay From ok@REDACTED Tue Sep 16 05:59:58 2014 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 16 Sep 2014 15:59:58 +1200 Subject: [erlang-questions] [Patch] XMERL Attributes without whitespaces In-Reply-To: References: Message-ID: On 15/09/2014, at 11:22 PM, Eranga Udesh wrote: > Hi Motiejus, > > It is not just W3C parser, but it seems PHP/Java XML parser(s) too support attributes without spaces. For what it's worth, my XML parser and the SWI Prolog XML parser and nsgmls with the -xwml flag accept missing white space, while xmllint(1) does not. It's not just XML 1.1 that says attributes must be separated by spaces. The grammar is the same in XML 1.0, and the HTML 5 specification says in http://www.w3.org/TR/html5/syntax.html#start-tags that "Attributes must be separated from each other by one or more space characters." One would hope that a validator would enforce this. My own parser is called 'qh' (for Quick Hack), so you can guess how much checking it does... From eranga.erl@REDACTED Tue Sep 16 06:05:09 2014 From: eranga.erl@REDACTED (Eranga Udesh) Date: Tue, 16 Sep 2014 09:35:09 +0530 Subject: [erlang-questions] [Patch] XMERL Attributes without whitespaces In-Reply-To: References: Message-ID: Richard, Is your parser open source? Where is it hosted? - Eranga On Tue, Sep 16, 2014 at 9:29 AM, Richard A. O'Keefe wrote: > > On 15/09/2014, at 11:22 PM, Eranga Udesh wrote: > > > Hi Motiejus, > > > > It is not just W3C parser, but it seems PHP/Java XML parser(s) too > support attributes without spaces. > > For what it's worth, my XML parser and the SWI Prolog XML > parser and nsgmls with the -xwml flag > accept missing white space, while xmllint(1) does not. > > It's not just XML 1.1 that says attributes must be separated > by spaces. The grammar is the same in XML 1.0, and the > HTML 5 specification says in > http://www.w3.org/TR/html5/syntax.html#start-tags > that "Attributes must be separated from each other by one > or more space characters." > > One would hope that a validator would enforce this. > My own parser is called 'qh' (for Quick Hack), so you can > guess how much checking it does... > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sperber@REDACTED Tue Sep 16 10:05:41 2014 From: sperber@REDACTED (Michael Sperber) Date: Tue, 16 Sep 2014 10:05:41 +0200 Subject: [erlang-questions] BOB 2015 - 2nd Call for Contributions (Deadline Sep 30) Message-ID: Submissions on/using/reporting on Erlang welcome! BOB Conference 2015 Berlin 23.1.2015 http://bobkonf.de/2015/ CALL FOR CONTRIBUTIONS English: http://bobkonf.de/2015/cfp.html German: http://bobkonf.de/2015/cfp.html Deadline: September 30, 2014 You drive advanced software engineering methods, implement ambitious architectures and are open to cutting-edge innovation? Attend this conference, meet people that share your goals, and get to know the best software tools and technologies available today. We strive to offer you a day full of new experiences and impressions that you can use to immediately improve your daily life as a software developer. If you share our vision and want to contribute, submit a proposal for a talk or tutorial! We are looking for talks about best-of-breed software technology, e.g.: - functional programming - reactive programming - micro-service architectures - persistent data structures and databases - ? everything really that isn?t mainstream, but you think should be. Presenters should provide the audience with information that is practically useful for software developers. This could take the form of e.g.: - experience reports - introductory talks on technical background - demos and how-tos We accept proposals for presentations of 45 minutes (40 minutes talk + 5 minutes questions), as well as 90 minute tutorials for beginners. The language of presentation should be either English or German. It should include (in your presentation language of choice): - an abstract of max. 1500 characters. - a short bio/cv - contact information (including at least email) - a list of 3-5 concrete ideas of how your work can be applied in a developer's daily life - additional material (websites, blogs, slides, videos of past presentations, ?) You can submit your proposal using the following form: https://docs.google.com/spreadsheet/viewform?formkey=dHJ0TjR1cEhUWmdBZFVITGVRVWN5VEE6MA - direct questions to bobkonf at active minus group dot de - proposal deadline: September 30, 2014 - notification: October 15, 2014 - program: October 2014, 2014 NOTE: The conference fee will be waived for presenters, but travel expenses will not be covered. Program Committee - Matthias Fischmann, zerobuzz UG - Matthias Neubauer, SICK AG - Michael Sperber, Active Group - Stefan Wehr, factis research Wissenschaftlicher Beirat - Annette Bieniusa, TU Kaiserslautern - Torsten Grust, Uni T?bingen - Peter Thiemann, Uni Freiburg From n.oxyde@REDACTED Tue Sep 16 11:16:52 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 16 Sep 2014 11:16:52 +0200 Subject: [erlang-questions] [Patch] XMERL Attributes without whitespaces In-Reply-To: References: Message-ID: <00427B23-641C-4C2F-883D-675572088943@gmail.com> How did you feed XML to the W3C validator given that you can't limit it to XML analysis? Le 15 sept. 2014 ? 13:22, Eranga Udesh a ?crit : > Hi Motiejus, > > It is not just W3C parser, but it seems PHP/Java XML parser(s) too support attributes without spaces. > > For one of our projects in Erlang, we had to customize xmerl, because their existing XML parsers are supporting the XML syntax without whitespaces in attributes. When validated the same XML in W3C, no errors shown, so being a new solution, we had to comply to their existing XML syntax. > > - Eranga > > On Mon, Sep 15, 2014 at 1:52 PM, Motiejus Jak?tys wrote: > On Mon, Sep 15, 2014 at 9:44 AM, Eranga Udesh wrote: > > Hi, > > > > However W3C (http://www.w3schools.com/xml/xml_validator.asp) validates above > > XLM without any errors. Therefore I guess xmerl better support that. > > Hi, Udesh, > > according to XML 1-1 specification[1], your example does not adhere to > the grammar. Relevant grammar sections: > > [3] S ::= (#x20 | #x9 | #xD | #xA)+ > [40] STag ::= '<' Name (S Attribute)* S? '>' > [41] Attribute ::= Name Eq AttValue > > Important bit: (S Attribute)*. As you can see, Attribute can never > follow Attribute without a whitespace between them. > > > Attached is a patch to fix that. It could help others too, if that patch is > > added to the xmerl package distributed in OTP distribution. > > Maybe that W3C parser supports non-spaced attributes is coincidental? > Or maybe I am looking at the wrong grammar? > > -- > Motiejus Jak?tys > > [1]: http://www.w3.org/TR/xml11/#sec-starttags > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From eranga.erl@REDACTED Tue Sep 16 11:27:59 2014 From: eranga.erl@REDACTED (Eranga Udesh) Date: Tue, 16 Sep 2014 14:57:59 +0530 Subject: [erlang-questions] [Patch] XMERL Attributes without whitespaces In-Reply-To: <00427B23-641C-4C2F-883D-675572088943@gmail.com> References: <00427B23-641C-4C2F-883D-675572088943@gmail.com> Message-ID: What do you mean? I use the online validator at http://www.w3schools.com/xml/xml_validator.asp - Eranga On Tue, Sep 16, 2014 at 2:46 PM, Anthony Ramine wrote: > How did you feed XML to the W3C validator given that you can't limit it to > XML analysis? > > Le 15 sept. 2014 ? 13:22, Eranga Udesh a ?crit : > > > Hi Motiejus, > > > > It is not just W3C parser, but it seems PHP/Java XML parser(s) too > support attributes without spaces. > > > > For one of our projects in Erlang, we had to customize xmerl, because > their existing XML parsers are supporting the XML syntax without > whitespaces in attributes. When validated the same XML in W3C, no errors > shown, so being a new solution, we had to comply to their existing XML > syntax. > > > > - Eranga > > > > On Mon, Sep 15, 2014 at 1:52 PM, Motiejus Jak?tys > wrote: > > On Mon, Sep 15, 2014 at 9:44 AM, Eranga Udesh > wrote: > > > Hi, > > > > > > However W3C (http://www.w3schools.com/xml/xml_validator.asp) > validates above > > > XLM without any errors. Therefore I guess xmerl better support that. > > > > Hi, Udesh, > > > > according to XML 1-1 specification[1], your example does not adhere to > > the grammar. Relevant grammar sections: > > > > [3] S ::= (#x20 | #x9 | #xD | #xA)+ > > [40] STag ::= '<' Name (S Attribute)* S? '>' > > [41] Attribute ::= Name Eq AttValue > > > > Important bit: (S Attribute)*. As you can see, Attribute can never > > follow Attribute without a whitespace between them. > > > > > Attached is a patch to fix that. It could help others too, if that > patch is > > > added to the xmerl package distributed in OTP distribution. > > > > Maybe that W3C parser supports non-spaced attributes is coincidental? > > Or maybe I am looking at the wrong grammar? > > > > -- > > Motiejus Jak?tys > > > > [1]: http://www.w3.org/TR/xml11/#sec-starttags > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From edgurgel@REDACTED Tue Sep 16 11:53:00 2014 From: edgurgel@REDACTED (Eduardo Gurgel) Date: Tue, 16 Sep 2014 21:53:00 +1200 Subject: [erlang-questions] [Patch] XMERL Attributes without whitespaces In-Reply-To: References: <00427B23-641C-4C2F-883D-675572088943@gmail.com> Message-ID: That's w3schools. Is it "the W3C" validator? On 16 September 2014 21:27, Eranga Udesh wrote: > What do you mean? I use the online validator at > http://www.w3schools.com/xml/xml_validator.asp > > - Eranga > > On Tue, Sep 16, 2014 at 2:46 PM, Anthony Ramine wrote: > >> How did you feed XML to the W3C validator given that you can't limit it >> to XML analysis? >> >> Le 15 sept. 2014 ? 13:22, Eranga Udesh a ?crit : >> >> > Hi Motiejus, >> > >> > It is not just W3C parser, but it seems PHP/Java XML parser(s) too >> support attributes without spaces. >> > >> > For one of our projects in Erlang, we had to customize xmerl, because >> their existing XML parsers are supporting the XML syntax without >> whitespaces in attributes. When validated the same XML in W3C, no errors >> shown, so being a new solution, we had to comply to their existing XML >> syntax. >> > >> > - Eranga >> > >> > On Mon, Sep 15, 2014 at 1:52 PM, Motiejus Jak?tys < >> desired.mta@REDACTED> wrote: >> > On Mon, Sep 15, 2014 at 9:44 AM, Eranga Udesh >> wrote: >> > > Hi, >> > > >> > > However W3C (http://www.w3schools.com/xml/xml_validator.asp) >> validates above >> > > XLM without any errors. Therefore I guess xmerl better support that. >> > >> > Hi, Udesh, >> > >> > according to XML 1-1 specification[1], your example does not adhere to >> > the grammar. Relevant grammar sections: >> > >> > [3] S ::= (#x20 | #x9 | #xD | #xA)+ >> > [40] STag ::= '<' Name (S Attribute)* S? '>' >> > [41] Attribute ::= Name Eq AttValue >> > >> > Important bit: (S Attribute)*. As you can see, Attribute can never >> > follow Attribute without a whitespace between them. >> > >> > > Attached is a patch to fix that. It could help others too, if that >> patch is >> > > added to the xmerl package distributed in OTP distribution. >> > >> > Maybe that W3C parser supports non-spaced attributes is coincidental? >> > Or maybe I am looking at the wrong grammar? >> > >> > -- >> > Motiejus Jak?tys >> > >> > [1]: http://www.w3.org/TR/xml11/#sec-starttags >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Eduardo -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidnwelton@REDACTED Tue Sep 16 12:14:09 2014 From: davidnwelton@REDACTED (David Welton) Date: Tue, 16 Sep 2014 12:14:09 +0200 Subject: [erlang-questions] Architectural quandaries In-Reply-To: <1F810C27-947B-4B26-96D5-0CEAB0B5EA81@duomark.com> References: <1F810C27-947B-4B26-96D5-0CEAB0B5EA81@duomark.com> Message-ID: Hi, On Mon, Sep 15, 2014 at 10:00 PM, Jay Nelson wrote: > David N. Welton wrote: > >> So you would advocate putting everything in one Erlang ?application' >> in order to take advantage of the restart capabilities such as >> rest_for_one? I was actually moving to break things up into separate >> applications with different git trees and everything, so that things >> could be developed in a more independent way: for instance, the report >> generation software gets its own application, separate from the >> hardware control software. > Don?t confuse development with operations. If you like, you can make > separate applications in separate github repos. That may make it easier > to test each component. I would do that and have separate PropEr > test suites run by common_test for each one (that?s my current style). > > It means managing separate repos, but if the components are generally > useful, it makes it convenient for others. > > In operations I would have one application that uses included_applications and > starts the root supervisor of each of your other components in the correct > dependency and startup sequence. Especially if you are writing all the > components, you will be intimately familiar with the dependencies and > start up behaviour of each one. Of course, this overlord application is the > real application you started talking about and would be a separate repo > of its own. Aha! I had missed included_applications, and indeed, that looks like a potentially good way of having both the separate applications as well as the supervision tree. It seems that not everyone is in favor of these: http://learnyousomeerlang.com/the-count-of-applications#included-applications - and I can see that more tightly coupling things is potentially problematic. Realistically though, a lot of our code won't be used without all the other things present either. > If you have several applications, rather than using included_applications, > you will have the possibility of a component failure which is undetected > and will not restart without manually restarting or writing your own code > to monitor and manage them. >> I was starting to think along the lines of a centralized system for >> monitoring some of these applications... > > Hmm. I prefer to use the OTP tools that are present, and use them to my > benefit to avoid such circumstances. Splitting into independent applications > defeats all the restart facilities of OTP, unless you use heart and make > them all permanent applications and are willing to wait for VM restarts > when things start to go sideways? Yes, that's part of what I'm after: how to keep things within OTP as much as possible. After thinking things through some, though, and after Fred H?bert kindly took the time to discuss some of this with me on #erlang, I have come to the conclusion that: OTP alone is not up to the task - there has to be some kind of extra layer or extra logic in there to deal with systems that might not be functioning. Perhaps this provokes a reaction in the reader along the lines of "he has a firm grasp of the obvious", but after drinking the OTP cool-aid, going outside it feels like "I wonder what I'm doing wrong or what I'm missing - they must have something for this, right?". Take, for instance, the hardware in our system - it shouldn't fail, and the system will not work as advertised if it does. *However*, sooner or later, it probably will fail somehow, and the system needs to stay up to aid the user in running diagnostics. Simply including the hardware in the supervision tree leads to things gradually falling over in an unacceptable way. Fred talks about these concepts some here: http://ferd.ca/it-s-about-the-guarantees.html To my way of thinking, it really seems like there should be something more out there in Erlang land for these situations; something that intermediate people like myself can easily find and make use of and feel confident we're doing the right thing. * Better documentation, at least. I think "the database for a web site" provides a great example. The web site should not fall over when the DB becomes unavailable. Code should be included. We hear plenty about letting it crash, but there's a significant number of use cases where no, it's actually more complex than that. * Some kind of gen_transient_service that gathers up the best practices and is a "good enough" solution in many cases. This would help for the "low level" case of a specific resource. It could come with a couple of strategies, and perhaps be pluggable in order to include more... things like exponential backoff. A lot of this code has to look pretty similar: have the connection status in the state, return errors if it's not connected, have a fast init as well as a callback that attempts the connection, and then whatever strategy to handle errors with the connection. Wrapping it up in a library seems possible even if it doesn't cover every corner case out there. * Perhaps some kind of application manager. I'm actually thinking of writing code along these lines, as the above is too specific in our case (I think, at least). Our hardware management stuff has a variety of programs that it takes care of, and having client portions of our code know about all of them is probably not a good idea. I'd rather just have the hardware application go down and have our application manager alert the user, and keep track of what's running: "the hardware system is up, but the report generation system is down". I'm still trying to work out in my head if this is a good idea or not though.... perhaps the gen_transient_service thing is better. Thoughts? Thanks again for reading, and apologies if my normally muddied thoughts are more silted up than usual; I'm a bit short on sleep. -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From daniel.abrahamsson@REDACTED Tue Sep 16 12:42:15 2014 From: daniel.abrahamsson@REDACTED (Daniel Abrahamsson) Date: Tue, 16 Sep 2014 12:42:15 +0200 Subject: [erlang-questions] Architectural quandaries In-Reply-To: References: <1F810C27-947B-4B26-96D5-0CEAB0B5EA81@duomark.com> Message-ID: To me it seems like what you want is a circuit breaker (like Jesper's fuse [0], or breaky [1]) in front of the hardware module. Construct your Erlang code so that it does not crash on hardware errors, but instead melts the fuse. The supervisor is thus only involved when something strange happens in your code and where a restart will bring you back to known state. We use something similar (managed by [2], the name collision with [0] is purely coincidental) for graceful degradation if our HSMs go down. We also use [2] for managing database connections. In fact, you can use this strategy for dealing with any kind of external service without risking taking down your supervision tree. //Daniel [0] https://github.com/jlouis/fuse [1] https://github.com/mmzeeman/breaky [2] https://github.com/ulfl/fuse-lb On Tue, Sep 16, 2014 at 12:14 PM, David Welton wrote: > Hi, > > On Mon, Sep 15, 2014 at 10:00 PM, Jay Nelson wrote: > > David N. Welton wrote: > > > >> So you would advocate putting everything in one Erlang ?application' > >> in order to take advantage of the restart capabilities such as > >> rest_for_one? I was actually moving to break things up into separate > >> applications with different git trees and everything, so that things > >> could be developed in a more independent way: for instance, the report > >> generation software gets its own application, separate from the > >> hardware control software. > > > Don?t confuse development with operations. If you like, you can make > > separate applications in separate github repos. That may make it easier > > to test each component. I would do that and have separate PropEr > > test suites run by common_test for each one (that?s my current style). > > > > It means managing separate repos, but if the components are generally > > useful, it makes it convenient for others. > > > > In operations I would have one application that uses > included_applications and > > starts the root supervisor of each of your other components in the > correct > > dependency and startup sequence. Especially if you are writing all the > > components, you will be intimately familiar with the dependencies and > > start up behaviour of each one. Of course, this overlord application is > the > > real application you started talking about and would be a separate repo > > of its own. > > Aha! I had missed included_applications, and indeed, that looks like > a potentially good way of having both the separate applications as > well as the supervision tree. > > It seems that not everyone is in favor of these: > > http://learnyousomeerlang.com/the-count-of-applications#included-applications > - and I can see that more tightly coupling things is potentially > problematic. Realistically though, a lot of our code won't be used > without all the other things present either. > > > If you have several applications, rather than using > included_applications, > > you will have the possibility of a component failure which is undetected > > and will not restart without manually restarting or writing your own code > > to monitor and manage them. > > >> I was starting to think along the lines of a centralized system for > >> monitoring some of these applications... > > > > Hmm. I prefer to use the OTP tools that are present, and use them to my > > benefit to avoid such circumstances. Splitting into independent > applications > > defeats all the restart facilities of OTP, unless you use heart and make > > them all permanent applications and are willing to wait for VM restarts > > when things start to go sideways? > > Yes, that's part of what I'm after: how to keep things within OTP as > much as possible. > > After thinking things through some, though, and after Fred H?bert > kindly took the time to discuss some of this with me on #erlang, I > have come to the conclusion that: > > OTP alone is not up to the task - there has to be some kind of extra > layer or extra logic in there to deal with systems that might not be > functioning. > > Perhaps this provokes a reaction in the reader along the lines of "he > has a firm grasp of the obvious", but after drinking the OTP cool-aid, > going outside it feels like "I wonder what I'm doing wrong or what I'm > missing - they must have something for this, right?". > > Take, for instance, the hardware in our system - it shouldn't fail, > and the system will not work as advertised if it does. *However*, > sooner or later, it probably will fail somehow, and the system needs > to stay up to aid the user in running diagnostics. Simply including > the hardware in the supervision tree leads to things gradually falling > over in an unacceptable way. > > Fred talks about these concepts some here: > http://ferd.ca/it-s-about-the-guarantees.html > > To my way of thinking, it really seems like there should be something > more out there in Erlang land for these situations; something that > intermediate people like myself can easily find and make use of and > feel confident we're doing the right thing. > > * Better documentation, at least. I think "the database for a web > site" provides a great example. The web site should not fall over > when the DB becomes unavailable. Code should be included. We hear > plenty about letting it crash, but there's a significant number of use > cases where no, it's actually more complex than that. > > * Some kind of gen_transient_service that gathers up the best > practices and is a "good enough" solution in many cases. This would > help for the "low level" case of a specific resource. It could come > with a couple of strategies, and perhaps be pluggable in order to > include more... things like exponential backoff. A lot of this code > has to look pretty similar: have the connection status in the state, > return errors if it's not connected, have a fast init as well as a > callback that attempts the connection, and then whatever strategy to > handle errors with the connection. Wrapping it up in a library seems > possible even if it doesn't cover every corner case out there. > > * Perhaps some kind of application manager. I'm actually thinking of > writing code along these lines, as the above is too specific in our > case (I think, at least). Our hardware management stuff has a variety > of programs that it takes care of, and having client portions of our > code know about all of them is probably not a good idea. I'd rather > just have the hardware application go down and have our application > manager alert the user, and keep track of what's running: "the > hardware system is up, but the report generation system is down". I'm > still trying to work out in my head if this is a good idea or not > though.... perhaps the gen_transient_service thing is better. > > Thoughts? > > Thanks again for reading, and apologies if my normally muddied > thoughts are more silted up than usual; I'm a bit short on sleep. > > -- > David N. Welton > > http://www.welton.it/davidw/ > > http://www.dedasys.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Tue Sep 16 12:56:15 2014 From: roger@REDACTED (Roger Lipscombe) Date: Tue, 16 Sep 2014 11:56:15 +0100 Subject: [erlang-questions] Avoiding crash dump when returning {stop, Reason} from a gen_server? Message-ID: The documentation for gen_server states that handle_call etc. can return {stop, Reason, NewState}, upon which terminate(Reason, NewState) is called, and the gen_server stops. The problem that I have is that it appears that Reason can only be one of 'normal' or 'shutdown', otherwise a crash dump is generated. Can I define other Reason values for which a crash dump is _not_ generated? Some background: I have a simple_one_for_one supervisor, with a set of transient workers. If these workers crash, I want a crash dump, and I want the worker restarted. The worker is restarted with the same arguments as originally provided. So far so good. I want the worker to have the ability to stop itself. This appears to be achievable by returning {stop, normal} from handle_call, handle_cast or handle_info. Again, this is good. However, the original arguments to supervisor:start_child/2 come from a data store. I also want a way for the worker to be able to request a full restart, where the data store is re-queried. So, I'd like to be able to return {stop, reload} from handle_call, etc.. Another process would be monitoring the workers, and would handle this by querying the DB and calling supervisor:start_child with the new values. But: anything other than 'normal' and 'shutdown' appear to cause an error to be reported, and a crash dump to be written. How could I implement this instead? From roger@REDACTED Tue Sep 16 13:20:05 2014 From: roger@REDACTED (Roger Lipscombe) Date: Tue, 16 Sep 2014 12:20:05 +0100 Subject: [erlang-questions] Avoiding crash dump when returning {stop, Reason} from a gen_server? In-Reply-To: References: Message-ID: On 16 September 2014 12:14, Viacheslav V. Kovalev wrote: > It looks like that you getting crash dumps because supervisor exhausts > max restarts count when trying infinitely restart children, that > stopping abnormally (from supervisor's point of view). No. I get crash dumps for the *worker* whenever it stops with anything other than 'normal' or 'shutdown'. I've not tried {shutdown, Term} yet, though. If the restart frequency is exceeded, *then* I get a crash dump for the supervisor. From roger@REDACTED Tue Sep 16 13:45:19 2014 From: roger@REDACTED (Roger Lipscombe) Date: Tue, 16 Sep 2014 12:45:19 +0100 Subject: [erlang-questions] Avoiding crash dump when returning {stop, Reason} from a gen_server? In-Reply-To: References: Message-ID: On 16 September 2014 12:20, Roger Lipscombe wrote: > On 16 September 2014 12:14, Viacheslav V. Kovalev wrote: >> It looks like that you getting crash dumps because supervisor exhausts >> max restarts count when trying infinitely restart children, that >> stopping abnormally (from supervisor's point of view). > > No. I get crash dumps for the *worker* whenever it stops with anything > other than 'normal' or 'shutdown'. I've not tried {shutdown, Term} > yet, though. If the restart frequency is exceeded, *then* I get a > crash dump for the supervisor. My apologies. I was getting confused. I'm using lager, and what I'm seeing is "CRASH REPORT" errors in the log file. I mistakenly thought that these equated to erl_crash.dump being written. They don't. These are scary enough by themselves, but by talking about "crash dumps", I was being confusing. I dug into the source code to attempt to answer some of the rest of my question: see https://github.com/erlang/otp/blob/maint/lib/stdlib/src/gen_server.erl#L636 and https://github.com/erlang/otp/blob/maint/lib/stdlib/src/gen_server.erl#L734. If handle_call, etc., return {stop, Reason}, terminate(Reason), and then there's a "case Reason of..." expression, where normal, shutdown and {shutdown, _} are translated to exit(Reason), but *anything* else results in a call to error_info, which calls error_logger:format, and then exit(Reason). The call to error_logger hits lager, which outputs the scary-looking message to the log, and then writes to its "crash.log" file. So, basically, it appears that -- assuming that the rest of my plan is sound -- I should return {stop, {shutdown, Term}, NewState}, and handle the {shutdown, Term} differently in my monitoring process. From roger@REDACTED Tue Sep 16 14:33:15 2014 From: roger@REDACTED (Roger Lipscombe) Date: Tue, 16 Sep 2014 13:33:15 +0100 Subject: [erlang-questions] Supervisor child type Message-ID: A child specification has a type, which tells whether the child is a supervisor or a worker. What is this actually used for? What's stopping me from putting 'worker' in the child spec, but actually calling supervisor:start_link? Or vice-versa? From essen@REDACTED Tue Sep 16 14:40:22 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 16 Sep 2014 14:40:22 +0200 Subject: [erlang-questions] [Patch] XMERL Attributes without whitespaces In-Reply-To: References: <00427B23-641C-4C2F-883D-675572088943@gmail.com> Message-ID: <54182FB6.9010701@ninenines.eu> It is *not* the W3C validator. The W3C validator is at http://validator.w3.org/ On 09/16/2014 11:53 AM, Eduardo Gurgel wrote: > That's w3schools. Is it "the W3C" validator? > > On 16 September 2014 21:27, Eranga Udesh > wrote: > > What do you mean? I use the online validator at > http://www.w3schools.com/xml/xml_validator.asp > > - Eranga > > On Tue, Sep 16, 2014 at 2:46 PM, Anthony Ramine > wrote: > > How did you feed XML to the W3C validator given that you can't > limit it to XML analysis? > > Le 15 sept. 2014 ? 13:22, Eranga Udesh > a ?crit : > > > Hi Motiejus, > > > > It is not just W3C parser, but it seems PHP/Java XML > parser(s) too support attributes without spaces. > > > > For one of our projects in Erlang, we had to customize xmerl, > because their existing XML parsers are supporting the XML syntax > without whitespaces in attributes. When validated the same XML > in W3C, no errors shown, so being a new solution, we had to > comply to their existing XML syntax. > > > > - Eranga > > > > On Mon, Sep 15, 2014 at 1:52 PM, Motiejus Jak?tys > > wrote: > > On Mon, Sep 15, 2014 at 9:44 AM, Eranga Udesh > > wrote: > > > Hi, > > > > > > However W3C > (http://www.w3schools.com/xml/xml_validator.asp) validates above > > > XLM without any errors. Therefore I guess xmerl better > support that. > > > > Hi, Udesh, > > > > according to XML 1-1 specification[1], your example does not > adhere to > > the grammar. Relevant grammar sections: > > > > [3] S ::= (#x20 | #x9 | #xD | #xA)+ > > [40] STag ::= '<' Name (S Attribute)* S? '>' > > [41] Attribute ::= Name Eq AttValue > > > > Important bit: (S Attribute)*. As you can see, Attribute can > never > > follow Attribute without a whitespace between them. > > > > > Attached is a patch to fix that. It could help others too, > if that patch is > > > added to the xmerl package distributed in OTP distribution. > > > > Maybe that W3C parser supports non-spaced attributes is > coincidental? > > Or maybe I am looking at the wrong grammar? > > > > -- > > Motiejus Jak?tys > > > > [1]: http://www.w3.org/TR/xml11/#sec-starttags > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Eduardo > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu From mononcqc@REDACTED Tue Sep 16 14:48:35 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 16 Sep 2014 08:48:35 -0400 Subject: [erlang-questions] Supervisor child type In-Reply-To: References: Message-ID: <20140916124834.GC10961@ferdair.local> On 09/16, Roger Lipscombe wrote: > A child specification has a type, which tells whether the child is a > supervisor or a worker. What is this actually used for? What's > stopping me from putting 'worker' in the child spec, but actually > calling supervisor:start_link? Or vice-versa? The value is used by the OTP code when traversing supervision trees to suspend/reload modules and do the proper calls to 'code change'. It more or less lets you annotate the supervision tree with the metadata required to do things in one swoop: is this a supervisor, is this a worker, and if it's a worker, what kind of code is it running (a module list or is it 'dynamic'?) You can put wrong data in there if you want to. Nothing will stop you. That sounds like a bad idea, but elans of civil disobedience can always be applauded if done for the right reasons. Finding such a right reason might be tricky in the current context, however. "Sticking it to the release handler" might have a good ring to it, but it would only be an obstruction to a middleman, shooting the messenger if you will. Much more efficient would be to confuse the maintainer or operator directly, which most developers already do by virtue of poor or plainly erroneous documentation; at least, in such a case, you can always yell a vindictive "UGH JUST READ THE SOURCE!!!" down a google or skype session (or if in the 90s, a phone), and get a similar effect. Regards, Fred. From eranga.erl@REDACTED Tue Sep 16 14:49:37 2014 From: eranga.erl@REDACTED (Eranga Udesh) Date: Tue, 16 Sep 2014 18:19:37 +0530 Subject: [erlang-questions] [Patch] XMERL Attributes without whitespaces In-Reply-To: <54182FB6.9010701@ninenines.eu> References: <00427B23-641C-4C2F-883D-675572088943@gmail.com> <54182FB6.9010701@ninenines.eu> Message-ID: It's my mistake. The XML parser I sent is w3schools and not the W3C. The W3C parser at http://validator.w3.org too parses the same XML I sent and return below status. This document was successfully checked as well-formed XML! - Eranga On Tue, Sep 16, 2014 at 6:10 PM, Lo?c Hoguin wrote: > It is *not* the W3C validator. The W3C validator is at > http://validator.w3.org/ > > On 09/16/2014 11:53 AM, Eduardo Gurgel wrote: > >> That's w3schools. Is it "the W3C" validator? >> >> On 16 September 2014 21:27, Eranga Udesh > > wrote: >> >> What do you mean? I use the online validator at >> http://www.w3schools.com/xml/xml_validator.asp >> >> - Eranga >> >> On Tue, Sep 16, 2014 at 2:46 PM, Anthony Ramine > > wrote: >> >> How did you feed XML to the W3C validator given that you can't >> limit it to XML analysis? >> >> Le 15 sept. 2014 ? 13:22, Eranga Udesh > > a ?crit : >> >> > Hi Motiejus, >> > >> > It is not just W3C parser, but it seems PHP/Java XML >> parser(s) too support attributes without spaces. >> > >> > For one of our projects in Erlang, we had to customize xmerl, >> because their existing XML parsers are supporting the XML syntax >> without whitespaces in attributes. When validated the same XML >> in W3C, no errors shown, so being a new solution, we had to >> comply to their existing XML syntax. >> > >> > - Eranga >> > >> > On Mon, Sep 15, 2014 at 1:52 PM, Motiejus Jak?tys >> > wrote: >> > On Mon, Sep 15, 2014 at 9:44 AM, Eranga Udesh >> > wrote: >> > > Hi, >> > > >> > > However W3C >> (http://www.w3schools.com/xml/xml_validator.asp) validates above >> > > XLM without any errors. Therefore I guess xmerl better >> support that. >> > >> > Hi, Udesh, >> > >> > according to XML 1-1 specification[1], your example does not >> adhere to >> > the grammar. Relevant grammar sections: >> > >> > [3] S ::= (#x20 | #x9 | #xD | #xA)+ >> > [40] STag ::= '<' Name (S Attribute)* S? '>' >> > [41] Attribute ::= Name Eq AttValue >> > >> > Important bit: (S Attribute)*. As you can see, Attribute can >> never >> > follow Attribute without a whitespace between them. >> > >> > > Attached is a patch to fix that. It could help others too, >> if that patch is >> > > added to the xmerl package distributed in OTP distribution. >> > >> > Maybe that W3C parser supports non-spaced attributes is >> coincidental? >> > Or maybe I am looking at the wrong grammar? >> > >> > -- >> > Motiejus Jak?tys >> > >> > [1]: http://www.w3.org/TR/xml11/#sec-starttags >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED > erlang.org> >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> -- >> Eduardo >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- > Lo?c Hoguin > http://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Tue Sep 16 15:06:48 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 16 Sep 2014 15:06:48 +0200 Subject: [erlang-questions] [Patch] XMERL Attributes without whitespaces In-Reply-To: References: <00427B23-641C-4C2F-883D-675572088943@gmail.com> <54182FB6.9010701@ninenines.eu> Message-ID: <71BB3CD6-9198-4516-8AAE-566D19881A8C@gmail.com> "The parser implementations we used for this check are based on OpenSP (SGML/XML) and libxml2 (XML)." Sounds like a bug within their validator or somewhere, because libxml2 is supposed to return an error in that case, see for exemple there: . It seems to me that this check is forgotten when the validator is asked to check *just* XML syntax. Le 16 sept. 2014 ? 14:49, Eranga Udesh a ?crit : > It's my mistake. The XML parser I sent is w3schools and not the W3C. The W3C parser at http://validator.w3.org too parses the same XML I sent and return below status. From zxq9@REDACTED Wed Sep 17 10:36:46 2014 From: zxq9@REDACTED (zxq9) Date: Wed, 17 Sep 2014 17:36:46 +0900 Subject: [erlang-questions] Internationalization Message-ID: <1447951.uxGCNzZ4b9@burrito> Is there a go-to internationalization solution for user interface strings? In particular, anything that works well with wxErlang or could be made to do so? I've looked and asked around a bit, but haven't found anything that appears to be very current or well used/known (but this could just be because of the relative lack of user-facing GUI code in Erlang). Any ideas would be appreciated. -Craig From taavi@REDACTED Wed Sep 17 10:49:07 2014 From: taavi@REDACTED (Taavi Talvik) Date: Wed, 17 Sep 2014 11:49:07 +0300 Subject: [erlang-questions] Internationalization In-Reply-To: <1447951.uxGCNzZ4b9@burrito> References: <1447951.uxGCNzZ4b9@burrito> Message-ID: <481D4938-8099-4435-89DC-0F007AFAE9C4@uninet.ee> On 17 Sep 2014, at 11:36, zxq9 wrote: > Is there a go-to internationalization solution for user interface strings? In > particular, anything that works well with wxErlang or could be made to do so? > > I've looked and asked around a bit, but haven't found anything that appears to > be very current or well used/known (but this could just be because of the > relative lack of user-facing GUI code in Erlang). There is erlang get text library: https://github.com/noss/erlang-gettext Adaption of it is used very successfully in zotonic: https://github.com/zotonic/zotonic/tree/master/src/i18n best regards, taavi From davidnwelton@REDACTED Wed Sep 17 11:54:14 2014 From: davidnwelton@REDACTED (David Welton) Date: Wed, 17 Sep 2014 11:54:14 +0200 Subject: [erlang-questions] Architectural quandaries In-Reply-To: References: <1F810C27-947B-4B26-96D5-0CEAB0B5EA81@duomark.com> Message-ID: On Tue, Sep 16, 2014 at 12:42 PM, Daniel Abrahamsson wrote: > To me it seems like what you want is a circuit breaker (like Jesper's fuse > [0], or breaky [1]) in front of the hardware module. Construct your Erlang > code so that it does not crash on hardware errors, but instead melts the > fuse. The supervisor is thus only involved when something strange happens in > your code and where a restart will bring you back to known state. We use > something similar (managed by [2], the name collision with [0] is purely > coincidental) for graceful degradation if our HSMs go down. We also use [2] > for managing database connections. In fact, you can use this strategy for > dealing with any kind of external service without risking taking down your > supervision tree. > > //Daniel > > [0] https://github.com/jlouis/fuse > [1] https://github.com/mmzeeman/breaky > [2] https://github.com/ulfl/fuse-lb Aha! Yes, that's probably what I want, or very close to it. It's a pity the concept is not more widely documented, as it's very important for dealing with external services that may be down at some point that should not, however, pull Erlang down with them. If it were up to me, I'd even put something like it in OTP, because it seems very likely that any large enough project will encounter a need like it. Thank you -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From kenneth.lundin@REDACTED Wed Sep 17 13:21:13 2014 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 17 Sep 2014 13:21:13 +0200 Subject: [erlang-questions] [ANN] Erlang/OTP 17.3 has been released Message-ID: Erlang/OTP 17.3 has been released. Erlang/OTP 17.3 is a service release on the 17 track with mostly bug fixes, but it does contain a number of new features and characteristics improvements as well. Some highlights of the release are: - erts: Introduced enif_schedule_nif() which allows a long running NIF to be broken into separate NIF invocations without the help of a wrapper function written in Erlang - common_test: Experimental support for running Quickcheck and PropEr tests from common_test suites is added. Examples of usage in the suites for the ssh and inets applications - Bugfixes and minor new features in applications such as asn1, erts, kernel, stdlib, diameter, ssh, mnesia, ssl, jinterface, ... Many thanks to 20 different contributors in this release You can find the README file with more detailed info at http://www.erlang.org/download/otp_src_17.3.readme You can download the full source distribution from http://www.erlang.org/download/otp_src_17.3.tar.gz Note: To unpack the TAR archive you need a GNU TAR compatible program. For installation instructions please read the README that is part of the distribution. You can also find this release at the official Erlang/OTP Git-repository at Github here: https://github.com/erlang/otp tagged "OTP-17.3" The Windows binary distribution can be downloaded from http://www.erlang.org/download/otp_win32_17.3.exe http://www.erlang.org/download/otp_win64_17.3.exe You can also download the complete HTML documentation or the Unix manual files http://www.erlang.org/download/otp_doc_html_17.3.tar.gz http://www.erlang.org/download/otp_doc_man_17.3.tar.gz We also want to thank those that sent us patches, suggestions and bug reports. The Erlang/OTP Team at Ericsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From mahesh@REDACTED Wed Sep 17 13:56:01 2014 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Wed, 17 Sep 2014 06:56:01 -0500 Subject: [erlang-questions] Fwd: [ANN] Erlang/OTP 17.3 has been released In-Reply-To: References: Message-ID: "common_test: Experimental support for running Quickcheck and PropEr tests from common_test suites is added. Examples of usage in the suites for the ssh and inets applications" Huh? The docs (http://erldocs.com/maint/common_test/ct_property_test.html) are delightfully cryptic, as is the example code ( https://github.com/erlang/otp/blob/maint/lib/ssh/test/ssh_property_test_SUITE.erl ) So, what *does* this do that I wasn't already doing vis-a-vis PropEr? cheers ---------- Forwarded message ---------- From: Kenneth Lundin Date: Wed, Sep 17, 2014 at 6:21 AM Subject: [erlang-questions] [ANN] Erlang/OTP 17.3 has been released To: Erlang Users' List , erlang-announce@REDACTED Erlang/OTP 17.3 has been released. Erlang/OTP 17.3 is a service release on the 17 track with mostly bug fixes, but it does contain a number of new features and characteristics improvements as well. Some highlights of the release are: - erts: Introduced enif_schedule_nif() which allows a long running NIF to be broken into separate NIF invocations without the help of a wrapper function written in Erlang - common_test: Experimental support for running Quickcheck and PropEr tests from common_test suites is added. Examples of usage in the suites for the ssh and inets applications - Bugfixes and minor new features in applications such as asn1, erts, kernel, stdlib, diameter, ssh, mnesia, ssl, jinterface, ... Many thanks to 20 different contributors in this release You can find the README file with more detailed info at http://www.erlang.org/download/otp_src_17.3.readme You can download the full source distribution from http://www.erlang.org/download/otp_src_17.3.tar.gz Note: To unpack the TAR archive you need a GNU TAR compatible program. For installation instructions please read the README that is part of the distribution. You can also find this release at the official Erlang/OTP Git-repository at Github here: https://github.com/erlang/otp tagged "OTP-17.3" The Windows binary distribution can be downloaded from http://www.erlang.org/download/otp_win32_17.3.exe http://www.erlang.org/download/otp_win64_17.3.exe You can also download the complete HTML documentation or the Unix manual files http://www.erlang.org/download/otp_doc_html_17.3.tar.gz http://www.erlang.org/download/otp_doc_man_17.3.tar.gz We also want to thank those that sent us patches, suggestions and bug reports. The Erlang/OTP Team at Ericsson _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -- *Mahesh Paolini-Subramanya That tall bald Indian guy..* *Google+ | Blog | Twitter | LinkedIn * -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Wed Sep 17 15:22:06 2014 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 17 Sep 2014 08:22:06 -0500 Subject: [erlang-questions] [ANN] Erlang/OTP 17.3 has been released In-Reply-To: References: Message-ID: Very, very nice cleanup on the dirty nif API. If I read it right, I can now mark a nif as dirty with a single flag: static ErlNifFunc nif_funcs[] = { {"open_device", 3, open_device}, {"create_context", 0, create_context, ERL_NIF_DIRTY_JOB_CPU_BOUND}, {"destroy_context", 1, destroy_context, ERL_NIF_DIRTY_JOB_IO_BOUND} }; On Wed, Sep 17, 2014 at 6:21 AM, Kenneth Lundin wrote: > Erlang/OTP 17.3 has been released. > > Erlang/OTP 17.3 is a service release on the 17 track with mostly bug > fixes, but it does contain a number of new features and characteristics > improvements as well. > > Some highlights of the release are: > > - erts: Introduced enif_schedule_nif() which allows a long running NIF > to be broken into separate NIF invocations without the help of a wrapper > function written in Erlang > - common_test: Experimental support for running Quickcheck and PropEr > tests from common_test suites is added. Examples of usage in the suites for > the ssh and inets applications > - Bugfixes and minor new features in applications such as asn1, erts, > kernel, stdlib, diameter, ssh, mnesia, ssl, jinterface, ... > > Many thanks to 20 different contributors in this release > > You can find the README file with more detailed info at > http://www.erlang.org/download/otp_src_17.3.readme > > You can download the full source distribution from > http://www.erlang.org/download/otp_src_17.3.tar.gz > > Note: To unpack the TAR archive you need a GNU TAR compatible program. For > installation instructions please read the README that is part of the > distribution. > > You can also find this release at the official Erlang/OTP Git-repository > at Github here: https://github.com/erlang/otp tagged "OTP-17.3" > > The Windows binary distribution can be downloaded from > > http://www.erlang.org/download/otp_win32_17.3.exe > > http://www.erlang.org/download/otp_win64_17.3.exe > > You can also download the complete HTML documentation or the Unix manual > files > > http://www.erlang.org/download/otp_doc_html_17.3.tar.gz > http://www.erlang.org/download/otp_doc_man_17.3.tar.gz > > We also want to thank those that sent us patches, suggestions and bug > reports. > > The Erlang/OTP Team at Ericsson > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Wed Sep 17 15:41:37 2014 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 17 Sep 2014 09:41:37 -0400 Subject: [erlang-questions] [ANN] Erlang/OTP 17.3 has been released In-Reply-To: References: Message-ID: Yes, that's correct. You can also now call enif_make_badarg() in a dirty NIF as well, so no need to hop through a regular NIF just to validate incoming argc/argv before scheduling a dirty call. For more detail see https://github.com/erlang/otp/commit/e167bca85a86cc7a149d53da5cdd08b0905e71a6 --steve On Wed, Sep 17, 2014 at 9:22 AM, Daniel Goertzen wrote: > Very, very nice cleanup on the dirty nif API. If I read it right, I can > now mark a nif as dirty with a single flag: > > static ErlNifFunc nif_funcs[] = { > {"open_device", 3, open_device}, > {"create_context", 0, create_context, ERL_NIF_DIRTY_JOB_CPU_BOUND}, > {"destroy_context", 1, destroy_context, ERL_NIF_DIRTY_JOB_IO_BOUND} > }; > > On Wed, Sep 17, 2014 at 6:21 AM, Kenneth Lundin > wrote: > >> Erlang/OTP 17.3 has been released. >> >> Erlang/OTP 17.3 is a service release on the 17 track with mostly bug >> fixes, but it does contain a number of new features and characteristics >> improvements as well. >> >> Some highlights of the release are: >> >> - erts: Introduced enif_schedule_nif() which allows a long running >> NIF to be broken into separate NIF invocations without the help of a >> wrapper function written in Erlang >> - common_test: Experimental support for running Quickcheck and PropEr >> tests from common_test suites is added. Examples of usage in the suites for >> the ssh and inets applications >> - Bugfixes and minor new features in applications such as asn1, erts, >> kernel, stdlib, diameter, ssh, mnesia, ssl, jinterface, ... >> >> Many thanks to 20 different contributors in this release >> >> You can find the README file with more detailed info at >> http://www.erlang.org/download/otp_src_17.3.readme >> >> You can download the full source distribution from >> http://www.erlang.org/download/otp_src_17.3.tar.gz >> >> Note: To unpack the TAR archive you need a GNU TAR compatible program. >> For installation instructions please read the README that is part of the >> distribution. >> >> You can also find this release at the official Erlang/OTP Git-repository >> at Github here: https://github.com/erlang/otp tagged "OTP-17.3" >> >> The Windows binary distribution can be downloaded from >> >> http://www.erlang.org/download/otp_win32_17.3.exe >> >> http://www.erlang.org/download/otp_win64_17.3.exe >> >> You can also download the complete HTML documentation or the Unix manual >> files >> >> http://www.erlang.org/download/otp_doc_html_17.3.tar.gz >> http://www.erlang.org/download/otp_doc_man_17.3.tar.gz >> >> We also want to thank those that sent us patches, suggestions and bug >> reports. >> >> The Erlang/OTP Team at Ericsson >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Wed Sep 17 15:51:04 2014 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Wed, 17 Sep 2014 16:51:04 +0300 Subject: [erlang-questions] [ANN] Erlang/OTP 17.3 has been released In-Reply-To: References: Message-ID: <319459BD-8A7B-4338-9149-AFDF4FDDBCC5@gmail.com> Awesome, Thanks! Since, you touched jinterface? Do you have any plans to update maven repository with recent interface build? http://mvnrepository.com/artifact/org.erlang.otp/jinterface Best Regards, Dmitry On 17 Sep 2014, at 14:21, Kenneth Lundin wrote: > Erlang/OTP 17.3 has been released. > > Erlang/OTP 17.3 is a service release on the 17 track with mostly bug fixes, but it does contain a number of new features and characteristics improvements as well. > > Some highlights of the release are: > erts: Introduced enif_schedule_nif() which allows a long running NIF to be broken into separate NIF invocations without the help of a wrapper function written in Erlang > common_test: Experimental support for running Quickcheck and PropEr tests from common_test suites is added. Examples of usage in the suites for the ssh and inets applications > Bugfixes and minor new features in applications such as asn1, erts, kernel, stdlib, diameter, ssh, mnesia, ssl, jinterface, ... > Many thanks to 20 different contributors in this release > > You can find the README file with more detailed info at http://www.erlang.org/download/otp_src_17.3.readme > > You can download the full source distribution from http://www.erlang.org/download/otp_src_17.3.tar.gz > > Note: To unpack the TAR archive you need a GNU TAR compatible program. For installation instructions please read the README that is part of the distribution. > > You can also find this release at the official Erlang/OTP Git-repository at Github here: https://github.com/erlang/otp tagged "OTP-17.3" > > The Windows binary distribution can be downloaded from > > http://www.erlang.org/download/otp_win32_17.3.exe > > http://www.erlang.org/download/otp_win64_17.3.exe > > You can also download the complete HTML documentation or the Unix manual files > > http://www.erlang.org/download/otp_doc_html_17.3.tar.gz > http://www.erlang.org/download/otp_doc_man_17.3.tar.gz > > We also want to thank those that sent us patches, suggestions and bug reports. > > The Erlang/OTP Team at Ericsson > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed Sep 17 17:45:08 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 17 Sep 2014 11:45:08 -0400 Subject: [erlang-questions] [ANN] Stuff Goes Bad: Erlang in Anger Message-ID: <20140917154506.GD32479@ferdair.local> Hi everyone, After a lot of time on the backburner, the Heroku routing team and I are excited to announce 'Stuff Goes Bad: Erlang in Anger' >From the introduction: This book intends to be a little guide about how to be the Erlang medic in a time of war. It is first and foremost a collection of tips and tricks to help understand where failures come from, and a dictionary of different code snippets and practices that helped developers debug production systems that were built in Erlang. This is our attempt at bridging the gap between most tutorials, books, training sessions, and actually being able to operate, diagnose, and debug running systems once they've made it to production. It's entirely free (as in beer), available as a PDF, and using a CC license (almost free as in freedom). We hope this will prove useful to the community! If you have feedback or whatever, you can send it directly to me, or meet some of our team members at the http://www.chicagoerlang.com/ conference early next week! Blog post at: http://engineering.heroku.com/blogs/2014-09-17-erlang-in-anger Download at: http://www.erlang-in-anger.com/ From garry@REDACTED Wed Sep 17 18:21:57 2014 From: garry@REDACTED (Garry Hodgson) Date: Wed, 17 Sep 2014 12:21:57 -0400 Subject: [erlang-questions] [ANN] Stuff Goes Bad: Erlang in Anger In-Reply-To: <20140917154506.GD32479@ferdair.local> References: <20140917154506.GD32479@ferdair.local> Message-ID: <5419B525.5070202@research.att.com> looks great at first glance. thanks very much for sharing. On 9/17/14 11:45 AM, Fred Hebert wrote: > Hi everyone, > > After a lot of time on the backburner, the Heroku routing team and I are > excited to announce 'Stuff Goes Bad: Erlang in Anger' > > From the introduction: > > This book intends to be a little guide about how to be the Erlang > medic in a time of war. It is first and foremost a collection of > tips and tricks to help understand where failures come from, and a > dictionary of different code snippets and practices that helped > developers debug production systems that were built in Erlang. > > This is our attempt at bridging the gap between most tutorials, books, > training sessions, and actually being able to operate, diagnose, and > debug running systems once they've made it to production. > > It's entirely free (as in beer), available as a PDF, and using a CC > license (almost free as in freedom). > > We hope this will prove useful to the community! If you have feedback or > whatever, you can send it directly to me, or meet some of our team > members at the http://www.chicagoerlang.com/ conference early next week! > > Blog post at: > http://engineering.heroku.com/blogs/2014-09-17-erlang-in-anger > > Download at: > http://www.erlang-in-anger.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From federico.carrone@REDACTED Wed Sep 17 19:12:22 2014 From: federico.carrone@REDACTED (Federico Carrone) Date: Wed, 17 Sep 2014 14:12:22 -0300 Subject: [erlang-questions] [ANN] Stuff Goes Bad: Erlang in Anger In-Reply-To: <5419B525.5070202@research.att.com> References: <20140917154506.GD32479@ferdair.local> <5419B525.5070202@research.att.com> Message-ID: This is awesome and the design is not from 1993 like most Erlang pages. On Wed, Sep 17, 2014 at 1:21 PM, Garry Hodgson wrote: > looks great at first glance. thanks very much for sharing. > > > On 9/17/14 11:45 AM, Fred Hebert wrote: > >> Hi everyone, >> >> After a lot of time on the backburner, the Heroku routing team and I are >> excited to announce 'Stuff Goes Bad: Erlang in Anger' >> >> From the introduction: >> >> This book intends to be a little guide about how to be the Erlang >> medic in a time of war. It is first and foremost a collection of >> tips and tricks to help understand where failures come from, and a >> dictionary of different code snippets and practices that helped >> developers debug production systems that were built in Erlang. >> >> This is our attempt at bridging the gap between most tutorials, books, >> training sessions, and actually being able to operate, diagnose, and >> debug running systems once they've made it to production. >> >> It's entirely free (as in beer), available as a PDF, and using a CC >> license (almost free as in freedom). >> >> We hope this will prove useful to the community! If you have feedback or >> whatever, you can send it directly to me, or meet some of our team >> members at the http://www.chicagoerlang.com/ conference early next week! >> >> Blog post at: >> http://engineering.heroku.com/blogs/2014-09-17-erlang-in-anger >> >> Download at: >> http://www.erlang-in-anger.com/ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- http://federicocarrone.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bryan@REDACTED Wed Sep 17 19:13:58 2014 From: bryan@REDACTED (Bryan) Date: Wed, 17 Sep 2014 10:13:58 -0700 Subject: [erlang-questions] [ANN] Stuff Goes Bad: Erlang in Anger In-Reply-To: References: <20140917154506.GD32479@ferdair.local> <5419B525.5070202@research.att.com> Message-ID: Yes. Thank you so much for making this real. This is super awesome. Cheers, Bryan ---- Bryan Hughes CTO and Founder / Go Factory http://www.go-factory.net Connecting the Internet of Things On Sep 17, 2014, at 10:12 AM, Federico Carrone wrote: > This is awesome and the design is not from 1993 like most Erlang pages. > > On Wed, Sep 17, 2014 at 1:21 PM, Garry Hodgson wrote: > looks great at first glance. thanks very much for sharing. > > > On 9/17/14 11:45 AM, Fred Hebert wrote: > Hi everyone, > > After a lot of time on the backburner, the Heroku routing team and I are > excited to announce 'Stuff Goes Bad: Erlang in Anger' > > From the introduction: > > This book intends to be a little guide about how to be the Erlang > medic in a time of war. It is first and foremost a collection of > tips and tricks to help understand where failures come from, and a > dictionary of different code snippets and practices that helped > developers debug production systems that were built in Erlang. > > This is our attempt at bridging the gap between most tutorials, books, > training sessions, and actually being able to operate, diagnose, and > debug running systems once they've made it to production. > > It's entirely free (as in beer), available as a PDF, and using a CC > license (almost free as in freedom). > > We hope this will prove useful to the community! If you have feedback or > whatever, you can send it directly to me, or meet some of our team > members at the http://www.chicagoerlang.com/ conference early next week! > > Blog post at: > http://engineering.heroku.com/blogs/2014-09-17-erlang-in-anger > > Download at: > http://www.erlang-in-anger.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > http://federicocarrone.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Sep 17 19:15:26 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 17 Sep 2014 19:15:26 +0200 Subject: [erlang-questions] [ANN] Stuff Goes Bad: Erlang in Anger In-Reply-To: <20140917154506.GD32479@ferdair.local> References: <20140917154506.GD32479@ferdair.local> Message-ID: <5419C1AE.6090309@ninenines.eu> Congratulations! Am I the only one disappointed by the lack of silly drawings? ;-) On 09/17/2014 05:45 PM, Fred Hebert wrote: > Hi everyone, > > After a lot of time on the backburner, the Heroku routing team and I are > excited to announce 'Stuff Goes Bad: Erlang in Anger' > > From the introduction: > > This book intends to be a little guide about how to be the Erlang > medic in a time of war. It is first and foremost a collection of > tips and tricks to help understand where failures come from, and a > dictionary of different code snippets and practices that helped > developers debug production systems that were built in Erlang. > > This is our attempt at bridging the gap between most tutorials, books, > training sessions, and actually being able to operate, diagnose, and > debug running systems once they've made it to production. > > It's entirely free (as in beer), available as a PDF, and using a CC > license (almost free as in freedom). > > We hope this will prove useful to the community! If you have feedback or > whatever, you can send it directly to me, or meet some of our team > members at the http://www.chicagoerlang.com/ conference early next week! > > Blog post at: > http://engineering.heroku.com/blogs/2014-09-17-erlang-in-anger > > Download at: > http://www.erlang-in-anger.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu From tony@REDACTED Wed Sep 17 20:41:53 2014 From: tony@REDACTED (Tony Rogvall) Date: Wed, 17 Sep 2014 20:41:53 +0200 Subject: [erlang-questions] [ANN] Stuff Goes Bad: Erlang in Anger In-Reply-To: References: <20140917154506.GD32479@ferdair.local> <5419B525.5070202@research.att.com> Message-ID: <18DD0AAF-7B19-4E03-A96D-5728F770D217@rogvall.se> Agree. Impressive. (1993 was a great year, then I started at Computer Science Lab :-) /Tony On 17 sep 2014, at 19:12, Federico Carrone wrote: > This is awesome and the design is not from 1993 like most Erlang pages. > > On Wed, Sep 17, 2014 at 1:21 PM, Garry Hodgson wrote: > looks great at first glance. thanks very much for sharing. > > > On 9/17/14 11:45 AM, Fred Hebert wrote: > Hi everyone, > > After a lot of time on the backburner, the Heroku routing team and I are > excited to announce 'Stuff Goes Bad: Erlang in Anger' > > From the introduction: > > This book intends to be a little guide about how to be the Erlang > medic in a time of war. It is first and foremost a collection of > tips and tricks to help understand where failures come from, and a > dictionary of different code snippets and practices that helped > developers debug production systems that were built in Erlang. > > This is our attempt at bridging the gap between most tutorials, books, > training sessions, and actually being able to operate, diagnose, and > debug running systems once they've made it to production. > > It's entirely free (as in beer), available as a PDF, and using a CC > license (almost free as in freedom). > > We hope this will prove useful to the community! If you have feedback or > whatever, you can send it directly to me, or meet some of our team > members at the http://www.chicagoerlang.com/ conference early next week! > > Blog post at: > http://engineering.heroku.com/blogs/2014-09-17-erlang-in-anger > > Download at: > http://www.erlang-in-anger.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > http://federicocarrone.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From beastie@REDACTED Wed Sep 17 19:38:38 2014 From: beastie@REDACTED (Mark Reynolds) Date: Wed, 17 Sep 2014 19:38:38 +0200 Subject: [erlang-questions] [ANN] Stuff Goes Bad: Erlang in Anger In-Reply-To: References: <20140917154506.GD32479@ferdair.local> <5419B525.5070202@research.att.com> Message-ID: <1410975518.1755825.168668605.56D7FA1B@webmail.messagingengine.com> Awesome Ferd! Thanks! :) Federico, you're right! It's a LaTeX design from 1984 ;) On Wed, Sep 17, 2014, at 19:12, Federico Carrone wrote: This is awesome and the design is not from 1993 like most Erlang pages. On Wed, Sep 17, 2014 at 1:21 PM, Garry Hodgson <[1]garry@REDACTED> wrote: looks great at first glance. thanks very much for sharing. On 9/17/14 11:45 AM, Fred Hebert wrote: Hi everyone, After a lot of time on the backburner, the Heroku routing team and I are excited to announce 'Stuff Goes Bad: Erlang in Anger' From the introduction: This book intends to be a little guide about how to be the Erlang medic in a time of war. It is first and foremost a collection of tips and tricks to help understand where failures come from, and a dictionary of different code snippets and practices that helped developers debug production systems that were built in Erlang. This is our attempt at bridging the gap between most tutorials, books, training sessions, and actually being able to operate, diagnose, and debug running systems once they've made it to production. It's entirely free (as in beer), available as a PDF, and using a CC license (almost free as in freedom). We hope this will prove useful to the community! If you have feedback or whatever, you can send it directly to me, or meet some of our team members at the [2]http://www.chicagoerlang.com/ conference early next week! Blog post at: [3]http://engineering.heroku.com/blogs/2014-09-17-erlang-in- anger Download at: [4]http://www.erlang-in-anger.com/ _______________________________________________ erlang-questions mailing list [5]erlang-questions@REDACTED [6]http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list [7]erlang-questions@REDACTED [8]http://erlang.org/mailman/listinfo/erlang-questions -- [9]http://federicocarrone.com/ _______________________________________________ erlang-questions mailing list [10]erlang-questions@REDACTED [11]http://erlang.org/mailman/listinfo/erlang-questions References 1. mailto:garry@REDACTED 2. http://www.chicagoerlang.com/ 3. http://engineering.heroku.com/blogs/2014-09-17-erlang-in-anger 4. http://www.erlang-in-anger.com/ 5. mailto:erlang-questions@REDACTED 6. http://erlang.org/mailman/listinfo/erlang-questions 7. mailto:erlang-questions@REDACTED 8. http://erlang.org/mailman/listinfo/erlang-questions 9. http://federicocarrone.com/ 10. mailto:erlang-questions@REDACTED 11. http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.nijhof@REDACTED Wed Sep 17 21:09:16 2014 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Wed, 17 Sep 2014 21:09:16 +0200 Subject: [erlang-questions] [ANN] Stuff Goes Bad: Erlang in Anger In-Reply-To: <20140917154506.GD32479@ferdair.local> References: <20140917154506.GD32479@ferdair.local> Message-ID: Nice! Will read when back home, Erlang stack traces have always been hugely cryptic to me. On Sep 17, 2014 5:45 PM, "Fred Hebert" wrote: > Hi everyone, > > After a lot of time on the backburner, the Heroku routing team and I are > excited to announce 'Stuff Goes Bad: Erlang in Anger' > > From the introduction: > > This book intends to be a little guide about how to be the Erlang > medic in a time of war. It is first and foremost a collection of > tips and tricks to help understand where failures come from, and a > dictionary of different code snippets and practices that helped > developers debug production systems that were built in Erlang. > > This is our attempt at bridging the gap between most tutorials, books, > training sessions, and actually being able to operate, diagnose, and > debug running systems once they've made it to production. > > It's entirely free (as in beer), available as a PDF, and using a CC > license (almost free as in freedom). > > We hope this will prove useful to the community! If you have feedback or > whatever, you can send it directly to me, or meet some of our team > members at the http://www.chicagoerlang.com/ conference early next week! > > Blog post at: > http://engineering.heroku.com/blogs/2014-09-17-erlang-in-anger > > Download at: > http://www.erlang-in-anger.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From define.null@REDACTED Wed Sep 17 21:17:30 2014 From: define.null@REDACTED (Vasily Demidenok) Date: Wed, 17 Sep 2014 23:17:30 +0400 Subject: [erlang-questions] Schedulers go to sleep even with +sfwi 500 option on R16B03 Message-ID: Hello list, We faced strange performance degradation on the long living node, and found out that only 3 schedulers out of 12 on a 12 - core machine had been active for a very long time before the failure. When the load suddenly increased these 3 schedulers tried to cope with it and hit 100% utilisation very fast, and even when after 10-15 minutes other schedulers woke up they also hit 100 utilisation but because of heavy mail boxes (2-3 millions of msgs) it was not in their power to cope with it. So then OOM came and killed the node. It was not the load which our application could not cope with, but there was very little load for a long time before sudden increase, that's why I guess there schedulers went to sleep. We have tried approach mentioned in the https://gist.github.com/chewbranca/07d9a6eed3da7b490b47 with +sfwi 500 options and it had been working rather well for a long time, but now even with this option we faced with this situation again, when after some period only some of schedulers (4-8) out of 12 begin to work (25-40% of utilisation). I've checked run_queue and found out that rather often there are peaks 500-2000 on the graph derieved from erlang:statistics(run_queue). So I guess when the schedulers wake up every 500 ms there should be a chance for them to steal some work. I will give an example: I check: 14> erlang:system_info(schedulers). 12 15> erlang:system_info(schedulers_online). 12 So all schedulers are online, but only 8 of them are actually working: 16> erlang:statistics(scheduler_wall_time). [{2,145209897883663,426710944077287}, {5,138589165714560,426710943955704}, {3,139109592659767,426710943835106}, {12,40582505715431,426710943217819}, {11,42243183369677,426710943767617}, {10,51368547792756,426710943763646}, {6,136285798256036,426710943776964}, {7,105950434195715,426710942867275}, {9,87470817950104,426710943696217}, {8,100726919140566,426710943762083}, {1,146252599566570,426710943763556}, {4,138618718320296,426710943749318}] Here we can see that 9-12 schedulers are less busy then others. As I can see from out metrics they remained in this state for 20-30 hours, which is crazy. I tried this solution: erlang:system_flag(schedulers_online, 1). timer:sleep(10). erlang:system_flag(schedulers_online, 12). It helped and activated all schedulers, but only for several hours. Then situation repeated. I wonder if there exist any solution to force schedulers not to go to sleep? -------------- next part -------------- An HTML attachment was scrubbed... URL: From stu.bailey@REDACTED Thu Sep 18 00:24:09 2014 From: stu.bailey@REDACTED (Stu Bailey) Date: Wed, 17 Sep 2014 15:24:09 -0700 Subject: [erlang-questions] Erlang History Question Message-ID: I was just wondering if there was any relationship between Strand http://en.wikipedia.org/wiki/Strand_(programming_language) and Erlang? Clearly there is a common ancestor in Prolog. :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Thu Sep 18 00:32:26 2014 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Thu, 18 Sep 2014 00:32:26 +0200 Subject: [erlang-questions] [ANN] Erlang/OTP 17.3 has been released Message-ID: Dirty Nif done dirty cheap ... (wink to AC/DC fans) Daniel Goertzen a ?crit?: >Very, very nice cleanup on the dirty nif API. If I read it right, I can >now mark a nif as dirty with a single flag: > >static ErlNifFunc nif_funcs[] = { > {"open_device", 3, open_device}, > {"create_context", 0, create_context, ERL_NIF_DIRTY_JOB_CPU_BOUND}, > {"destroy_context", 1, destroy_context, ERL_NIF_DIRTY_JOB_IO_BOUND} >}; > >On Wed, Sep 17, 2014 at 6:21 AM, Kenneth Lundin >wrote: > >> Erlang/OTP 17.3 has been released. >> >> Erlang/OTP 17.3 is a service release on the 17 track with mostly bug >> fixes, but it does contain a number of new features and characteristics >> improvements as well. >> >> Some highlights of the release are: >> >> - erts: Introduced enif_schedule_nif() which allows a long running NIF >> to be broken into separate NIF invocations without the help of a wrapper >> function written in Erlang >> - common_test: Experimental support for running Quickcheck and PropEr >> tests from common_test suites is added. Examples of usage in the suites for >> the ssh and inets applications >> - Bugfixes and minor new features in applications such as asn1, erts, >> kernel, stdlib, diameter, ssh, mnesia, ssl, jinterface, ... >> >> Many thanks to 20 different contributors in this release >> >> You can find the README file with more detailed info at >> http://www.erlang.org/download/otp_src_17.3.readme >> >> You can download the full source distribution from >> http://www.erlang.org/download/otp_src_17.3.tar.gz >> >> Note: To unpack the TAR archive you need a GNU TAR compatible program. For >> installation instructions please read the README that is part of the >> distribution. >> >> You can also find this release at the official Erlang/OTP Git-repository >> at Github here: https://github.com/erlang/otp tagged "OTP-17.3" >> >> The Windows binary distribution can be downloaded from >> >> http://www.erlang.org/download/otp_win32_17.3.exe >> >> http://www.erlang.org/download/otp_win64_17.3.exe >> >> You can also download the complete HTML documentation or the Unix manual >> files >> >> http://www.erlang.org/download/otp_doc_html_17.3.tar.gz >> http://www.erlang.org/download/otp_doc_man_17.3.tar.gz >> >> We also want to thank those that sent us patches, suggestions and bug >> reports. >> >> The Erlang/OTP Team at Ericsson >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Sep 18 04:13:12 2014 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 18 Sep 2014 14:13:12 +1200 Subject: [erlang-questions] [ANN] Stuff Goes Bad: Erlang in Anger In-Reply-To: <20140917154506.GD32479@ferdair.local> References: <20140917154506.GD32479@ferdair.local> Message-ID: <234D9F21-EC10-4161-A955-52B5E5803FEF@cs.otago.ac.nz> Do you mind if I start calling you "Fred HerObert"? *THANK YOU* From fritchie@REDACTED Thu Sep 18 06:49:50 2014 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 17 Sep 2014 23:49:50 -0500 Subject: [erlang-questions] Schedulers go to sleep even with +sfwi 500 option on R16B03 In-Reply-To: Message of "Wed, 17 Sep 2014 23:17:30 +0400." Message-ID: <54260.1411015790@snookles.snookles.com> Vasily Demidenok wrote: vd> It helped and activated all schedulers, but only for several hours. vd> Then situation repeated. One way is to periodically check and use the same hack to reset. The check doesn't come for free, but at least it avoids manual checking and manual intervention. Many thanks to Steve Vinoski for collaboration on that code. https://gist.github.com/slfritchie/23220d2bf61b08b9a8b9 Meanwhile, the only way that I know of to avoid scheduler collapse is to use both the +sfwi and +scl flags together, e.g. "+sfwi 500 +scl false". The effect on performance of busy systems varies quite a bit: you will want to measure your common workloads yourself to see if the medicine is worse than the disease. Thanks for pointing out the gist about the CouchDB VM version dilemma. It's a great summary. -Scott From erlangsiri@REDACTED Thu Sep 18 10:08:56 2014 From: erlangsiri@REDACTED (Siri Hansen) Date: Thu, 18 Sep 2014 10:08:56 +0200 Subject: [erlang-questions] Avoiding crash dump when returning {stop, Reason} from a gen_server? In-Reply-To: References: Message-ID: 2014-09-16 13:45 GMT+02:00 Roger Lipscombe : > > So, basically, it appears that -- assuming that the rest of my plan is > sound -- I should return {stop, {shutdown, Term}, NewState}, and > handle the {shutdown, Term} differently in my monitoring process. > Yes, that is probably what you are looking for - {shutdown,Reason} was introduced for reasons similar to this. It is handled exactly like shutdown from supervisor's, gen_*'s and proc_lib's point of view. You can read about it in the reference manual for these modules also. Regards /siri -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Thu Sep 18 10:09:15 2014 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 18 Sep 2014 10:09:15 +0200 Subject: [erlang-questions] Erlang History Question In-Reply-To: References: Message-ID: There is some, actually. Although the first prototype of Erlang was implemented on top of Prolog, Armstrong et al. then looked at compiling to Parlog and Strand to see if they could get better performance throught the built-in parallelism of those languages. However, the results weren't good, and they came to the conclusion that the concurrency model of Erlang was too different, so they moved on to making their own abstract machine. See http://www.erlang.se/publications/prac_appl_prolog.pdf for more details. Also, in the first Erlang book "Concurrent Programming in Erlang" (Prentice Hall, 1993), they wrote "The early syntax of Erlang owed much to Strand, though the current syntax is more reminiscent of an untyped ML. The model of concurrency is similar to SDL." (CCITT Specification and Description Language, for those not familiar with telecom specs, not the Simple DirectMedia Layer.) /Richard On Thu, Sep 18, 2014 at 12:24 AM, Stu Bailey wrote: > I was just wondering if there was any relationship between Strand > > http://en.wikipedia.org/wiki/Strand_(programming_language) > > and Erlang? > > Clearly there is a common ancestor in Prolog. :-) > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Thu Sep 18 12:28:13 2014 From: jinni.park@REDACTED (Park, Sungjin) Date: Thu, 18 Sep 2014 19:28:13 +0900 Subject: [erlang-questions] [ANN] Stuff Goes Bad: Erlang in Anger In-Reply-To: <234D9F21-EC10-4161-A955-52B5E5803FEF@cs.otago.ac.nz> References: <20140917154506.GD32479@ferdair.local> <234D9F21-EC10-4161-A955-52B5E5803FEF@cs.otago.ac.nz> Message-ID: THANK YOU FRED! YOU DID A REAL GREAT JOB. On Thu, Sep 18, 2014 at 11:13 AM, Richard A. O'Keefe wrote: > Do you mind if I start calling you "Fred HerObert"? > > *THANK YOU* > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Thu Sep 18 12:36:11 2014 From: sdl.web@REDACTED (Leo Liu) Date: Thu, 18 Sep 2014 18:36:11 +0800 Subject: [erlang-questions] why doesn't int:attach(Pid, Function) take a fun? Message-ID: Is there any reason why int:attach or int:autoattach not take a fun? Thanks, Leo From vladdu55@REDACTED Thu Sep 18 14:31:16 2014 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 18 Sep 2014 14:31:16 +0200 Subject: [erlang-questions] why doesn't int:attach(Pid, Function) take a fun? In-Reply-To: References: Message-ID: Hi, On Thu, Sep 18, 2014 at 12:36 PM, Leo Liu wrote: > Is there any reason why int:attach or int:autoattach not take a fun? > But it does: http://www.erlang.org/doc/man/int.html#auto_attach-2 regards, Vlad > Thanks, Leo > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu Sep 18 14:52:26 2014 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 18 Sep 2014 14:52:26 +0200 Subject: [erlang-questions] why doesn't int:attach(Pid, Function) take a fun? In-Reply-To: <20140918124318.GC98186@ferdair.local> References: <20140918124318.GC98186@ferdair.local> Message-ID: On Thu, Sep 18, 2014 at 2:43 PM, Fred Hebert wrote: > On 09/18, Vlad Dumitrescu wrote: > > On Thu, Sep 18, 2014 at 12:36 PM, Leo Liu wrote: > > > > > Is there any reason why int:attach or int:autoattach not take a fun? > > > > > > > But it does: http://www.erlang.org/doc/man/int.html#auto_attach-2 > > > > It takes a {M,F} or {M,F,Args}, not a fun(). > > https://github.com/erlang/otp/blob/maint/lib/debugger/src/int.erl#L180-L184 > : > > auto_attach(Flags, {Mod, Func}) -> > auto_attach(Flags, {Mod, Func, []}); > auto_attach(Flags, {Mod, Func, Args}) when > is_atom(Mod),is_atom(Func),is_list(Args) -> > check_flags(Flags), > dbg_iserver:safe_cast({set_auto_attach, Flags, {Mod, Func, Args}}). > Ah, okay, I had misunderstood the question. I would presume to guess that the reason is that the API just didn't follow the language's development and nobody asked for an updated API. regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Thu Sep 18 15:03:32 2014 From: sdl.web@REDACTED (Leo Liu) Date: Thu, 18 Sep 2014 21:03:32 +0800 Subject: [erlang-questions] why doesn't int:attach(Pid, Function) take a fun? In-Reply-To: (Vlad Dumitrescu's message of "Thu, 18 Sep 2014 14:52:26 +0200") References: <20140918124318.GC98186@ferdair.local> Message-ID: On 2014-09-18 14:52 +0200, Vlad Dumitrescu wrote: > I would presume to guess that the reason is that the API just didn't follow > the language's development and nobody asked for an updated API. Thanks for the info. Where to ask for an update? Leo From vladdu55@REDACTED Thu Sep 18 15:07:34 2014 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 18 Sep 2014 15:07:34 +0200 Subject: [erlang-questions] why doesn't int:attach(Pid, Function) take a fun? In-Reply-To: References: <20140918124318.GC98186@ferdair.local> Message-ID: On Thu, Sep 18, 2014 at 3:03 PM, Leo Liu wrote: > On 2014-09-18 14:52 +0200, Vlad Dumitrescu wrote: > > I would presume to guess that the reason is that the API just didn't > follow > > the language's development and nobody asked for an updated API. > > Thanks for the info. Where to ask for an update? I think the easiest way would be to supply a patch for that. Since it's an addition, it won't break backwards compatibility and it's a simple fix. Otherwise, try to get more "me too"s from people on this list :-) regards, Vlad > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andreas@REDACTED Thu Sep 18 15:20:54 2014 From: andreas@REDACTED (Andreas Schumacher) Date: Thu, 18 Sep 2014 15:20:54 +0200 Subject: [erlang-questions] Fwd: FW: Fwd: FW: Fwd: Fwd: SSL peer verification in httpc with Mozilla's certificate store In-Reply-To: References: <314D10C4-C1B3-4DE0-A195-8B659B4065A9@ericsson.com> <1410636769.2265370.167128165.71538F05@webmail.messagingengine.com> <20140914020009.GD1653@ferdair.local> Message-ID: On request from our customers, we sometimes have to provide or backport fixes on old releases. In the future, those tags are also going to be pushed to Github; although, only for OTP 17 onwards. Andreas ---------- Forwarded message ---------- From: Andreas Schumacher Date: Thu, Sep 18, 2014 at 3:10 PM Subject: FW: [erlang-questions] Fwd: FW: Fwd: Fwd: SSL peer verification in httpc with Mozilla's certificate store To: "andreasschu@REDACTED" On 14/09/14 20:28, "Tuncer Ayaz" wrote: >On Sun, Sep 14, 2014 at 5:17 PM, Andreas Schumacher wrote: >> External maintainers can ignore the versions that are not officially >> announced as a major release or service package, and for which >> neither source/binary packages nor documentation are published on >> erlang.org. Nevertheless, in order to increase transparency and to >> avoid similar confusion and discussions in the future, we could >> announce those patch packages on erlang-questions. Then external >> maintainers can decide themselves whether they want to include those >> packages into their products. > >I don't mind tag-only releases, and actually _appreciate_ it very much, >so thanks a lot! The confusion is, from what I can tell, based on the >fact that 17.1 was announced but 17.2 was not. > >If commit messages and git history is maintained in a descriptive and >consistent state, you could automate release notes with links to the >relevant commits. But right now there are somtimes intermediate and >other commits you'd have to filter from auto-generated release notes. >For example, try 'git shortlog OTP-17.1.2...OTP-17.2.2', and you'll >find 'wip' commits that ideally would have been amended or squashed. > >That said, does the OTP team backport fixes to old releases? If they >do, such tags (without an official release) can be useful for various >production nodes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From adelzhang@REDACTED Fri Sep 19 06:21:26 2014 From: adelzhang@REDACTED (Adel Zhang) Date: Fri, 19 Sep 2014 12:21:26 +0800 Subject: [erlang-questions] what is spawn_request/1 ? Message-ID: hi, I find the following comments in application_master.erl: %%%----------------------------------------------------------------- %%% The logical and physical process structrure is as follows: %%% %%% logical physical %%% %%% -------- -------- %%% |AM(GL)| |AM(GL)| %%% -------- -------- %%% | | %%% -------- -------- %%% |Appl P| | X | %%% -------- -------- %%% | %%% -------- %%% |Appl P| %%% -------- %%% %%% Where AM(GL) == Application Master (Group Leader) %%% Appl P == The application specific root process (child to AM) %%% X == A special 'invisible' process %%% The reason for not using the logical structrure is that %%% the application start function is synchronous, and %%% that the AM is GL. This means that if AM executed the start %%% function, and this function uses spawn_request/1 %%% or io, deadlock would occur. Therefore, this function is %%% executed by the process X. Also, AM needs three loops; %%% init_loop (waiting for the start function to return) %%% main_loop %%% terminate_loop (waiting for the process to die) %%% In each of these loops, io and other requests are handled. It mentions that if GL uses *spawn_request/1*, deadlock would occur. What is spawn_request/1? Cannot find the function in Erlang/OTP library. From davidnwelton@REDACTED Fri Sep 19 10:59:42 2014 From: davidnwelton@REDACTED (David Welton) Date: Fri, 19 Sep 2014 10:59:42 +0200 Subject: [erlang-questions] Strange crash and hang Message-ID: Hi, We're getting an odd crash that I suspect is related to one we had in the past, where we were getting a bad term from a C node. Since the term being sent is pretty large (20+ megabytes), when this gets caught up in the error logger and turned from a large binary into a linked-list of the string representation of that binary, it kills Erlang. What's particularly odd about it is that many times, it doesn't finish writing the crash dump, it just sits there: Gdb: #0 0x00007f0a0c020e93 in select () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00000000005495e0 in erts_sys_main_thread () #2 0x00000000004561d0 in erl_start () #3 0x00000000004367d9 in main () strace -f -p 9205 Process 9205 attached with 19 threads - interrupt to quit [pid 9248] futex(0x7f0a0b0c0550, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9247] futex(0x7f0a0b0c0510, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9246] futex(0x7f0a0b0c04d0, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9245] futex(0x7f0a0b0c0490, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9244] futex(0x7f0a0b0c0450, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9243] wait4(-1, [pid 9241] futex(0x7f0a0b0c0390, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9242] futex(0x7f0a0b0c03d0, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9240] futex(0x7f0a0b0c0350, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9239] futex(0x7f0a0b0c0310, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9238] futex(0x7f0a0b0c02d0, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9236] futex(0x7f0a0b0c0250, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9237] futex(0x7f0a0b0c0290, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9235] futex(0x7f0a0b0c0210, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9234] futex(0x7f0a0b0c01d0, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9233] futex(0x7f0a0b0c0190, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 9232] futex(0x888484, FUTEX_WAIT_PRIVATE, 3, NULL [pid 9231] read(6, [pid 9205] select(0, NULL, NULL, NULL, NULL^C It's not writing to the crash dump file or doing anything really, it's just stuck in that select. Any ideas what may be happening? Here's the beginning of the crash dump. Slogan: eheap_alloc: Cannot allocate 3049303816 bytes of memory (of type "heap_frag"). System version: Erlang/OTP 17 [erts-6.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] Compiled: Mon Jul 7 15:00:04 2014 Taints: crypto Thoughts or ideas? I'm going to try dumping in a recompiled beam.smp in order to see if I can dump a simple debug message when things crash on a bad term. Thanks -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From ivan.martinez@REDACTED Fri Sep 19 11:00:36 2014 From: ivan.martinez@REDACTED (=?UTF-8?B?SXbDoW4gTWFydMOtbmV6?=) Date: Fri, 19 Sep 2014 11:00:36 +0200 Subject: [erlang-questions] SSL handshake fails Message-ID: Hello all, I just hired a CentOS 7 server that came with very little software installed. I installed Erlang 17.3 from sources, attached is output of the configure step. Now I'm trying to install zotonic but it fails when trying to do a SSL handshake with github, see below: [ivan@REDACTED zotonic]$ make erl -noshell -s inets -s ssl \ -eval '{ok, saved_to_file} = httpc:request(get, {" https://github.com/rebar/rebar/wiki/rebar", []}, [], [{stream, "./rebar"}])' \ -s init stop {"init terminating in do_boot",{{badmatch,{error,{failed_connect,[{to_address,{"github.com",443}},{inet,[inet],{eoptions,{{{badmatch,<<0 bytes>>},[{ssl_handshake,dec_hello_extensions,2,[{file,"ssl_handshake.erl"},{line,1737}]},{ssl_handshake,decode_handshake,3,[{file,"ssl_handshake.erl"},{line,926}]},{tls_handshake,get_tls_handshake_aux,3,[{file,"tls_handshake.erl"},{line,155}]},{tls_connection,next_state,4,[{file,"tls_connection.erl"},{line,433}]},{gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,503}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]},{gen_fsm,sync_send_all_state_event,[<0.54.0>,{start,infinity},infinity]}}}}]}}},[{erl_eval,expr,3,[]}]}} Crash dump was written to: erl_crash.dump init terminating in do_boot () make: *** [rebar] Error 1 I tried to do the handshake with openssl and apparently it works: [ivan@REDACTED zotonic]$ openssl s_client -host github.com -port 443 CONNECTED(00000003) depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High Assurance EV Root CA verify return:1 depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 Extended Validation Server CA verify return:1 depth=0 businessCategory = Private Organization, 1.3.6.1.4.1.311.60.2.1.3 = US, 1.3.6.1.4.1.311.60.2.1.2 = Delaware, serialNumber = 5157550, street = 548 4th Street, postalCode = 94107, C = US, ST = California, L = San Francisco, O = "GitHub, Inc.", CN = github.com verify return:1 --- Certificate chain 0 s:/businessCategory=Private Organization/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/serialNumber=5157550/street=548 4th Street/postalCode=94107/C=US/ST=California/L=San Francisco/O=GitHub, Inc./CN=github.com i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended Validation Server CA 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended Validation Server CA i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA --- Server certificate -----BEGIN CERTIFICATE----- MIIF4DCCBMigAwIBAgIQDACTENIG2+M3VTWAEY3chzANBgkqhkiG9w0BAQsFADB1 ... XX4C2NesiZcLYbc2n7B9O+63M2k= -----END CERTIFICATE----- subject=/businessCategory=Private Organization/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/serialNumber=5157550/street=548 4th Street/postalCode=94107/C=US/ST=California/L=San Francisco/O=GitHub, Inc./CN=github.com issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended Validation Server CA --- No client certificate CA names sent Server Temp Key: ECDH, prime256v1, 256 bits --- SSL handshake has read 3233 bytes and written 375 bytes --- New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256 Server public key is 2048 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE SSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-AES128-GCM-SHA256 Session-ID: DDEF6E78852287351EC5B20FFDD2578F8996E7226CB883A5F1A94325048B79C6 Session-ID-ctx: Master-Key: D6C6283F463BFCD5A160E0CCE0CC8962CF944E5C98153040E4BC20466981B1622A5327C1E6BBED5F1751A049782908E5 Key-Arg : None Krb5 Principal: None PSK identity: None PSK identity hint: None Start Time: 1411113552 Timeout : 300 (sec) Verify return code: 0 (ok) --- closed What can be wrong?. Thank you. Ivan -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: erlconf.out Type: application/octet-stream Size: 36452 bytes Desc: not available URL: From jon@REDACTED Fri Sep 19 11:38:35 2014 From: jon@REDACTED (Jon Schneider) Date: Fri, 19 Sep 2014 10:38:35 +0100 Subject: [erlang-questions] Strange crash and hang In-Reply-To: References: Message-ID: <0c30d082ee3b1a5190298a743e89f8cc.squirrel@mail.jschneider.net> Avoiding David's actual problem this is surely yet another instance of traditional linked lists causing massive trouble just because they're how LISP did lists decades ago when pointers weren't 64 bits. As a means of storing strings they're diabolical. Would it not be worth purging lists from some of the workings of Erlang ? Jon > end up in the error logger and turned from a large binary into a > linked-list of the string representation of that binary From semmitmondo@REDACTED Fri Sep 19 14:33:30 2014 From: semmitmondo@REDACTED (semmit mondo) Date: Fri, 19 Sep 2014 14:33:30 +0200 (CEST) Subject: [erlang-questions] metaprogramming with -spec Message-ID: Hi, Is it possible to figure out the spec of functions in a compiled module? Given a module name I want to be able to know what is the return typeof a function in that module and what are the types of the arguments.But all I know at the moment is the arity of the function (via module_info).Is that info lost when compiling? -------------- next part -------------- An HTML attachment was scrubbed... URL: From benmmurphy@REDACTED Fri Sep 19 14:50:19 2014 From: benmmurphy@REDACTED (Ben Murphy) Date: Fri, 19 Sep 2014 13:50:19 +0100 Subject: [erlang-questions] SSL handshake fails In-Reply-To: References: Message-ID: I think it is because it sends an empty server_name extension and otp cannot handle it. A server that receives a client hello containing the "server_name" extension MAY use the information contained in the extension to guide its selection of an appropriate certificate to return to the client, and/or other aspects of security policy. In this event, the server SHALL include an extension of type "server_name" in the (extended) server hello. The "extension_data" field of this extension SHALL be empty. On Fri, Sep 19, 2014 at 10:00 AM, Iv?n Mart?nez wrote: > Hello all, > I just hired a CentOS 7 server that came with very little software > installed. I installed Erlang 17.3 from sources, attached is output of the > configure step. Now I'm trying to install zotonic but it fails when trying > to do a SSL handshake with github, see below: > > [ivan@REDACTED zotonic]$ make > erl -noshell -s inets -s ssl \ > -eval '{ok, saved_to_file} = httpc:request(get, > {"https://github.com/rebar/rebar/wiki/rebar", []}, [], [{stream, > "./rebar"}])' \ > -s init stop > {"init terminating in > do_boot",{{badmatch,{error,{failed_connect,[{to_address,{"github.com",443}},{inet,[inet],{eoptions,{{{badmatch,<<0 > bytes>>},[{ssl_handshake,dec_hello_extensions,2,[{file,"ssl_handshake.erl"},{line,1737}]},{ssl_handshake,decode_handshake,3,[{file,"ssl_handshake.erl"},{line,926}]},{tls_handshake,get_tls_handshake_aux,3,[{file,"tls_handshake.erl"},{line,155}]},{tls_connection,next_state,4,[{file,"tls_connection.erl"},{line,433}]},{gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,503}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]},{gen_fsm,sync_send_all_state_event,[<0.54.0>,{start,infinity},infinity]}}}}]}}},[{erl_eval,expr,3,[]}]}} > > Crash dump was written to: erl_crash.dump > init terminating in do_boot () > make: *** [rebar] Error 1 > > I tried to do the handshake with openssl and apparently it works: > > [ivan@REDACTED zotonic]$ openssl s_client -host github.com -port 443 > CONNECTED(00000003) > depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High > Assurance EV Root CA > verify return:1 > depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 > Extended Validation Server CA > verify return:1 > depth=0 businessCategory = Private Organization, 1.3.6.1.4.1.311.60.2.1.3 = > US, 1.3.6.1.4.1.311.60.2.1.2 = Delaware, serialNumber = 5157550, street = > 548 4th Street, postalCode = 94107, C = US, ST = California, L = San > Francisco, O = "GitHub, Inc.", CN = github.com > verify return:1 > --- > Certificate chain > 0 s:/businessCategory=Private > Organization/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/serialNumber=5157550/street=548 > 4th Street/postalCode=94107/C=US/ST=California/L=San Francisco/O=GitHub, > Inc./CN=github.com > i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended > Validation Server CA > 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended > Validation Server CA > i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV > Root CA > --- > Server certificate > -----BEGIN CERTIFICATE----- > MIIF4DCCBMigAwIBAgIQDACTENIG2+M3VTWAEY3chzANBgkqhkiG9w0BAQsFADB1 > ... > XX4C2NesiZcLYbc2n7B9O+63M2k= > -----END CERTIFICATE----- > subject=/businessCategory=Private > Organization/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/serialNumber=5157550/street=548 > 4th Street/postalCode=94107/C=US/ST=California/L=San Francisco/O=GitHub, > Inc./CN=github.com > issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended > Validation Server CA > --- > No client certificate CA names sent > Server Temp Key: ECDH, prime256v1, 256 bits > --- > SSL handshake has read 3233 bytes and written 375 bytes > --- > New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256 > Server public key is 2048 bit > Secure Renegotiation IS supported > Compression: NONE > Expansion: NONE > SSL-Session: > Protocol : TLSv1.2 > Cipher : ECDHE-RSA-AES128-GCM-SHA256 > Session-ID: > DDEF6E78852287351EC5B20FFDD2578F8996E7226CB883A5F1A94325048B79C6 > Session-ID-ctx: > Master-Key: > D6C6283F463BFCD5A160E0CCE0CC8962CF944E5C98153040E4BC20466981B1622A5327C1E6BBED5F1751A049782908E5 > Key-Arg : None > Krb5 Principal: None > PSK identity: None > PSK identity hint: None > Start Time: 1411113552 > Timeout : 300 (sec) > Verify return code: 0 (ok) > --- > closed > > What can be wrong?. Thank you. > Ivan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From vladdu55@REDACTED Fri Sep 19 14:52:09 2014 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 19 Sep 2014 14:52:09 +0200 Subject: [erlang-questions] metaprogramming with -spec In-Reply-To: References: Message-ID: On Fri, Sep 19, 2014 at 2:33 PM, semmit mondo wrote: > > Is it possible to figure out the spec of functions in a compiled module? > > Given a module name I want to be able to know what is the return type > of a function in that module and what are the types of the arguments. > But all I know at the moment is the arity of the function (via > module_info). > Is that info lost when compiling? > If the module was compiled with debug info, then you can call beam_lib:chunks(Module, [abstract_code]) and traverse the result looking for the spec attributes. I would have expected that the specs would be listed in the 'attributes' chunk, but it looks that they aren't. Is there a reason for that? regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Fri Sep 19 14:56:47 2014 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 19 Sep 2014 14:56:47 +0200 Subject: [erlang-questions] metaprogramming with -spec In-Reply-To: References: Message-ID: See also http://erlang.org/pipermail/erlang-questions/2011-March/056896.html On Fri, Sep 19, 2014 at 2:52 PM, Vlad Dumitrescu wrote: > On Fri, Sep 19, 2014 at 2:33 PM, semmit mondo > wrote: > >> >> Is it possible to figure out the spec of functions in a compiled module? >> >> Given a module name I want to be able to know what is the return type >> of a function in that module and what are the types of the arguments. >> But all I know at the moment is the arity of the function (via >> module_info). >> Is that info lost when compiling? >> > > If the module was compiled with debug info, then you can call > beam_lib:chunks(Module, [abstract_code]) and traverse the result looking > for the spec attributes. > > I would have expected that the specs would be listed in the 'attributes' > chunk, but it looks that they aren't. Is there a reason for that? > > regards, > Vlad > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan.martinez@REDACTED Fri Sep 19 15:40:00 2014 From: ivan.martinez@REDACTED (=?UTF-8?B?SXbDoW4gTWFydMOtbmV6?=) Date: Fri, 19 Sep 2014 15:40:00 +0200 Subject: [erlang-questions] SSL handshake fails In-Reply-To: References: Message-ID: I realized it's just attempting to download rebar, so I just put it in the project folder myself and it doesn't complain anymore. Still that's a workaround, there is some issue. Will report it to zotonic guys. Thank you, Ivan 2014-09-19 14:50 GMT+02:00 Ben Murphy : > I think it is because it sends an empty server_name extension and otp > cannot handle it. > > A server that receives a client hello containing the "server_name" > extension MAY use the information contained in the extension to guide > its selection of an appropriate certificate to return to the client, > and/or other aspects of security policy. In this event, the server > SHALL include an extension of type "server_name" in the (extended) > server hello. The "extension_data" field of this extension SHALL be > empty. > > > > On Fri, Sep 19, 2014 at 10:00 AM, Iv?n Mart?nez > wrote: > > Hello all, > > I just hired a CentOS 7 server that came with very little software > > installed. I installed Erlang 17.3 from sources, attached is output of > the > > configure step. Now I'm trying to install zotonic but it fails when > trying > > to do a SSL handshake with github, see below: > > > > [ivan@REDACTED zotonic]$ make > > erl -noshell -s inets -s ssl \ > > -eval '{ok, saved_to_file} = httpc:request(get, > > {"https://github.com/rebar/rebar/wiki/rebar", []}, [], [{stream, > > "./rebar"}])' \ > > -s init stop > > {"init terminating in > > do_boot",{{badmatch,{error,{failed_connect,[{to_address,{"github.com > ",443}},{inet,[inet],{eoptions,{{{badmatch,<<0 > > > bytes>>},[{ssl_handshake,dec_hello_extensions,2,[{file,"ssl_handshake.erl"},{line,1737}]},{ssl_handshake,decode_handshake,3,[{file,"ssl_handshake.erl"},{line,926}]},{tls_handshake,get_tls_handshake_aux,3,[{file,"tls_handshake.erl"},{line,155}]},{tls_connection,next_state,4,[{file,"tls_connection.erl"},{line,433}]},{gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,503}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]},{gen_fsm,sync_send_all_state_event,[<0.54.0>,{start,infinity},infinity]}}}}]}}},[{erl_eval,expr,3,[]}]}} > > > > Crash dump was written to: erl_crash.dump > > init terminating in do_boot () > > make: *** [rebar] Error 1 > > > > I tried to do the handshake with openssl and apparently it works: > > > > [ivan@REDACTED zotonic]$ openssl s_client -host github.com -port 443 > > CONNECTED(00000003) > > depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert > High > > Assurance EV Root CA > > verify return:1 > > depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert > SHA2 > > Extended Validation Server CA > > verify return:1 > > depth=0 businessCategory = Private Organization, > 1.3.6.1.4.1.311.60.2.1.3 = > > US, 1.3.6.1.4.1.311.60.2.1.2 = Delaware, serialNumber = 5157550, street = > > 548 4th Street, postalCode = 94107, C = US, ST = California, L = San > > Francisco, O = "GitHub, Inc.", CN = github.com > > verify return:1 > > --- > > Certificate chain > > 0 s:/businessCategory=Private > > > Organization/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/serialNumber=5157550/street=548 > > 4th Street/postalCode=94107/C=US/ST=California/L=San Francisco/O=GitHub, > > Inc./CN=github.com > > i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended > > Validation Server CA > > 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended > > Validation Server CA > > i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High > Assurance EV > > Root CA > > --- > > Server certificate > > -----BEGIN CERTIFICATE----- > > MIIF4DCCBMigAwIBAgIQDACTENIG2+M3VTWAEY3chzANBgkqhkiG9w0BAQsFADB1 > > ... > > XX4C2NesiZcLYbc2n7B9O+63M2k= > > -----END CERTIFICATE----- > > subject=/businessCategory=Private > > > Organization/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/serialNumber=5157550/street=548 > > 4th Street/postalCode=94107/C=US/ST=California/L=San Francisco/O=GitHub, > > Inc./CN=github.com > > issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 > Extended > > Validation Server CA > > --- > > No client certificate CA names sent > > Server Temp Key: ECDH, prime256v1, 256 bits > > --- > > SSL handshake has read 3233 bytes and written 375 bytes > > --- > > New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256 > > Server public key is 2048 bit > > Secure Renegotiation IS supported > > Compression: NONE > > Expansion: NONE > > SSL-Session: > > Protocol : TLSv1.2 > > Cipher : ECDHE-RSA-AES128-GCM-SHA256 > > Session-ID: > > DDEF6E78852287351EC5B20FFDD2578F8996E7226CB883A5F1A94325048B79C6 > > Session-ID-ctx: > > Master-Key: > > > D6C6283F463BFCD5A160E0CCE0CC8962CF944E5C98153040E4BC20466981B1622A5327C1E6BBED5F1751A049782908E5 > > Key-Arg : None > > Krb5 Principal: None > > PSK identity: None > > PSK identity hint: None > > Start Time: 1411113552 > > Timeout : 300 (sec) > > Verify return code: 0 (ok) > > --- > > closed > > > > What can be wrong?. Thank you. > > Ivan > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From define.null@REDACTED Fri Sep 19 16:57:14 2014 From: define.null@REDACTED (Defnull) Date: Fri, 19 Sep 2014 07:57:14 -0700 (PDT) Subject: [erlang-questions] Schedulers go to sleep even with +sfwi 500 option on R16B03 In-Reply-To: <54260.1411015790@snookles.snookles.com> References: <54260.1411015790@snookles.snookles.com> Message-ID: Thank you Scott! I added '+scl false' option, 1.5 days have passed and it looks good now. Maybe i should also play with '+sub true' because it have similar effect according to docs. Not sure which one will give better performance. Does it also eliminate schedule collapse? ???????, 18 ???????? 2014??., 8:50:11 UTC+4 ???????????? Scott Lystig Fritchie ???????: > Vasily Demidenok wrote: > > > > vd> It helped and activated all schedulers, but only for several hours. > > vd> Then situation repeated. > > > > One way is to periodically check and use the same hack to reset. The > > check doesn't come for free, but at least it avoids manual checking and > > manual intervention. Many thanks to Steve Vinoski for collaboration on > > that code. > > > > https://gist.github.com/slfritchie/23220d2bf61b08b9a8b9 > > > > Meanwhile, the only way that I know of to avoid scheduler collapse is to > > use both the +sfwi and +scl flags together, e.g. "+sfwi 500 +scl > > false". The effect on performance of busy systems varies quite a bit: > > you will want to measure your common workloads yourself to see if the > > medicine is worse than the disease. > > > > Thanks for pointing out the gist about the CouchDB VM version dilemma. > > It's a great summary. > > > > -Scott > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions From mjtruog@REDACTED Fri Sep 19 16:57:28 2014 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 19 Sep 2014 07:57:28 -0700 Subject: [erlang-questions] metaprogramming with -spec In-Reply-To: References: Message-ID: <541C4458.2080303@gmail.com> That link is a bit old. The code is here https://gist.github.com/okeuday/76724b20dca62c27420f On 09/19/2014 05:56 AM, Vlad Dumitrescu wrote: > See also http://erlang.org/pipermail/erlang-questions/2011-March/056896.html > > > On Fri, Sep 19, 2014 at 2:52 PM, Vlad Dumitrescu > wrote: > > On Fri, Sep 19, 2014 at 2:33 PM, semmit mondo > wrote: > > > Is it possible to figure out the spec of functions in a compiled module? > Given a module name I want to be able to know what is the return type > of a function in that module and what are the types of the arguments. > But all I know at the moment is the arity of the function (via module_info). > Is that info lost when compiling? > > > If the module was compiled with debug info, then you can call beam_lib:chunks(Module, [abstract_code]) and traverse the result looking for the spec attributes. > > I would have expected that the specs would be listed in the 'attributes' chunk, but it looks that they aren't. Is there a reason for that? > > regards, > Vlad > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From fritchie@REDACTED Sat Sep 20 06:48:25 2014 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Fri, 19 Sep 2014 23:48:25 -0500 Subject: [erlang-questions] Schedulers go to sleep even with +sfwi 500 option on R16B03 In-Reply-To: Message of "Fri, 19 Sep 2014 07:57:14 PDT." Message-ID: <43820.1411188505@snookles.snookles.com> Defnull wrote: > Maybe i should also play with '+sub true' because it have > similar effect according to docs. Not sure which one will give better > performance. Does it also eliminate schedule collapse? Hrm, no idea, I haven't tried it, sorry -- it's new enough that we at Basho haven't done any evaluation. -Scott From spyl@REDACTED Sat Sep 20 10:32:42 2014 From: spyl@REDACTED (Alexey D. Semiletov) Date: Sat, 20 Sep 2014 15:32:42 +0700 Subject: [erlang-questions] Gun and websockets In-Reply-To: References: Message-ID: I got same problem. Have any news about this issue? Nothing happens after getting message through loop/1 in gun.erl when matching this clause: {ws_upgrade, Owner, Path, Headers} when Protocol =/= gun_spdy -> %% @todo io:format("we are ws_upgrade clause",[]), ProtoState2 = Protocol:ws_upgrade(ProtoState, Path, Headers), io:format("going to loop with ProtoState2~n",[ProtoState2]), ws_loop(State#state{protocol=gun_ws, protocol_state= ProtoState2}); io:format("going to loop with ProtoState2~n",[ProtoState2]) never happens. On 14 August 2014 05:20, Robert Virding wrote: > I had gotten the answer in #erlang chat but missed it. It is depressingly > simple, it is not implemented yet. :-( > > Robert > > > > On 13 August 2014 14:57, Robert Virding wrote: > >> I am having problems trying to open a websocket with gun >> https://github.com/extend/gun. So I have been trying: >> >> {ok,Pid} = gun:open("10.0.1.86", 8080, [{type,tcp}]), >> gun:ws_upgrade(Pid, "/content/chats/connect/"), >> >> and nothing happens, it just times out. Checking, the process Pid started >> by gun:open dies with an undef error but I don't see which function it is >> trying to call. Using http://www.websocket.org/echo.html and opening >> ws://10.0.1.86:8080/content/chats/connect works fine. >> >> Any suggestions? >> >> Robert >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Sat Sep 20 15:07:08 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sat, 20 Sep 2014 15:07:08 +0200 Subject: [erlang-questions] Gun and websockets In-Reply-To: References: Message-ID: Maybe gun should stop advertising WebSocket support when it's not here yet? That's certainly very misleading. Le 20 sept. 2014 ? 10:32, Alexey D. Semiletov a ?crit : > I got same problem. Have any news about this issue? From essen@REDACTED Sat Sep 20 16:16:18 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 20 Sep 2014 16:16:18 +0200 Subject: [erlang-questions] Gun and websockets In-Reply-To: References: Message-ID: <541D8C32.9080900@ninenines.eu> > Le 20 sept. 2014 ? 10:32, Alexey D. Semiletov a ?crit : > >> I got same problem. Have any news about this issue? This isn't a priority at this time. If you need it quicker you can sponsor the task and I will get right on it. On 09/20/2014 03:07 PM, Anthony Ramine wrote: > Maybe gun should stop advertising WebSocket support when it's not here yet? > > That's certainly very misleading. The project hasn't been released yet. -- Lo?c Hoguin http://ninenines.eu From dmytro.lytovchenko@REDACTED Sat Sep 20 16:58:56 2014 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Sat, 20 Sep 2014 16:58:56 +0200 Subject: [erlang-questions] =?utf-8?b?W0FOTl0gdHgg4oCUIGFuIEVybGFuZyB0ZXJt?= =?utf-8?q?_explorer?= Message-ID: Problem: Sometimes when debugging or working on your application you get very long terms which are impossible to print or look through. Solution: tx is a small application which starts an inets webserver on localhost, saves whatever you drop into it in ETS table and and displays URL to open with your browser (Pastebin style). Call tx:show(Term). from Erlang shell, or put it anywhere in your code, tx will be started automatically on the first call. Screenshots, README and code is here https://github.com/kvakvs/tx -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Sat Sep 20 16:28:01 2014 From: t@REDACTED (Tristan Sloughter) Date: Sat, 20 Sep 2014 09:28:01 -0500 Subject: [erlang-questions] Gun and websockets In-Reply-To: <541D8C32.9080900@ninenines.eu> References: <541D8C32.9080900@ninenines.eu> Message-ID: <1411223281.1301349.169758877.28A36B1B@webmail.messagingengine.com> It is public on github. Should at least make it clear it isn't done in the readme. -- Tristan Sloughter t@REDACTED On Sat, Sep 20, 2014, at 09:16 AM, Lo?c Hoguin wrote: > > Le 20 sept. 2014 ? 10:32, Alexey D. Semiletov a > ?crit : > > > >> I got same problem. Have any news about this issue? > > This isn't a priority at this time. If you need it quicker you can > sponsor the task and I will get right on it. > > On 09/20/2014 03:07 PM, Anthony Ramine wrote: > > Maybe gun should stop advertising WebSocket support when it's not here yet? > > > > That's certainly very misleading. > > The project hasn't been released yet. > > -- > Lo?c Hoguin > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From gumm@REDACTED Sat Sep 20 21:27:07 2014 From: gumm@REDACTED (Jesse Gumm) Date: Sat, 20 Sep 2014 14:27:07 -0500 Subject: [erlang-questions] =?utf-8?b?W0FOTl0gdHgg4oCUIGFuIEVybGFuZyB0ZXJt?= =?utf-8?q?_explorer?= In-Reply-To: References: Message-ID: This seems like a useful tool. Thanks for sharing! -Jesse -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm On Sep 20, 2014 9:59 AM, "Dmytro Lytovchenko" wrote: > Problem: > > Sometimes when debugging or working on your application you get very long > terms which are impossible to print or look through. > > Solution: > > tx is a small application which starts an inets webserver on localhost, > saves whatever you drop into it in ETS table and and displays URL to open > with your browser (Pastebin style). Call tx:show(Term). from Erlang shell, > or put it anywhere in your code, tx will be started automatically on the > first call. > > Screenshots, README and code is here https://github.com/kvakvs/tx > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From james.hague@REDACTED Sat Sep 20 22:07:47 2014 From: james.hague@REDACTED (James Hague) Date: Sat, 20 Sep 2014 15:07:47 -0500 Subject: [erlang-questions] Starting EShell via code Message-ID: Under OS X Mavericks and Erlang/OTP 17 I can start a working EShell easily enough from code, but some keys don't behave the same as the default shell: no tab completion, no arrow key movement. What am I missing? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Sat Sep 20 22:21:19 2014 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Sat, 20 Sep 2014 22:21:19 +0200 Subject: [erlang-questions] Starting EShell via code In-Reply-To: References: Message-ID: Afaik there is no easy way to *correctly* start the shell after the VM boots because the user and group processes have already been registered. You always get a limited version of the shell. If someone knows a way of doing so, I would love to hear too. On Saturday, September 20, 2014, James Hague wrote: > Under OS X Mavericks and Erlang/OTP 17 I can start a working EShell easily > enough from code, but some keys don't behave the same as the default shell: > no tab completion, no arrow key movement. > > What am I missing? > -- *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Lead Developer -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Sat Sep 20 22:54:30 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 20 Sep 2014 16:54:30 -0400 Subject: [erlang-questions] Starting EShell via code In-Reply-To: References: Message-ID: <20140920205428.GK4681@ferdmbp.local> Gonna answer both emails at once here. On Saturday, September 20, 2014, James Hague wrote: > Under OS X Mavericks and Erlang/OTP 17 I can start a working EShell easily > enough from code, but some keys don't behave the same as the default shell: > no tab completion, no arrow key movement. > > What am I missing? > How are you starting things? Usually, tab completion and arrow movement is done through a ping pong between edlin.erl and group.erl: https://github.com/blackberry/Erlang-OTP/blob/master/lib/stdlib/src/edlin.erl https://github.com/blackberry/Erlang-OTP/blob/master/lib/kernel/src/group.erl The input and display of a few characters that are broken can be due to having used the old shell (which didn't handle these and was implemented in user.erl, an older and rarely used version of the shell: https://github.com/blackberry/Erlang-OTP/blob/master/lib/kernel/src/user.erl) I tried to document the general structure of the Erlang shell in http://ferd.ca/repl-a-bit-more-and-less-than-that.html. You can read my response to Jos? below where I do mention a way that appears to work decently to programatically start a shell. On 09/20, Jos? Valim wrote: > Afaik there is no easy way to *correctly* start the shell after the > VM boots because the user and group processes have already been registered. > You always get a limited version of the shell. > This can be specified by some special trickery by starting with -nouser, which will start no shell, no user, etc. and then having to substitute in your own mechanisms to try and handle it with the special file descriptor {fd, 0, 1} (stdio, stdout) and {fd, 2, 2}?(stderr), and moving them around. The problem there is that things may also depend on the tty program being around. Something similar is done with 'rebar shell' as a command, which nevertheless hijacks the current IO stuff by killing it, and taking over talking with the tty program -- though these are somewhat constrained by virtue of running in an escript: https://github.com/rebar/rebar/blob/master/src/rebar_shell.erl That stuff still leaves a few things a bit wonky here and there from what I've seen using it, but generally seems to do an okay job. Regards, Fred. From stu.bailey@REDACTED Sun Sep 21 16:46:51 2014 From: stu.bailey@REDACTED (Stu Bailey) Date: Sun, 21 Sep 2014 07:46:51 -0700 Subject: [erlang-questions] Baffling lists Message-ID: I'm sure I'm missing something obvious, but here goes: What is going on here? Is L4 a list? If so, why can't I reverse it? 70> L4. [[<<"8">>,<<"7">>,<<"6">>,<<"5">>], [<<"4">>,<<"3">>,<<"2">>,<<"1">>]| 0] 71> lists:reverse(L4). ** exception error: bad argument in function lists:reverse/2 called as lists:reverse(0, [[<<"4">>,<<"3">>,<<"2">>,<<"1">>], [<<"8">>,<<"7">>,<<"6">>,<<"5">>]]) in call from lists:reverse/1 (lists.erl, line 152) 72> [Head1|Rest1] = L4. [[<<"8">>,<<"7">>,<<"6">>,<<"5">>], [<<"4">>,<<"3">>,<<"2">>,<<"1">>]| 0] 73> Rest1. [[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0] 74> lists:reverse(Rest1). ** exception error: no function clause matching lists:reverse([[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0]) (lists.erl, line 145) 75> [Head2|Rest2] = Rest1. [[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0] 76> Head2. [<<"4">>,<<"3">>,<<"2">>,<<"1">>] 77> Rest2. 0 78> In practice, the kind of structure demonstrated in L4 is being returned by a function from another library I'm trying to use, but I'm trying to understand what's going on in general. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Sun Sep 21 16:51:54 2014 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Sun, 21 Sep 2014 16:51:54 +0200 Subject: [erlang-questions] Baffling lists In-Reply-To: References: Message-ID: It is irregular list, that is list which doesn't have [] as tail element, and has 0 there instead. While its legal to have such a value, some list handling functions will be waiting for [] as tail element thus they will be confused. Solution: don't have 0 there, or cut it away temporarily and append later as X = [CutList | 0]. On Sun, Sep 21, 2014 at 4:46 PM, Stu Bailey wrote: > I'm sure I'm missing something obvious, but here goes: > > What is going on here? Is L4 a list? If so, why can't I reverse it? > > 70> L4. > > [[<<"8">>,<<"7">>,<<"6">>,<<"5">>], > > [<<"4">>,<<"3">>,<<"2">>,<<"1">>]| > > 0] > > 71> lists:reverse(L4). > > ** exception error: bad argument > > in function lists:reverse/2 > > called as lists:reverse(0, > > [[<<"4">>,<<"3">>,<<"2">>,<<"1">>], > > [<<"8">>,<<"7">>,<<"6">>,<<"5">>]]) > > in call from lists:reverse/1 (lists.erl, line 152) > > 72> [Head1|Rest1] = L4. > > [[<<"8">>,<<"7">>,<<"6">>,<<"5">>], > > [<<"4">>,<<"3">>,<<"2">>,<<"1">>]| > > 0] > > 73> Rest1. > > [[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0] > > 74> lists:reverse(Rest1). > > ** exception error: no function clause matching > lists:reverse([[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0]) (lists.erl, line 145) > > 75> [Head2|Rest2] = Rest1. > > [[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0] > > 76> Head2. > > [<<"4">>,<<"3">>,<<"2">>,<<"1">>] > > 77> Rest2. > > 0 > > 78> > > > In practice, the kind of structure demonstrated in L4 is being returned by > a function from another library I'm trying to use, but I'm trying to > understand what's going on in general. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Sun Sep 21 17:00:32 2014 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Sun, 21 Sep 2014 17:00:32 +0200 Subject: [erlang-questions] Baffling lists In-Reply-To: References: Message-ID: Oh and also its list of another list and 0 as tail element. You probably want to reverse first element of L4, like lists:reverse(hd(L4)). On Sun, Sep 21, 2014 at 4:51 PM, Dmytro Lytovchenko < dmytro.lytovchenko@REDACTED> wrote: > It is irregular list, that is list which doesn't have [] as tail element, > and has 0 there instead. While its legal to have such a value, some list > handling functions will be waiting for [] as tail element thus they will be > confused. > Solution: don't have 0 there, or cut it away temporarily and append later > as X = [CutList | 0]. > > On Sun, Sep 21, 2014 at 4:46 PM, Stu Bailey wrote: > >> I'm sure I'm missing something obvious, but here goes: >> >> What is going on here? Is L4 a list? If so, why can't I reverse it? >> >> 70> L4. >> >> [[<<"8">>,<<"7">>,<<"6">>,<<"5">>], >> >> [<<"4">>,<<"3">>,<<"2">>,<<"1">>]| >> >> 0] >> >> 71> lists:reverse(L4). >> >> ** exception error: bad argument >> >> in function lists:reverse/2 >> >> called as lists:reverse(0, >> >> [[<<"4">>,<<"3">>,<<"2">>,<<"1">>], >> >> [<<"8">>,<<"7">>,<<"6">>,<<"5">>]]) >> >> in call from lists:reverse/1 (lists.erl, line 152) >> >> 72> [Head1|Rest1] = L4. >> >> [[<<"8">>,<<"7">>,<<"6">>,<<"5">>], >> >> [<<"4">>,<<"3">>,<<"2">>,<<"1">>]| >> >> 0] >> >> 73> Rest1. >> >> [[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0] >> >> 74> lists:reverse(Rest1). >> >> ** exception error: no function clause matching >> lists:reverse([[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0]) (lists.erl, line 145) >> >> 75> [Head2|Rest2] = Rest1. >> >> [[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0] >> >> 76> Head2. >> >> [<<"4">>,<<"3">>,<<"2">>,<<"1">>] >> >> 77> Rest2. >> >> 0 >> >> 78> >> >> >> In practice, the kind of structure demonstrated in L4 is being returned >> by a function from another library I'm trying to use, but I'm trying to >> understand what's going on in general. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stu.bailey@REDACTED Sun Sep 21 17:07:41 2014 From: stu.bailey@REDACTED (Stu Bailey) Date: Sun, 21 Sep 2014 08:07:41 -0700 Subject: [erlang-questions] Baffling lists In-Reply-To: References: Message-ID: Thanks. I actually wanted to reverse the list of lists lists:reverse(L4). I did not know about "irregular lists". Now I do. Thanks again! On Sun, Sep 21, 2014 at 8:00 AM, Dmytro Lytovchenko < dmytro.lytovchenko@REDACTED> wrote: > Oh and also its list of another list and 0 as tail element. You probably > want to reverse first element of L4, like lists:reverse(hd(L4)). > > On Sun, Sep 21, 2014 at 4:51 PM, Dmytro Lytovchenko < > dmytro.lytovchenko@REDACTED> wrote: > >> It is irregular list, that is list which doesn't have [] as tail element, >> and has 0 there instead. While its legal to have such a value, some list >> handling functions will be waiting for [] as tail element thus they will be >> confused. >> Solution: don't have 0 there, or cut it away temporarily and append later >> as X = [CutList | 0]. >> >> On Sun, Sep 21, 2014 at 4:46 PM, Stu Bailey wrote: >> >>> I'm sure I'm missing something obvious, but here goes: >>> >>> What is going on here? Is L4 a list? If so, why can't I reverse it? >>> >>> 70> L4. >>> >>> [[<<"8">>,<<"7">>,<<"6">>,<<"5">>], >>> >>> [<<"4">>,<<"3">>,<<"2">>,<<"1">>]| >>> >>> 0] >>> >>> 71> lists:reverse(L4). >>> >>> ** exception error: bad argument >>> >>> in function lists:reverse/2 >>> >>> called as lists:reverse(0, >>> >>> [[<<"4">>,<<"3">>,<<"2">>,<<"1">>], >>> >>> [<<"8">>,<<"7">>,<<"6">>,<<"5">>]]) >>> >>> in call from lists:reverse/1 (lists.erl, line 152) >>> >>> 72> [Head1|Rest1] = L4. >>> >>> [[<<"8">>,<<"7">>,<<"6">>,<<"5">>], >>> >>> [<<"4">>,<<"3">>,<<"2">>,<<"1">>]| >>> >>> 0] >>> >>> 73> Rest1. >>> >>> [[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0] >>> >>> 74> lists:reverse(Rest1). >>> >>> ** exception error: no function clause matching >>> lists:reverse([[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0]) (lists.erl, line 145) >>> >>> 75> [Head2|Rest2] = Rest1. >>> >>> [[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0] >>> >>> 76> Head2. >>> >>> [<<"4">>,<<"3">>,<<"2">>,<<"1">>] >>> >>> 77> Rest2. >>> >>> 0 >>> >>> 78> >>> >>> >>> In practice, the kind of structure demonstrated in L4 is being returned >>> by a function from another library I'm trying to use, but I'm trying to >>> understand what's going on in general. >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stu.bailey@REDACTED Sun Sep 21 17:52:18 2014 From: stu.bailey@REDACTED (Stu Bailey) Date: Sun, 21 Sep 2014 08:52:18 -0700 Subject: [erlang-questions] Baffling lists In-Reply-To: References: Message-ID: Thanks to another comment, I see this is "cons" at work making lists where the last element is not a list, as Dmytro said. I got it now. 130> [[hello] | hello]. [[hello]|hello] 131> [world,[hello] | hello]. [world,[hello]|hello] 132> List5 = [world,[hello] | hello]. [world,[hello]|hello] 133> lists:reverse(List5). ** exception error: bad argument in function lists:reverse/2 called as lists:reverse(hello,[[hello],world]) in call from lists:reverse/1 (lists.erl, line 152) 134> List5. [world,[hello]|hello] 135> is_list(List5). true 136> That is a bit quirky, but hey it's Erlang! The quick hack for my code was to replace "lists:reverse(L4)" with my little strip_lists_rev(L4). strip_lists_rev(0,Acc)-> Acc; strip_lists_rev([H|Rest],Acc)-> strip_lists_rev(Rest,[H|Acc]). Hopefully, this won't impact performance and reliability too much. Thanks again. On Sun, Sep 21, 2014 at 8:07 AM, Stu Bailey wrote: > Thanks. I actually wanted to reverse the list of lists > lists:reverse(L4). I did not know about "irregular lists". Now I do. > > Thanks again! > > > On Sun, Sep 21, 2014 at 8:00 AM, Dmytro Lytovchenko < > dmytro.lytovchenko@REDACTED> wrote: > >> Oh and also its list of another list and 0 as tail element. You probably >> want to reverse first element of L4, like lists:reverse(hd(L4)). >> >> On Sun, Sep 21, 2014 at 4:51 PM, Dmytro Lytovchenko < >> dmytro.lytovchenko@REDACTED> wrote: >> >>> It is irregular list, that is list which doesn't have [] as tail >>> element, and has 0 there instead. While its legal to have such a value, >>> some list handling functions will be waiting for [] as tail element thus >>> they will be confused. >>> Solution: don't have 0 there, or cut it away temporarily and append >>> later as X = [CutList | 0]. >>> >>> On Sun, Sep 21, 2014 at 4:46 PM, Stu Bailey >>> wrote: >>> >>>> I'm sure I'm missing something obvious, but here goes: >>>> >>>> What is going on here? Is L4 a list? If so, why can't I reverse it? >>>> >>>> 70> L4. >>>> >>>> [[<<"8">>,<<"7">>,<<"6">>,<<"5">>], >>>> >>>> [<<"4">>,<<"3">>,<<"2">>,<<"1">>]| >>>> >>>> 0] >>>> >>>> 71> lists:reverse(L4). >>>> >>>> ** exception error: bad argument >>>> >>>> in function lists:reverse/2 >>>> >>>> called as lists:reverse(0, >>>> >>>> [[<<"4">>,<<"3">>,<<"2">>,<<"1">>], >>>> >>>> [<<"8">>,<<"7">>,<<"6">>,<<"5">>]]) >>>> >>>> in call from lists:reverse/1 (lists.erl, line 152) >>>> >>>> 72> [Head1|Rest1] = L4. >>>> >>>> [[<<"8">>,<<"7">>,<<"6">>,<<"5">>], >>>> >>>> [<<"4">>,<<"3">>,<<"2">>,<<"1">>]| >>>> >>>> 0] >>>> >>>> 73> Rest1. >>>> >>>> [[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0] >>>> >>>> 74> lists:reverse(Rest1). >>>> >>>> ** exception error: no function clause matching >>>> lists:reverse([[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0]) (lists.erl, line 145) >>>> >>>> 75> [Head2|Rest2] = Rest1. >>>> >>>> [[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0] >>>> >>>> 76> Head2. >>>> >>>> [<<"4">>,<<"3">>,<<"2">>,<<"1">>] >>>> >>>> 77> Rest2. >>>> >>>> 0 >>>> >>>> 78> >>>> >>>> >>>> In practice, the kind of structure demonstrated in L4 is being returned >>>> by a function from another library I'm trying to use, but I'm trying to >>>> understand what's going on in general. >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhuoyikang@REDACTED Sun Sep 21 16:38:35 2014 From: zhuoyikang@REDACTED (=?UTF-8?B?5Y2T5LiA5oqX?=) Date: Sun, 21 Sep 2014 22:38:35 +0800 Subject: [erlang-questions] Erlang segfault ?? OMG Message-ID: hello?everybody : my erlang node crash and dmesg report: segfault at 18 ip 00000000004565f4 sp 00007fffe9178698 error 4 in beam[400000+1e9000] and i objdump -d beam to track this code : 456597: 74 5b je 4565f4 456599: 48 8b 71 08 mov 0x8(%rcx),%rsi 45659d: 40 f6 c6 02 test $0x2,%sil 4565a1: 74 51 je 4565f4 4565f4: 48 8b 41 18 mov 0x18(%rcx),%rax so simple code => void erts_bfalc_init(void) { atoms_initialized = 0; } .//erts/emulator/beam/erl_bestfit_alloc.h:39:void erts_bfalc_init(void); does it a bug on erlang rb15b02? and anyone occur the same question like me ? I need some help..... -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.joseph.davis@REDACTED Sun Sep 21 20:24:58 2014 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Sun, 21 Sep 2014 13:24:58 -0500 Subject: [erlang-questions] Starting EShell via code In-Reply-To: <20140920205428.GK4681@ferdmbp.local> References: <20140920205428.GK4681@ferdmbp.local> Message-ID: Here's a working example based on a talk by Geoff Cant. https://gist.github.com/davisp/439e29ced1e6a9d2b108 I was playing around trying to get tab completion working based off the code loaded in the remote node. I haven't used it extensively but tab completion and arrow keys move just fine. Paul On Sat, Sep 20, 2014 at 3:54 PM, Fred Hebert wrote: > Gonna answer both emails at once here. > > On Saturday, September 20, 2014, James Hague wrote: >> Under OS X Mavericks and Erlang/OTP 17 I can start a working EShell easily >> enough from code, but some keys don't behave the same as the default shell: >> no tab completion, no arrow key movement. >> >> What am I missing? >> > > How are you starting things? Usually, tab completion and arrow movement > is done through a ping pong between edlin.erl and group.erl: > > https://github.com/blackberry/Erlang-OTP/blob/master/lib/stdlib/src/edlin.erl > https://github.com/blackberry/Erlang-OTP/blob/master/lib/kernel/src/group.erl > > The input and display of a few characters that are broken can be due to > having used the old shell (which didn't handle these and was implemented > in user.erl, an older and rarely used version of the shell: > https://github.com/blackberry/Erlang-OTP/blob/master/lib/kernel/src/user.erl) > > I tried to document the general structure of the Erlang shell in > http://ferd.ca/repl-a-bit-more-and-less-than-that.html. > > You can read my response to Jos? below where I do mention a way that > appears to work decently to programatically start a shell. > > On 09/20, Jos? Valim wrote: >> Afaik there is no easy way to *correctly* start the shell after the >> VM boots because the user and group processes have already been registered. >> You always get a limited version of the shell. >> > > This can be specified by some special trickery by starting with -nouser, > which will start no shell, no user, etc. and then having to substitute > in your own mechanisms to try and handle it with the special file > descriptor {fd, 0, 1} (stdio, stdout) and {fd, 2, 2} (stderr), and > moving them around. > > The problem there is that things may also depend on the tty program > being around. > > Something similar is done with 'rebar shell' as a command, which > nevertheless hijacks the current IO stuff by killing it, and taking over > talking with the tty program -- though these are somewhat constrained by > virtue of running in an escript: > https://github.com/rebar/rebar/blob/master/src/rebar_shell.erl > > That stuff still leaves a few things a bit wonky here and there from > what I've seen using it, but generally seems to do an okay job. > > > Regards, > Fred. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lukas@REDACTED Mon Sep 22 10:06:52 2014 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 22 Sep 2014 10:06:52 +0200 Subject: [erlang-questions] Erlang segfault ?? OMG In-Reply-To: References: Message-ID: Hello, This is not a known issue. You will have to provide a little more information in order for me to understand what is going on. * From the place where the error happens I assume that this is during start up. Is that a correct assumption? * Is it possible to reproduce? * Can you get a full core file? * What OS are you using? * How did you install Erlang/OTP? * Are you running any non-Erlang/OTP nifs/linked-in drivers? Lukas On Sun, Sep 21, 2014 at 4:38 PM, ??? wrote: > hello?everybody : > > my erlang node crash and dmesg report: > > segfault at 18 ip 00000000004565f4 sp 00007fffe9178698 error 4 in > beam[400000+1e9000] > > and i objdump -d beam to track this code : > > 456597: 74 5b je 4565f4 > > 456599: 48 8b 71 08 mov 0x8(%rcx),%rsi > 45659d: 40 f6 c6 02 test $0x2,%sil > 4565a1: 74 51 je 4565f4 > > 4565f4: 48 8b 41 18 mov 0x18(%rcx),%rax > > so simple code => > > void > erts_bfalc_init(void) > { > atoms_initialized = 0; > } > .//erts/emulator/beam/erl_bestfit_alloc.h:39:void erts_bfalc_init(void); > > > does it a bug on erlang rb15b02? and anyone occur the same question like > me ? > > I need some help..... > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhuoyikang@REDACTED Mon Sep 22 10:19:53 2014 From: zhuoyikang@REDACTED (=?UTF-8?B?5Y2T5LiA5oqX?=) Date: Mon, 22 Sep 2014 16:19:53 +0800 Subject: [erlang-questions] Erlang segfault ?? OMG In-Reply-To: References: Message-ID: thanks * From the place where the error happens I assume that this is during start up. Is that a correct assumption? => it's not start up , my program run few hours and get the segment fault. * Is it possible to reproduce? sorry, i try to reproduct it in qa environmnet ,but having success.. * Can you get a full core file? => the core file is too large , more 2G , it not write successfully.. * What OS are you using? => ubuntu 12.04 * How did you install Erlang/OTP? => R1502 - download from https://packages.erlang-solutions.com/erlang/ http://packages.erlang-solutions.com/erlang/esl-erlang/FLAVOUR_1_general/esl-erlang_15.b.2~ubuntu~lucid_amd64.deb * Are you running any non-Erlang/OTP nifs/linked-in drivers? not nifs.... competely erlang/otp 2014-09-22 16:06 GMT+08:00 Lukas Larsson : > Hello, > > This is not a known issue. > > You will have to provide a little more information in order for me to > understand what is going on. > > * From the place where the error happens I assume that this is during > start up. Is that a correct assumption? > * Is it possible to reproduce? > * Can you get a full core file? > * What OS are you using? > * How did you install Erlang/OTP? > * Are you running any non-Erlang/OTP nifs/linked-in drivers? > > Lukas > > On Sun, Sep 21, 2014 at 4:38 PM, ??? wrote: > >> hello?everybody : >> >> my erlang node crash and dmesg report: >> >> segfault at 18 ip 00000000004565f4 sp 00007fffe9178698 error 4 in >> beam[400000+1e9000] >> >> and i objdump -d beam to track this code : >> >> 456597: 74 5b je 4565f4 >> >> 456599: 48 8b 71 08 mov 0x8(%rcx),%rsi >> 45659d: 40 f6 c6 02 test $0x2,%sil >> 4565a1: 74 51 je 4565f4 >> >> 4565f4: 48 8b 41 18 mov 0x18(%rcx),%rax >> >> so simple code => >> >> void >> erts_bfalc_init(void) >> { >> atoms_initialized = 0; >> } >> .//erts/emulator/beam/erl_bestfit_alloc.h:39:void erts_bfalc_init(void); >> >> >> does it a bug on erlang rb15b02? and anyone occur the same question >> like me ? >> >> I need some help..... >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Sep 22 10:50:19 2014 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 22 Sep 2014 20:50:19 +1200 Subject: [erlang-questions] Baffling lists In-Reply-To: References: Message-ID: On 22/09/2014, at 3:07 AM, Stu Bailey wrote: > Thanks. I actually wanted to reverse the list of lists lists:reverse(L4). I did not know about "irregular lists". Now I do. In the Prolog world they're usually called "improper lists". In fact, that's what they are called in Erlang too. From http://www.erlang.org/doc/reference_manual/typespec.html List :: list(Type) %% Proper list ([]-terminated) | maybe_improper_list(Type1, Type2) %% Type1=contents, Type2=termination | nonempty_improper_list(Type1, Type2) %% Type1 and Type2 as above | nonempty_list(Type) %% Proper non-empty list list() = [any()] maybe_improper_list() = maybe_improper_list(any(), any()) nonempty_list() = nonempty_list(any()) "What's going on in general" is that "list" is not a primitive data structure in Erlang (or Prolog, or Lisp, where it all started). [] is a primitive data structure. A pair [H|T] is a primitive data structure. But a list is just one way to build things out of pairs. See iolist, for example: iolist() = maybe_improper_list(byte() | binary() | iolist(), binary() | []) From lukas@REDACTED Mon Sep 22 10:49:51 2014 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 22 Sep 2014 10:49:51 +0200 Subject: [erlang-questions] Erlang segfault ?? OMG In-Reply-To: References: Message-ID: On Mon, Sep 22, 2014 at 10:19 AM, ??? wrote: > > * From the place where the error happens I assume that this is during > start up. Is that a correct assumption? => > > it's not start up , my program run few hours and get the segment fault. > Then the error did not happen in erts_bfalc_init as that is only called during startup before any schedulers are even started. > > * Can you get a full core file? > > => the core file is too large , more 2G , it not write successfully.. > > It should be possible to get quite a lot of information out of a partial core. Have you tried just setting ulimit -c to 1 GB and then check if you can get any info from that core? > > > * Are you running any non-Erlang/OTP nifs/linked-in drivers? > > not nifs.... competely erlang/otp > Are you 100% sure? No third party libs use nifs/linked-in drivers? This type of error almost always turn out to be nif/linked-in driver related. > > 2014-09-22 16:06 GMT+08:00 Lukas Larsson : > >> Hello, >> >> This is not a known issue. >> >> You will have to provide a little more information in order for me to >> understand what is going on. >> >> * From the place where the error happens I assume that this is during >> start up. Is that a correct assumption? >> * Is it possible to reproduce? >> * Can you get a full core file? >> * What OS are you using? >> * How did you install Erlang/OTP? >> * Are you running any non-Erlang/OTP nifs/linked-in drivers? >> >> Lukas >> >> On Sun, Sep 21, 2014 at 4:38 PM, ??? wrote: >> >>> hello?everybody : >>> >>> my erlang node crash and dmesg report: >>> >>> segfault at 18 ip 00000000004565f4 sp 00007fffe9178698 error 4 in >>> beam[400000+1e9000] >>> >>> and i objdump -d beam to track this code : >>> >>> 456597: 74 5b je 4565f4 >>> >>> 456599: 48 8b 71 08 mov 0x8(%rcx),%rsi >>> 45659d: 40 f6 c6 02 test $0x2,%sil >>> 4565a1: 74 51 je 4565f4 >>> >>> 4565f4: 48 8b 41 18 mov 0x18(%rcx),%rax >>> >>> so simple code => >>> >>> void >>> erts_bfalc_init(void) >>> { >>> atoms_initialized = 0; >>> } >>> .//erts/emulator/beam/erl_bestfit_alloc.h:39:void >>> erts_bfalc_init(void); >>> >>> >>> does it a bug on erlang rb15b02? and anyone occur the same question >>> like me ? >>> >>> I need some help..... >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenji@REDACTED Mon Sep 22 13:34:17 2014 From: kenji@REDACTED (Kenji Rikitake) Date: Mon, 22 Sep 2014 20:34:17 +0900 Subject: [erlang-questions] [ANN] xorshift128+ and xorshift64* PRNGs for Erlang Message-ID: I've released yet another non-cryptographic random number generator implementations for Erlang. These are still in the alpha phase, so bug reports and enhancements welcome. xorshift+ and xorshift* pseudo random number generators (PRNGs) look promising as faster and more compact alternatives to Mersenne Twister PRNGs for 64-bit execution environments. Here are the implementations in Erlang without NIFs, with the MIT license: xorshift+128 (exsplus) https://github.com/jj1bdx/exsplus/ xorshift*64 (exs64) https://github.com/jj1bdx/exs64/ xorshift+128 and xorshift*64 both generate 64-bit integer sequences. xorshift+128 has two 64-bit integers for the internal state, and the period is (2^127-1), the same as tinymt-erlang. The execution time of exsplus is ~x1.5 of tinymt-erlang (which generates 32-bit integer sequences) on OTP 17.3 with HiPE on OS X x86_64 and FreeBSD amd64 environments. xorshift*64 has one 64-bit integer for the internal state, and the period is (2^64-1). The execution time of exs64 is roughly the same but slightly faster than exsplus. Note: the random module algorithm (AS183), designed in 1980s, has only three 15-bit integers for the internal state, and the period is ~(2^43). See http://xorshift.di.unimi.it/ for the further details of xorshift+ and xorshift* algorithms. Regards, Kenji Rikitake -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Mon Sep 22 14:21:36 2014 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 22 Sep 2014 07:21:36 -0500 Subject: [erlang-questions] Baffling lists In-Reply-To: References: Message-ID: Re performance: The 0 should have not been at the end of the list in the first place. It is the program that generated the list that is wrong and should be changed, not the program that consumes the list ... Do this and *both* programs might be faster. It's not really a good idea to have "improper" lists - so I'd check the code base to see this doesn't happen - there are rare "I'll save a word" cases where you might want a non proper list - but the pain of not being able to use standard libraries will outweigh the pleasure. /Joe On Sun, Sep 21, 2014 at 10:52 AM, Stu Bailey wrote: > Thanks to another comment, I see this is "cons" at work making lists where > the last element is not a list, as Dmytro said. I got it now. > > 130> [[hello] | hello]. > > [[hello]|hello] > > 131> [world,[hello] | hello]. > > [world,[hello]|hello] > > 132> List5 = [world,[hello] | hello]. > > [world,[hello]|hello] > > 133> lists:reverse(List5). > > ** exception error: bad argument > > in function lists:reverse/2 > > called as lists:reverse(hello,[[hello],world]) > > in call from lists:reverse/1 (lists.erl, line 152) > > 134> List5. > > [world,[hello]|hello] > > 135> is_list(List5). > > true > > 136> > > > That is a bit quirky, but hey it's Erlang! The quick hack for my code was > to replace "lists:reverse(L4)" with my little strip_lists_rev(L4). > > > strip_lists_rev(0,Acc)-> > Acc; > strip_lists_rev([H|Rest],Acc)-> > strip_lists_rev(Rest,[H|Acc]). > > > Hopefully, this won't impact performance and reliability too much. > > Thanks again. > > On Sun, Sep 21, 2014 at 8:07 AM, Stu Bailey wrote: >> >> Thanks. I actually wanted to reverse the list of lists lists:reverse(L4). >> I did not know about "irregular lists". Now I do. >> >> Thanks again! >> >> >> On Sun, Sep 21, 2014 at 8:00 AM, Dmytro Lytovchenko >> wrote: >>> >>> Oh and also its list of another list and 0 as tail element. You probably >>> want to reverse first element of L4, like lists:reverse(hd(L4)). >>> >>> On Sun, Sep 21, 2014 at 4:51 PM, Dmytro Lytovchenko >>> wrote: >>>> >>>> It is irregular list, that is list which doesn't have [] as tail >>>> element, and has 0 there instead. While its legal to have such a value, some >>>> list handling functions will be waiting for [] as tail element thus they >>>> will be confused. >>>> Solution: don't have 0 there, or cut it away temporarily and append >>>> later as X = [CutList | 0]. >>>> >>>> On Sun, Sep 21, 2014 at 4:46 PM, Stu Bailey >>>> wrote: >>>>> >>>>> I'm sure I'm missing something obvious, but here goes: >>>>> >>>>> What is going on here? Is L4 a list? If so, why can't I reverse it? >>>>> >>>>> 70> L4. >>>>> >>>>> [[<<"8">>,<<"7">>,<<"6">>,<<"5">>], >>>>> >>>>> [<<"4">>,<<"3">>,<<"2">>,<<"1">>]| >>>>> >>>>> 0] >>>>> >>>>> 71> lists:reverse(L4). >>>>> >>>>> ** exception error: bad argument >>>>> >>>>> in function lists:reverse/2 >>>>> >>>>> called as lists:reverse(0, >>>>> >>>>> [[<<"4">>,<<"3">>,<<"2">>,<<"1">>], >>>>> >>>>> [<<"8">>,<<"7">>,<<"6">>,<<"5">>]]) >>>>> >>>>> in call from lists:reverse/1 (lists.erl, line 152) >>>>> >>>>> 72> [Head1|Rest1] = L4. >>>>> >>>>> [[<<"8">>,<<"7">>,<<"6">>,<<"5">>], >>>>> >>>>> [<<"4">>,<<"3">>,<<"2">>,<<"1">>]| >>>>> >>>>> 0] >>>>> >>>>> 73> Rest1. >>>>> >>>>> [[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0] >>>>> >>>>> 74> lists:reverse(Rest1). >>>>> >>>>> ** exception error: no function clause matching >>>>> lists:reverse([[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0]) (lists.erl, line 145) >>>>> >>>>> 75> [Head2|Rest2] = Rest1. >>>>> >>>>> [[<<"4">>,<<"3">>,<<"2">>,<<"1">>]|0] >>>>> >>>>> 76> Head2. >>>>> >>>>> [<<"4">>,<<"3">>,<<"2">>,<<"1">>] >>>>> >>>>> 77> Rest2. >>>>> >>>>> 0 >>>>> >>>>> 78> >>>>> >>>>> >>>>> In practice, the kind of structure demonstrated in L4 is being returned >>>>> by a function from another library I'm trying to use, but I'm trying to >>>>> understand what's going on in general. >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> >>> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf@REDACTED Mon Sep 22 15:02:13 2014 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 22 Sep 2014 15:02:13 +0200 Subject: [erlang-questions] Baffling lists In-Reply-To: References: Message-ID: On 22 Sep 2014, at 14:21, Joe Armstrong wrote: > It's not really a good idea to have "improper" lists - so I'd check > the code base to > see this doesn't happen - there are rare "I'll save a word" cases where you > might want a non proper list - but the pain of not being able to use standard > libraries will outweigh the pleasure. A bit O.T. ... There is one particular place where improper lists are extremely useful: in the match head of an ets:select(). This allows you to perform a prefix search on list-structured keys, for example: ets:select(Tab, [ { #stats{[requests, http | ?_?], _ = ?_?}, [], [?$_?] }]) The main problem with doing this is that Dialyzer will have opinions about both improper lists being used and ?_? violating the type signature for the record. I understand the problem (match specifications are data - not patterns), but it?s still annoying. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From eranga.erl@REDACTED Mon Sep 22 16:11:56 2014 From: eranga.erl@REDACTED (Eranga Udesh) Date: Mon, 22 Sep 2014 19:41:56 +0530 Subject: [erlang-questions] Erlang Memory Question Message-ID: Hi, I'm trying to optimize my memory consumption in Erlang VM and to garbage collect as soon as possible. Let's say I have a large object, "X", After some processing, I only need to work on small part of X, called "x". Can someone advice me if below process flow will put the large object X in to garbage collection, while waiting for the long running job to continue? function1() -> X = ... fetch a large object.... ... some processing... x = ... extract a part of X... ... long running job.... If it's not putting X into garbage collection, does below change do that? function1() -> X = ... fetch a large object.... ... some processing... x = ... extract a part of X... function2(x). function2(x) -> ... long running job... Tks, - Eranga -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidnwelton@REDACTED Mon Sep 22 16:50:24 2014 From: davidnwelton@REDACTED (David Welton) Date: Mon, 22 Sep 2014 16:50:24 +0200 Subject: [erlang-questions] Strange crash and hang In-Reply-To: References: Message-ID: > We're getting an odd crash that I suspect is related to one we had in > the past, where we were getting a bad term from a C node. This hypothesis seems to be correct. I hacked up a compile of Erlang to print something in bad_dist_ext, so I know it's hitting that. Now, to figure out what's going wrong in the C++ code sending the bad term... but that's not an Erlang problem. I hacked up a little C code that can crash Erlang if it's allowed to connect, to demonstrate how badly things go when Erlang gets a large bad term: https://github.com/erlang/otp/pull/359#issuecomment-56383778 -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From tty.erlang@REDACTED Mon Sep 22 18:54:07 2014 From: tty.erlang@REDACTED (T Ty) Date: Mon, 22 Sep 2014 17:54:07 +0100 Subject: [erlang-questions] [ANN] RSS Feedparser 0.1.0 Message-ID: Hello, https://github.com/ttyerlsol/rss A basic RSS v2.0 feedparser. There doesn't seem to be one for Erlang and I needed it to pull popular Delicious tags. Enjoy, tty -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Sep 22 19:56:11 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 22 Sep 2014 19:56:11 +0200 Subject: [erlang-questions] Erlang Memory Question In-Reply-To: References: Message-ID: In general: No it won't. GC will trigger once the process has allocated enough data. If your processing is allocating enough data, this will quickly happen and you don't have to worry. If not, you may have to gently persuade the process to do so. There are several ways: * Set fullsweep_after on the process with a low value (0) and run erlang:garbage_collect() * Go through a hibernation with an immediate wakeup (feels ugly to me) * Handle fetching and extraction in a separate process and send the extracted part 'x' back as a message. This will free up memory almost immediately for other processes to use (simple and elegant) * Don't fetch the large object in the first place, but make it possible to ask for an extraction only. Or stream data a bit at a time and handle the stream in chunks rather than everything at once. On Mon, Sep 22, 2014 at 4:11 PM, Eranga Udesh wrote: > Hi, > > I'm trying to optimize my memory consumption in Erlang VM and to garbage > collect as soon as possible. > > Let's say I have a large object, "X", After some processing, I only need > to work on small part of X, called "x". > > Can someone advice me if below process flow will put the large object X in > to garbage collection, while waiting for the long running job to continue? > > function1() -> > > X = ... fetch a large object.... > > ... some processing... > > x = ... extract a part of X... > ... long running job.... > > > If it's not putting X into garbage collection, does below change do that? > > function1() -> > > X = ... fetch a large object.... > > ... some processing... > > x = ... extract a part of X... > function2(x). > > function2(x) -> > > ... long running job... > > > Tks, > - Eranga > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From federico.carrone@REDACTED Mon Sep 22 20:23:59 2014 From: federico.carrone@REDACTED (Federico Carrone) Date: Mon, 22 Sep 2014 15:23:59 -0300 Subject: [erlang-questions] Invoke shell from command test in case of error Message-ID: In many languages, for example pry/plymouth in Ruby or pytest in Python, you can configure the test framework so that it invokes the interpreter or shell on failure and you can inspect the program state. I would like to know if this is possible in Erlang using common test. Regards, Federico. -- http://federicocarrone.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dpezely@REDACTED Mon Sep 22 20:39:48 2014 From: dpezely@REDACTED (Daniel Pezely) Date: Mon, 22 Sep 2014 18:39:48 +0000 Subject: [erlang-questions] Emacs 'erlang-mode-hook Message-ID: > have Emacs erlang-shell play nice with multiple components > built by rebar (invoked from Makefiles). In case anyone was using the earlier 'erlang-mode-hook for Emacs, here is a corrected version: ;; Add paths to ../../*/deps/ebin for running & compiling: (add-hook 'erlang-mode-hook (lambda () (setq inferior-erlang-machine-options (append (remove nil (mapcan (lambda (app-path) (let ((ebin (concat app-path "/ebin"))) (when (file-readable-p ebin) (list "-pa" ebin)))) (directory-files "../.." t "[^.]$"))) (remove nil (mapcan (lambda (dep) (let ((dep-path (concat dep "/deps"))) (when (file-readable-p dep-path) (mapcan (lambda (dir) (list "-pz" (concat dir "/ebin"))) (directory-files dep-path t "[^.]$"))))) (directory-files "../.." t "[^.]$")))) erlang-compile-extra-opts (append (remove nil (mapcan (lambda (app-path) (let ((ebin (concat app-path "/ebin"))) (when (file-readable-p ebin) (list 'i ebin)))) (directory-files "../.." t "[^.]$"))) (remove nil (mapcan (lambda (dep) (let ((dep-path (concat dep "/deps"))) (when (file-readable-p dep-path) (mapcar (lambda (dir) (cons 'i (concat dir "/ebin"))) (directory-files dep-path t "[^.]$"))))) (directory-files "../.." t "[^.]$"))))))) Further context is within original message. See http://erlang.org/pipermail/erlang-questions/2014-July/080149.html -Daniel -- dpezely@REDACTED From wallentin.dahlberg@REDACTED Mon Sep 22 20:40:13 2014 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Mon, 22 Sep 2014 20:40:13 +0200 Subject: [erlang-questions] Erlang Memory Question In-Reply-To: References: Message-ID: It should suffice by running an erlang:garbage_collect/0 directly after extracting some parts of the structure: function1() -> X1 = ... fetch a large object .... ... some processing... X2 = ... extract a part of X1 ... erlang:garbage_collect(), ... long running job.... As long as the object of X1 is not referenced by the process after the explicit call to the gc, you're fine. Note also, an explicit call to garbage_collect/0 will always to a 'fullsweep'. Ofc, as Jesper mentioned, the it's probably preferable not to fetch the whole object if possible. // Bj?rn-Egil 2014-09-22 19:56 GMT+02:00 Jesper Louis Andersen < jesper.louis.andersen@REDACTED>: > In general: No it won't. GC will trigger once the process has allocated > enough data. If your processing is allocating enough data, this will > quickly happen and you don't have to worry. If not, you may have to gently > persuade the process to do so. There are several ways: > > * Set fullsweep_after on the process with a low value (0) and run > erlang:garbage_collect() > * Go through a hibernation with an immediate wakeup (feels ugly to me) > * Handle fetching and extraction in a separate process and send the > extracted part 'x' back as a message. This will free up memory almost > immediately for other processes to use (simple and elegant) > * Don't fetch the large object in the first place, but make it possible to > ask for an extraction only. Or stream data a bit at a time and handle the > stream in chunks rather than everything at once. > > > On Mon, Sep 22, 2014 at 4:11 PM, Eranga Udesh > wrote: > >> Hi, >> >> I'm trying to optimize my memory consumption in Erlang VM and to garbage >> collect as soon as possible. >> >> Let's say I have a large object, "X", After some processing, I only need >> to work on small part of X, called "x". >> >> Can someone advice me if below process flow will put the large object X >> in to garbage collection, while waiting for the long running job to >> continue? >> >> function1() -> >> >> X = ... fetch a large object.... >> >> ... some processing... >> >> x = ... extract a part of X... >> ... long running job.... >> >> >> If it's not putting X into garbage collection, does below change do that? >> >> function1() -> >> >> X = ... fetch a large object.... >> >> ... some processing... >> >> x = ... extract a part of X... >> function2(x). >> >> function2(x) -> >> >> ... long running job... >> >> >> Tks, >> - Eranga >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > J. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From juan.facorro@REDACTED Tue Sep 23 00:34:26 2014 From: juan.facorro@REDACTED (=?UTF-8?B?SnVhbiBNYXJ0w61u?=) Date: Mon, 22 Sep 2014 19:34:26 -0300 Subject: [erlang-questions] Server Configuration Guidelines, Tips & Tricks - Anywhere? Message-ID: Hi all, While tuning an ubuntu server to run an Erlang web application (i.e. setting open file limits, TCP connections, etc.) we started wondering if there is some resource that explains how and what needs to be done to squeeze the performance out of a server, or just general considerations and pointers to keep in mind when preparing a server for production. Something like the following but Erlang servers would be great http://docs.neo4j.org/chunked/stable/kernel-configuration.html. Thanks! -- Juan Facorro -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenji@REDACTED Tue Sep 23 03:55:51 2014 From: kenji@REDACTED (Kenji Rikitake) Date: Tue, 23 Sep 2014 10:55:51 +0900 Subject: [erlang-questions] [ANN] xorshift1024* PRNG for Erlang In-Reply-To: References: Message-ID: <20140923015551.GA29759@k2r.org> I've released one more non-cryptographic random number generator implementations for Erlang. This is still in the alpha phase, so bug reports and enhancements welcome. xorshift1024* (exs1024) https://github.com/jj1bdx/exs1024/ xorshift+ and xorshift* pseudo random number generators (PRNGs) look promising as faster and more compact alternatives to Mersenne Twister PRNGs for 64-bit execution environments. The implementations is in Erlang without NIFs, with the MIT license. xorshift1024* generates 64-bit integer sequences. xorshift1024* has sixteen (16) 64-bit integers for the internal state, and the period is (2^1024-1), much longer than tinymt-erlang, and comparatively a good alternative against sfmt-erlang. The execution time of exs1024 is ~x2 of random module on OTP 17.3 with HiPE on OS X x86_64 and FreeBSD amd64 environments. See http://xorshift.di.unimi.it/ for the further details of xorshift+ and xorshift* algorithms. Regards, Kenji Rikitake From eranga.erl@REDACTED Tue Sep 23 04:24:09 2014 From: eranga.erl@REDACTED (Eranga Udesh) Date: Tue, 23 Sep 2014 07:54:09 +0530 Subject: [erlang-questions] Erlang Memory Question In-Reply-To: References: Message-ID: Thanks for the information received so far. Wouldn't it be good for Erlang to have a single object garbage collection function/bif? For example, when I no longer require a large object, I force to garbage collect it, without making a full sweep? As mentioned in the document, a full sweep may degrade the performance. - Eranga On Tue, Sep 23, 2014 at 12:10 AM, Bj?rn-Egil Dahlberg < wallentin.dahlberg@REDACTED> wrote: > It should suffice by running an erlang:garbage_collect/0 directly after > extracting some parts of the structure: > > function1() -> > > X1 = ... fetch a large object .... > > ... some processing... > > X2 = ... extract a part of X1 ... > erlang:garbage_collect(), > ... long running job.... > > > As long as the object of X1 is not referenced by the process after the > explicit call to the gc, you're fine. > Note also, an explicit call to garbage_collect/0 will always to a > 'fullsweep'. > > Ofc, as Jesper mentioned, the it's probably preferable not to fetch the > whole object if possible. > > // Bj?rn-Egil > > > 2014-09-22 19:56 GMT+02:00 Jesper Louis Andersen < > jesper.louis.andersen@REDACTED>: > >> In general: No it won't. GC will trigger once the process has allocated >> enough data. If your processing is allocating enough data, this will >> quickly happen and you don't have to worry. If not, you may have to gently >> persuade the process to do so. There are several ways: >> >> * Set fullsweep_after on the process with a low value (0) and run >> erlang:garbage_collect() >> * Go through a hibernation with an immediate wakeup (feels ugly to me) >> * Handle fetching and extraction in a separate process and send the >> extracted part 'x' back as a message. This will free up memory almost >> immediately for other processes to use (simple and elegant) >> * Don't fetch the large object in the first place, but make it possible >> to ask for an extraction only. Or stream data a bit at a time and handle >> the stream in chunks rather than everything at once. >> >> >> On Mon, Sep 22, 2014 at 4:11 PM, Eranga Udesh >> wrote: >> >>> Hi, >>> >>> I'm trying to optimize my memory consumption in Erlang VM and to garbage >>> collect as soon as possible. >>> >>> Let's say I have a large object, "X", After some processing, I only need >>> to work on small part of X, called "x". >>> >>> Can someone advice me if below process flow will put the large object X >>> in to garbage collection, while waiting for the long running job to >>> continue? >>> >>> function1() -> >>> >>> X = ... fetch a large object.... >>> >>> ... some processing... >>> >>> x = ... extract a part of X... >>> ... long running job.... >>> >>> >>> If it's not putting X into garbage collection, does below change do that? >>> >>> function1() -> >>> >>> X = ... fetch a large object.... >>> >>> ... some processing... >>> >>> x = ... extract a part of X... >>> function2(x). >>> >>> function2(x) -> >>> >>> ... long running job... >>> >>> >>> Tks, >>> - Eranga >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> -- >> J. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Tue Sep 23 07:12:33 2014 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 22 Sep 2014 22:12:33 -0700 Subject: [erlang-questions] Erlang Memory Question In-Reply-To: References: Message-ID: <54210141.5050000@gmail.com> If garbage collection isn't happening quick enough, you can generate the problematic data within a temporary process to make the temporary process' death trigger garbage collection naturally. Doing that is a natural way to approach it. The tuning is something additional that is optional. I have a module I had used in the past to be as harsh as possible to the garbage collector (i.e., force it as much as possible) here https://gist.github.com/okeuday/dee991d580eeb00cd02c but I don't think it is necessary with a decent architecture in-place (being that harsh shouldn't be necessary). If you need to check the memory consumption of your processes http://www.erlang.org/doc/man/instrument.html is very helpful. On 09/22/2014 07:24 PM, Eranga Udesh wrote: > Thanks for the information received so far. > > Wouldn't it be good for Erlang to have a single object garbage collection function/bif? For example, when I no longer require a large object, I force to garbage collect it, without making a full sweep? > > As mentioned in the document, a full sweep may degrade the performance. > > - Eranga > > On Tue, Sep 23, 2014 at 12:10 AM, Bj?rn-Egil Dahlberg > wrote: > > It should suffice by running an erlang:garbage_collect/0 directly after extracting some parts of the structure: > > function1() -> > > X1 = ... fetch a large object .... > > ... some processing... > > X2 = ... extract a part of X1 ... > erlang:garbage_collect(), > ... long running job.... > > > As long as the object of X1 is not referenced by the process after the explicit call to the gc, you're fine. > Note also, an explicit call to garbage_collect/0 will always to a 'fullsweep'. > > Ofc, as Jesper mentioned, the it's probably preferable not to fetch the whole object if possible. > > // Bj?rn-Egil > > > 2014-09-22 19:56 GMT+02:00 Jesper Louis Andersen >: > > In general: No it won't. GC will trigger once the process has allocated enough data. If your processing is allocating enough data, this will quickly happen and you don't have to worry. If not, you may have to gently persuade the process to do so. There are several ways: > > * Set fullsweep_after on the process with a low value (0) and run erlang:garbage_collect() > * Go through a hibernation with an immediate wakeup (feels ugly to me) > * Handle fetching and extraction in a separate process and send the extracted part 'x' back as a message. This will free up memory almost immediately for other processes to use (simple and elegant) > * Don't fetch the large object in the first place, but make it possible to ask for an extraction only. Or stream data a bit at a time and handle the stream in chunks rather than everything at once. > > > On Mon, Sep 22, 2014 at 4:11 PM, Eranga Udesh > wrote: > > Hi, > > I'm trying to optimize my memory consumption in Erlang VM and to garbage collect as soon as possible. > > Let's say I have a large object, "X", After some processing, I only need to work on small part of X, called "x". > > Can someone advice me if below process flow will put the large object X in to garbage collection, while waiting for the long running job to continue? > > function1() -> > > X = ... fetch a large object.... > > ... some processing... > > x = ... extract a part of X... > ... long running job.... > > > If it's not putting X into garbage collection, does below change do that? > > function1() -> > > X = ... fetch a large object.... > > ... some processing... > > x = ... extract a part of X... > function2(x). > > function2(x) -> > > ... long running job... > > > Tks, > - Eranga > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > J. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Sep 23 09:12:12 2014 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 23 Sep 2014 19:12:12 +1200 Subject: [erlang-questions] Erlang Memory Question In-Reply-To: References: Message-ID: <9A5764B7-9252-4560-9AD6-BEC0DAE0305E@cs.otago.ac.nz> On 23/09/2014, at 2:24 PM, Eranga Udesh wrote: > Thanks for the information received so far. > > Wouldn't it be good for Erlang to have a single object garbage collection function/bif? NO. In C it's called free() and it's a major cause of errors. > For example, when I no longer require a large object, I force to garbage collect it, without making a full sweep? Why should the garbage collector *believe* you that the "object" is free? You could be deliberately lying. You could (and probably are) be mistaken. How is it to know which bits you want to keep without doing its usual thing? In a shared heap implementation (which Erlang has had and may have again) the fact that *you've* finished with the object doesn't mean everyone *else* has. A meaningful operation should not depend on implementation details like that. > > As mentioned in the document, a full sweep may degrade the performance. Not half as much as freeing too much would! This is micro-optimisation. Avoid passing around large objects in the first place. From ingela.andin@REDACTED Tue Sep 23 14:29:01 2014 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 23 Sep 2014 14:29:01 +0200 Subject: [erlang-questions] SSL handshake fails In-Reply-To: References: Message-ID: Hi! After some investigation I have concluded that the server may send an SNI-extension, and that if it does so, it shall be empty. "In this event, the server SHALL include an extension of type "server_name" in the (extended) server hello. The "extension_data" field of this extension SHALL be empty." I do not really see the point in include an empty SNI-extension on the server side, but as the RFC says so here comes a patch to handle it. diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index 22673e4..eee33ef 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -1732,6 +1732,9 @@ dec_hello_extensions(<>, Acc) when Len == 0 -> + dec_hello_extensions(Rest, Acc#hello_extensions{sni = ""}); %% Server may send an empy SNI + dec_hello_extensions(<>, Acc) -> <> = ExtData, Regards Ingela Erlang/OTP Team - Ericsson AB 2014-09-19 11:00 GMT+02:00 Iv?n Mart?nez : > Hello all, > I just hired a CentOS 7 server that came with very little software > installed. I installed Erlang 17.3 from sources, attached is output of the > configure step. Now I'm trying to install zotonic but it fails when trying > to do a SSL handshake with github, see below: > > [ivan@REDACTED zotonic]$ make > erl -noshell -s inets -s ssl \ > -eval '{ok, saved_to_file} = httpc:request(get, {" > https://github.com/rebar/rebar/wiki/rebar", []}, [], [{stream, > "./rebar"}])' \ > -s init stop > {"init terminating in > do_boot",{{badmatch,{error,{failed_connect,[{to_address,{"github.com",443}},{inet,[inet],{eoptions,{{{badmatch,<<0 > bytes>>},[{ssl_handshake,dec_hello_extensions,2,[{file,"ssl_handshake.erl"},{line,1737}]},{ssl_handshake,decode_handshake,3,[{file,"ssl_handshake.erl"},{line,926}]},{tls_handshake,get_tls_handshake_aux,3,[{file,"tls_handshake.erl"},{line,155}]},{tls_connection,next_state,4,[{file,"tls_connection.erl"},{line,433}]},{gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,503}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]},{gen_fsm,sync_send_all_state_event,[<0.54.0>,{start,infinity},infinity]}}}}]}}},[{erl_eval,expr,3,[]}]}} > > Crash dump was written to: erl_crash.dump > init terminating in do_boot () > make: *** [rebar] Error 1 > > I tried to do the handshake with openssl and apparently it works: > > [ivan@REDACTED zotonic]$ openssl s_client -host github.com -port 443 > CONNECTED(00000003) > depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert > High Assurance EV Root CA > verify return:1 > depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert > SHA2 Extended Validation Server CA > verify return:1 > depth=0 businessCategory = Private Organization, 1.3.6.1.4.1.311.60.2.1.3 > = US, 1.3.6.1.4.1.311.60.2.1.2 = Delaware, serialNumber = 5157550, street = > 548 4th Street, postalCode = 94107, C = US, ST = California, L = San > Francisco, O = "GitHub, Inc.", CN = github.com > verify return:1 > --- > Certificate chain > 0 s:/businessCategory=Private > Organization/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/serialNumber=5157550/street=548 > 4th Street/postalCode=94107/C=US/ST=California/L=San Francisco/O=GitHub, > Inc./CN=github.com > i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended > Validation Server CA > 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended > Validation Server CA > i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance > EV Root CA > --- > Server certificate > -----BEGIN CERTIFICATE----- > MIIF4DCCBMigAwIBAgIQDACTENIG2+M3VTWAEY3chzANBgkqhkiG9w0BAQsFADB1 > ... > XX4C2NesiZcLYbc2n7B9O+63M2k= > -----END CERTIFICATE----- > subject=/businessCategory=Private > Organization/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/serialNumber=5157550/street=548 > 4th Street/postalCode=94107/C=US/ST=California/L=San Francisco/O=GitHub, > Inc./CN=github.com > issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended > Validation Server CA > --- > No client certificate CA names sent > Server Temp Key: ECDH, prime256v1, 256 bits > --- > SSL handshake has read 3233 bytes and written 375 bytes > --- > New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256 > Server public key is 2048 bit > Secure Renegotiation IS supported > Compression: NONE > Expansion: NONE > SSL-Session: > Protocol : TLSv1.2 > Cipher : ECDHE-RSA-AES128-GCM-SHA256 > Session-ID: > DDEF6E78852287351EC5B20FFDD2578F8996E7226CB883A5F1A94325048B79C6 > Session-ID-ctx: > Master-Key: > D6C6283F463BFCD5A160E0CCE0CC8962CF944E5C98153040E4BC20466981B1622A5327C1E6BBED5F1751A049782908E5 > Key-Arg : None > Krb5 Principal: None > PSK identity: None > PSK identity hint: None > Start Time: 1411113552 > Timeout : 300 (sec) > Verify return code: 0 (ok) > --- > closed > > What can be wrong?. Thank you. > Ivan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Tue Sep 23 22:59:04 2014 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 23 Sep 2014 22:59:04 +0200 Subject: [erlang-questions] Help find bugs before OTP-18.0 is released Message-ID: Now that the --enable-sanitizers patch has been merged, it would be great if more users could test their code with an OTP install built with one or more sanitizers enabled in a dev/test environment. To be clear, it's _not_ a good idea to use in a production node. So far using only UBSan (undefined behavior sanitizer), we already found a couple bugs in the VM, and there's still time to find and fix more issues before OTP-18.0. The way sanitizers work is that the generated code is instrumented to find bugs at runtime. Basically, such an Erlang/OTP install will print what condition it encountered and possibly terminate. If you're interested and are using gcc >= 4.8 or clang >= 3.3, just make sure to enable it during ./configure as follows: # enable only undefined behavior sanitizer $ ./configure --enable-sanitizers=undefined # enable both address sanitizer and undefined behavior sanitizer $ ./configure --enable-sanitizers # enable just address sanitizer $ ./configure --enable-sanitizers=address There are heavier and more specific sanitizers you can enable, and if you'd like to learn more, see the following links: http://clang.llvm.org/docs/UsersManual.html#controlling-code-generation https://code.google.com/p/address-sanitizer/ http://clang.llvm.org/docs/AddressSanitizer.html https://code.google.com/p/thread-sanitizer/ http://clang.llvm.org/docs/ThreadSanitizer.html From shayan@REDACTED Wed Sep 24 00:00:47 2014 From: shayan@REDACTED (Shayan Pooya) Date: Tue, 23 Sep 2014 18:00:47 -0400 Subject: [erlang-questions] Help find bugs before OTP-18.0 is released In-Reply-To: References: Message-ID: I tried building the source code with gcc 4.8.3 and it complains: > gcc: error: unrecognized command line option ?-fsanitize=address,undefined? > It seems like this option is added to gcc 4.9: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60275 On Tue, Sep 23, 2014 at 4:59 PM, Tuncer Ayaz wrote: > Now that the --enable-sanitizers patch has been merged, it would be > great if more users could test their code with an OTP install built > with one or more sanitizers enabled in a dev/test environment. To be > clear, it's _not_ a good idea to use in a production node. > > So far using only UBSan (undefined behavior sanitizer), we already > found a couple bugs in the VM, and there's still time to find and fix > more issues before OTP-18.0. > > The way sanitizers work is that the generated code is instrumented to > find bugs at runtime. Basically, such an Erlang/OTP install will print > what condition it encountered and possibly terminate. > > If you're interested and are using gcc >= 4.8 or clang >= 3.3, just > make sure to enable it during ./configure as follows: > > # enable only undefined behavior sanitizer > $ ./configure --enable-sanitizers=undefined > > # enable both address sanitizer and undefined behavior sanitizer > $ ./configure --enable-sanitizers > > # enable just address sanitizer > $ ./configure --enable-sanitizers=address > > There are heavier and more specific sanitizers you can enable, and if > you'd like to learn more, see the following links: > http://clang.llvm.org/docs/UsersManual.html#controlling-code-generation > https://code.google.com/p/address-sanitizer/ > http://clang.llvm.org/docs/AddressSanitizer.html > https://code.google.com/p/thread-sanitizer/ > http://clang.llvm.org/docs/ThreadSanitizer.html > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Wed Sep 24 00:11:39 2014 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 24 Sep 2014 00:11:39 +0200 Subject: [erlang-questions] Help find bugs before OTP-18.0 is released In-Reply-To: References: Message-ID: On Wed, Sep 24, 2014 at 12:00 AM, Shayan Pooya wrote: > I tried building the source code with gcc 4.8.3 and it complains: > > > > gcc: error: unrecognized command line option > > ?-fsanitize=address,undefined? > > > It seems like this option is added to gcc 4.9: > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60275 Sorry about that, I've tested UBSan only with gcc 4.9, so that information was incorrect. But ASan is available in 4.8.3, right? Also, I forgot to say that issues found should be reported to erlang-bugs@REDACTED > On Tue, Sep 23, 2014 at 4:59 PM, Tuncer Ayaz wrote: > > > > Now that the --enable-sanitizers patch has been merged, it would be > > great if more users could test their code with an OTP install built > > with one or more sanitizers enabled in a dev/test environment. To be > > clear, it's _not_ a good idea to use in a production node. > > > > So far using only UBSan (undefined behavior sanitizer), we already > > found a couple bugs in the VM, and there's still time to find and fix > > more issues before OTP-18.0. > > > > The way sanitizers work is that the generated code is instrumented to > > find bugs at runtime. Basically, such an Erlang/OTP install will print > > what condition it encountered and possibly terminate. > > > > If you're interested and are using gcc >= 4.8 or clang >= 3.3, just > > make sure to enable it during ./configure as follows: > > > > # enable only undefined behavior sanitizer > > $ ./configure --enable-sanitizers=undefined > > > > # enable both address sanitizer and undefined behavior sanitizer > > $ ./configure --enable-sanitizers > > > > # enable just address sanitizer > > $ ./configure --enable-sanitizers=address > > > > There are heavier and more specific sanitizers you can enable, and if > > you'd like to learn more, see the following links: > > http://clang.llvm.org/docs/UsersManual.html#controlling-code-generation > > https://code.google.com/p/address-sanitizer/ > > http://clang.llvm.org/docs/AddressSanitizer.html > > https://code.google.com/p/thread-sanitizer/ > > http://clang.llvm.org/docs/ThreadSanitizer.html From eranga.erl@REDACTED Wed Sep 24 03:13:51 2014 From: eranga.erl@REDACTED (Eranga Udesh) Date: Wed, 24 Sep 2014 06:43:51 +0530 Subject: [erlang-questions] Erlang Memory Question In-Reply-To: <9A5764B7-9252-4560-9AD6-BEC0DAE0305E@cs.otago.ac.nz> References: <9A5764B7-9252-4560-9AD6-BEC0DAE0305E@cs.otago.ac.nz> Message-ID: Well, yes I may deliberately lie. However my suggestion is to, instead of doing a full sweep by the garbage collector (GC) to identify data going out of scope and reclaim, can the program (or rather I) deliberately say I (the calling process) is finished using the said data, so the GC may free that part. Then the GC may carry out it's own logic, which it currently does to verify if the same data is referenced by any other processes, etc., and decide if to GC or not. = Eranga On Tue, Sep 23, 2014 at 12:42 PM, Richard A. O'Keefe wrote: > > On 23/09/2014, at 2:24 PM, Eranga Udesh wrote: > > > Thanks for the information received so far. > > > > Wouldn't it be good for Erlang to have a single object garbage > collection function/bif? > > NO. In C it's called free() and it's a major cause of errors. > > > For example, when I no longer require a large object, I force to garbage > collect it, without making a full sweep? > > Why should the garbage collector *believe* you that the "object" > is free? You could be deliberately lying. You could (and > probably are) be mistaken. How is it to know which bits you > want to keep without doing its usual thing? In a shared heap > implementation (which Erlang has had and may have again) the > fact that *you've* finished with the object doesn't mean > everyone *else* has. A meaningful operation should not depend > on implementation details like that. > > > > As mentioned in the document, a full sweep may degrade the performance. > > Not half as much as freeing too much would! > > This is micro-optimisation. Avoid passing around large > objects in the first place. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Wed Sep 24 04:55:18 2014 From: jinni.park@REDACTED (Park, Sungjin) Date: Wed, 24 Sep 2014 11:55:18 +0900 Subject: [erlang-questions] zlib design flaw? Message-ID: Hi, I'm about to report a problem with erlang's zlib library interface which I think is a design flaw at this point of time. We recently had some malicious packets which were not very big in the first place but inflated to really big ones - hundreds of megabytes each. As a result, the server crashed with out-of-memory by the processes calling zlib:inflate/2. Urgency forced us to make a custom NIF library with inflation size limit. We also studied erlang reference manual but couldn't find anything useful. The zlib library source code shows even zlib:setBufSize/2 does not prevent producing very big binaries. Not being able to know how big the data would become after inflation, it should be a quite common problem. So I'm curious if I missed something very simple and nice. Is there anything like that? -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Wed Sep 24 05:15:52 2014 From: g@REDACTED (Guilherme Andrade) Date: Wed, 24 Sep 2014 04:15:52 +0100 Subject: [erlang-questions] zlib design flaw? In-Reply-To: References: Message-ID: <54223768.7010906@gandrade.net> Hi Sungjin, I've recently dealt this with very same issue, albeit only as a security hardening and a prevention measure; and, like you, I've looked into the zlib C code that's bundled with erts and arrived at that same conclusion. I was only able to limit it on a theoretical basis: the zlib guys themselves state[1] that a maximum compression ratio of 1032:1 is achievable (with big blobs of zeroes.) Therefore, if I want to limit the uncompressed content to less than, let's say, 5 MiB, I'll only accept compressed content of up to ~5 KiB. This thinking might be missing something, though. If there's a better/cleaner way to deal with this, I would love to know. Cheers, [1]: http://www.zlib.net/zlib_tech.html On 24-09-2014 03:55, Park, Sungjin wrote: > Hi, I'm about to report a problem with erlang's zlib library interface > which I think is a design flaw at this point of time. > > We recently had some malicious packets which were not very big in the > first place but inflated to really big ones - hundreds of megabytes > each. As a result, the server crashed with out-of-memory by the > processes calling zlib:inflate/2. Urgency forced us to make a custom > NIF library with inflation size limit. We also studied erlang > reference manual but couldn't find anything useful. The zlib library > source code shows even zlib:setBufSize/2 does not prevent producing > very big binaries. > > Not being able to know how big the data would become after inflation, > it should be a quite common problem. So I'm curious if I missed > something very simple and nice. Is there anything like that? > > -- > Park, Sungjin > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > ------------------------------------------------------------------------------------------------------------------- > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Guilherme https://www.gandrade.net/ PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From cjsvance@REDACTED Wed Sep 24 05:41:29 2014 From: cjsvance@REDACTED (Christopher Vance) Date: Wed, 24 Sep 2014 13:41:29 +1000 Subject: [erlang-questions] zlib design flaw? In-Reply-To: <54223768.7010906@gandrade.net> References: <54223768.7010906@gandrade.net> Message-ID: The zlib protocol is fully specified, and it appears the code is working correctly, so the only issue you might have is deciding what to do when you see input which could be seen as malicious (but which is still properly defined). Do you want to abort handling the input stream, or do you have an alternate suggestion? If you don't like the way zlib does it, or prefer a compression scheme which is more predictable, why not try a different compression algorithm, and see if it does any better in these pathological cases. I can specify you several 1:1 "compression" schemes where you only ever get out the same number of bytes you put in, but you'd probably think these safe method are too boring. On Wed, Sep 24, 2014 at 1:15 PM, Guilherme Andrade wrote: > Hi Sungjin, > > I've recently dealt this with very same issue, albeit only as a security > hardening and a prevention measure; and, like you, I've looked into the > zlib C code that's bundled with erts and arrived at that same conclusion. > > I was only able to limit it on a theoretical basis: the zlib guys > themselves state[1] that a maximum compression ratio of 1032:1 is > achievable (with big blobs of zeroes.) Therefore, if I want to limit the > uncompressed content to less than, let's say, 5 MiB, I'll only accept > compressed content of up to ~5 KiB. This thinking might be missing > something, though. > > If there's a better/cleaner way to deal with this, I would love to know. > > Cheers, > > > [1]: http://www.zlib.net/zlib_tech.html > > > > On 24-09-2014 03:55, Park, Sungjin wrote: > > Hi, I'm about to report a problem with erlang's zlib library interface > which I think is a design flaw at this point of time. > > We recently had some malicious packets which were not very big in the > first place but inflated to really big ones - hundreds of megabytes each. > As a result, the server crashed with out-of-memory by the processes calling > zlib:inflate/2. Urgency forced us to make a custom NIF library with > inflation size limit. We also studied erlang reference manual but couldn't > find anything useful. The zlib library source code shows even > zlib:setBufSize/2 does not prevent producing very big binaries. > > Not being able to know how big the data would become after inflation, it > should be a quite common problem. So I'm curious if I missed something > very simple and nice. Is there anything like that? > > -- > Park, Sungjin > > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > > ------------------------------------------------------------------------------------------------------------------- > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > -- > Guilherme > https://www.gandrade.net/ > PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Christopher Vance -------------- next part -------------- An HTML attachment was scrubbed... URL: From samjbarney@REDACTED Wed Sep 24 04:40:42 2014 From: samjbarney@REDACTED (Samuel Barney) Date: Tue, 23 Sep 2014 20:40:42 -0600 Subject: [erlang-questions] Question about the VM Message-ID: I'm developing a language on top of the Erlang VM, and there are a couple of flow-control constructs that I can't implement without skipping the Erlang AST and using the erlang assembly code. Is there a specific amount of X and Y registers? Or is there a way that I can find that out via Erlang? Also, what is the purpose of the Y registers? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Wed Sep 24 07:03:18 2014 From: jinni.park@REDACTED (Park, Sungjin) Date: Wed, 24 Sep 2014 14:03:18 +0900 Subject: [erlang-questions] zlib design flaw? In-Reply-To: References: <54223768.7010906@gandrade.net> Message-ID: Christopher, I don't have protocol level knowledge about zlib. But in my opinion - in programmer's perspective, one of the following should be possible. 1) an API to get inflated data size before actually inflating the data 2) an API to get part of the inflated data by given size and let the aggregation be done by the user 3) an API option to set limit in the size of the inflated data Currently, we are doing 3) with a custom NIF. Better of course if I can do any of the above with standard library. I don't understand about your 1:1 compression scheme suggestion. AFAIK, compression is used to reduce number of bytes on flight after all, isn't it? On Wed, Sep 24, 2014 at 12:41 PM, Christopher Vance wrote: > The zlib protocol is fully specified, and it appears the code is working > correctly, so the only issue you might have is deciding what to do when you > see input which could be seen as malicious (but which is still properly > defined). Do you want to abort handling the input stream, or do you have an > alternate suggestion? > > If you don't like the way zlib does it, or prefer a compression scheme > which is more predictable, why not try a different compression algorithm, > and see if it does any better in these pathological cases. I can specify > you several 1:1 "compression" schemes where you only ever get out the same > number of bytes you put in, but you'd probably think these safe method are > too boring. > > On Wed, Sep 24, 2014 at 1:15 PM, Guilherme Andrade wrote: > >> Hi Sungjin, >> >> I've recently dealt this with very same issue, albeit only as a security >> hardening and a prevention measure; and, like you, I've looked into the >> zlib C code that's bundled with erts and arrived at that same conclusion. >> >> I was only able to limit it on a theoretical basis: the zlib guys >> themselves state[1] that a maximum compression ratio of 1032:1 is >> achievable (with big blobs of zeroes.) Therefore, if I want to limit the >> uncompressed content to less than, let's say, 5 MiB, I'll only accept >> compressed content of up to ~5 KiB. This thinking might be missing >> something, though. >> >> If there's a better/cleaner way to deal with this, I would love to know. >> >> Cheers, >> >> >> [1]: http://www.zlib.net/zlib_tech.html >> >> >> >> On 24-09-2014 03:55, Park, Sungjin wrote: >> >> Hi, I'm about to report a problem with erlang's zlib library interface >> which I think is a design flaw at this point of time. >> >> We recently had some malicious packets which were not very big in the >> first place but inflated to really big ones - hundreds of megabytes each. >> As a result, the server crashed with out-of-memory by the processes calling >> zlib:inflate/2. Urgency forced us to make a custom NIF library with >> inflation size limit. We also studied erlang reference manual but couldn't >> find anything useful. The zlib library source code shows even >> zlib:setBufSize/2 does not prevent producing very big binaries. >> >> Not being able to know how big the data would become after inflation, >> it should be a quite common problem. So I'm curious if I missed something >> very simple and nice. Is there anything like that? >> >> -- >> Park, Sungjin >> >> ------------------------------------------------------------------------------------------------------------------- >> Peculiar travel suggestions are dancing lessons from god. >> -- The Books of Bokonon >> >> ------------------------------------------------------------------------------------------------------------------- >> >> >> _______________________________________________ >> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >> >> >> -- >> Guilherme >> https://www.gandrade.net/ >> PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Christopher Vance > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From cjsvance@REDACTED Wed Sep 24 09:22:47 2014 From: cjsvance@REDACTED (Christopher Vance) Date: Wed, 24 Sep 2014 17:22:47 +1000 Subject: [erlang-questions] zlib design flaw? In-Reply-To: References: <54223768.7010906@gandrade.net> Message-ID: I'm not sure whether to call you Park, or Sungjin, or something else, but Hello in any case. The simplest way I can think of to do (1) is to read all the data, doing the decompression as you go, but throwing away the actual decompressed data and everything else not needed in calculating the decompressed size. This is only useful if you can then decide to go back to the beginning of the data and read it all over again if you decide to keep the result of the decompression after all. (This is fine for reading a file which will still be there after you work out the decompressed size, but may require writing all data to a temporary file if you're reading data from a network stream.) The simplest way I can think of to do (2) is to read all the data, keeping what you decompress until you discover the next part won't fit. You then have to stop, return the data you have so far (probably less than the full size permitted) and you have to be able to resume at the beginning of the part that didn't fit, keeping any compression/decompression state necessary to continue. The simplest way I can think of to do (3) is to start doing (2) and abort the procedure altogether if the next part won't fit. My mention of 1:1 compression was intended to be facetious - a joke. My apologies if this wasn't clear. The easiest method I can think of is to return the input stream unaltered, so the "compressed" data is the same size as the original. There are other methods which are also trivially reversed with no risk of buffer overflow. If you haven't saved any space it's not actually useful, and it's not really compression, but you may want to be able to specify such an algorithm when chaining functions together. The proportion of compression you'll get from an algorithm will depend on the nature and variability of the content, and on the way the algorithm adapts to that variability. If you know the data is all zero values, a good compression algorithm is merely to report the length of the data; on the other hand, a stream of random (or pseudo-random) bits is theoretically uncompressable. So if mixing compression and encryption, you can usually save space by compressing before encrypting, and decrypting before decompressing, but the other way around will almost certainly increase your size because the compression algorithm will give a larger output than the input. If you know the data is sufficiently random, you may choose not to attempt compression at all. On Wed, Sep 24, 2014 at 3:03 PM, Park, Sungjin wrote: > Christopher, > > I don't have protocol level knowledge about zlib. But in my opinion - in > programmer's perspective, one of the following should be possible. > > 1) an API to get inflated data size before actually inflating the data > 2) an API to get part of the inflated data by given size and let the > aggregation be done by the user > 3) an API option to set limit in the size of the inflated data > > Currently, we are doing 3) with a custom NIF. Better of course if I can > do any of the above with standard library. > > I don't understand about your 1:1 compression scheme suggestion. AFAIK, > compression is used to reduce number of bytes on flight after all, isn't it? > > > On Wed, Sep 24, 2014 at 12:41 PM, Christopher Vance > wrote: > >> The zlib protocol is fully specified, and it appears the code is working >> correctly, so the only issue you might have is deciding what to do when you >> see input which could be seen as malicious (but which is still properly >> defined). Do you want to abort handling the input stream, or do you have an >> alternate suggestion? >> >> If you don't like the way zlib does it, or prefer a compression scheme >> which is more predictable, why not try a different compression algorithm, >> and see if it does any better in these pathological cases. I can specify >> you several 1:1 "compression" schemes where you only ever get out the same >> number of bytes you put in, but you'd probably think these safe method are >> too boring. >> >> On Wed, Sep 24, 2014 at 1:15 PM, Guilherme Andrade >> wrote: >> >>> Hi Sungjin, >>> >>> I've recently dealt this with very same issue, albeit only as a security >>> hardening and a prevention measure; and, like you, I've looked into the >>> zlib C code that's bundled with erts and arrived at that same conclusion. >>> >>> I was only able to limit it on a theoretical basis: the zlib guys >>> themselves state[1] that a maximum compression ratio of 1032:1 is >>> achievable (with big blobs of zeroes.) Therefore, if I want to limit the >>> uncompressed content to less than, let's say, 5 MiB, I'll only accept >>> compressed content of up to ~5 KiB. This thinking might be missing >>> something, though. >>> >>> If there's a better/cleaner way to deal with this, I would love to know. >>> >>> Cheers, >>> >>> >>> [1]: http://www.zlib.net/zlib_tech.html >>> >>> >>> >>> On 24-09-2014 03:55, Park, Sungjin wrote: >>> >>> Hi, I'm about to report a problem with erlang's zlib library interface >>> which I think is a design flaw at this point of time. >>> >>> We recently had some malicious packets which were not very big in the >>> first place but inflated to really big ones - hundreds of megabytes each. >>> As a result, the server crashed with out-of-memory by the processes calling >>> zlib:inflate/2. Urgency forced us to make a custom NIF library with >>> inflation size limit. We also studied erlang reference manual but couldn't >>> find anything useful. The zlib library source code shows even >>> zlib:setBufSize/2 does not prevent producing very big binaries. >>> >>> Not being able to know how big the data would become after inflation, >>> it should be a quite common problem. So I'm curious if I missed something >>> very simple and nice. Is there anything like that? >>> >>> -- >>> Park, Sungjin >>> >>> ------------------------------------------------------------------------------------------------------------------- >>> Peculiar travel suggestions are dancing lessons from god. >>> -- The Books of Bokonon >>> >>> ------------------------------------------------------------------------------------------------------------------- >>> >>> >>> _______________________________________________ >>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> -- >>> Guilherme >>> https://www.gandrade.net/ >>> PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> -- >> Christopher Vance >> > > > > -- > Park, Sungjin > > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > > ------------------------------------------------------------------------------------------------------------------- > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Christopher Vance -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Wed Sep 24 10:01:55 2014 From: jinni.park@REDACTED (Park, Sungjin) Date: Wed, 24 Sep 2014 17:01:55 +0900 Subject: [erlang-questions] zlib design flaw? In-Reply-To: References: <54223768.7010906@gandrade.net> Message-ID: Chris, you can call me Sungjin as it's my first name. The problem is that the server crashes with out-of-memory as soon as it reads all the data. I have no chance to decide whether to drop or not before the zlib library tosses those big binaries. So what I want is to let the zlib library toss moderate sized fragments of the inflated data so that I might stop processing further so as not to crash whole vm. And I also think this is kind of serious problem that I can kill any web server written in erlang in this simple way. Choosing appropriate compression scheme according to content type is, sorry, not the answer to the topic I originally issued. I was issuing a security problem against malicious hackers who send one megabytes of deflated zeros pretending to be images. Every those fake images will consume gigabytes of system memory with inflated zero binary as Guilherme noted. On Wed, Sep 24, 2014 at 4:22 PM, Christopher Vance wrote: > I'm not sure whether to call you Park, or Sungjin, or something else, but > Hello in any case. > > The simplest way I can think of to do (1) is to read all the data, doing > the decompression as you go, but throwing away the actual decompressed data > and everything else not needed in calculating the decompressed size. This > is only useful if you can then decide to go back to the beginning of the > data and read it all over again if you decide to keep the result of the > decompression after all. (This is fine for reading a file which will still > be there after you work out the decompressed size, but may require writing > all data to a temporary file if you're reading data from a network stream.) > > The simplest way I can think of to do (2) is to read all the data, keeping > what you decompress until you discover the next part won't fit. You then > have to stop, return the data you have so far (probably less than the full > size permitted) and you have to be able to resume at the beginning of the > part that didn't fit, keeping any compression/decompression state necessary > to continue. > > The simplest way I can think of to do (3) is to start doing (2) and abort > the procedure altogether if the next part won't fit. > > My mention of 1:1 compression was intended to be facetious - a joke. My > apologies if this wasn't clear. The easiest method I can think of is to > return the input stream unaltered, so the "compressed" data is the same > size as the original. There are other methods which are also trivially > reversed with no risk of buffer overflow. If you haven't saved any space > it's not actually useful, and it's not really compression, but you may want > to be able to specify such an algorithm when chaining functions together. > > The proportion of compression you'll get from an algorithm will depend on > the nature and variability of the content, and on the way the algorithm > adapts to that variability. If you know the data is all zero values, a good > compression algorithm is merely to report the length of the data; on the > other hand, a stream of random (or pseudo-random) bits is theoretically > uncompressable. > > So if mixing compression and encryption, you can usually save space by > compressing before encrypting, and decrypting before decompressing, but the > other way around will almost certainly increase your size because the > compression algorithm will give a larger output than the input. If you know > the data is sufficiently random, you may choose not to attempt compression > at all. > > On Wed, Sep 24, 2014 at 3:03 PM, Park, Sungjin > wrote: > >> Christopher, >> >> I don't have protocol level knowledge about zlib. But in my opinion - in >> programmer's perspective, one of the following should be possible. >> >> 1) an API to get inflated data size before actually inflating the data >> 2) an API to get part of the inflated data by given size and let the >> aggregation be done by the user >> 3) an API option to set limit in the size of the inflated data >> >> Currently, we are doing 3) with a custom NIF. Better of course if I can >> do any of the above with standard library. >> >> I don't understand about your 1:1 compression scheme suggestion. AFAIK, >> compression is used to reduce number of bytes on flight after all, isn't it? >> >> >> On Wed, Sep 24, 2014 at 12:41 PM, Christopher Vance >> wrote: >> >>> The zlib protocol is fully specified, and it appears the code is working >>> correctly, so the only issue you might have is deciding what to do when you >>> see input which could be seen as malicious (but which is still properly >>> defined). Do you want to abort handling the input stream, or do you have an >>> alternate suggestion? >>> >>> If you don't like the way zlib does it, or prefer a compression scheme >>> which is more predictable, why not try a different compression algorithm, >>> and see if it does any better in these pathological cases. I can specify >>> you several 1:1 "compression" schemes where you only ever get out the same >>> number of bytes you put in, but you'd probably think these safe method are >>> too boring. >>> >>> On Wed, Sep 24, 2014 at 1:15 PM, Guilherme Andrade >>> wrote: >>> >>>> Hi Sungjin, >>>> >>>> I've recently dealt this with very same issue, albeit only as a >>>> security hardening and a prevention measure; and, like you, I've looked >>>> into the zlib C code that's bundled with erts and arrived at that same >>>> conclusion. >>>> >>>> I was only able to limit it on a theoretical basis: the zlib guys >>>> themselves state[1] that a maximum compression ratio of 1032:1 is >>>> achievable (with big blobs of zeroes.) Therefore, if I want to limit the >>>> uncompressed content to less than, let's say, 5 MiB, I'll only accept >>>> compressed content of up to ~5 KiB. This thinking might be missing >>>> something, though. >>>> >>>> If there's a better/cleaner way to deal with this, I would love to know. >>>> >>>> Cheers, >>>> >>>> >>>> [1]: http://www.zlib.net/zlib_tech.html >>>> >>>> >>>> >>>> On 24-09-2014 03:55, Park, Sungjin wrote: >>>> >>>> Hi, I'm about to report a problem with erlang's zlib library interface >>>> which I think is a design flaw at this point of time. >>>> >>>> We recently had some malicious packets which were not very big in the >>>> first place but inflated to really big ones - hundreds of megabytes each. >>>> As a result, the server crashed with out-of-memory by the processes calling >>>> zlib:inflate/2. Urgency forced us to make a custom NIF library with >>>> inflation size limit. We also studied erlang reference manual but couldn't >>>> find anything useful. The zlib library source code shows even >>>> zlib:setBufSize/2 does not prevent producing very big binaries. >>>> >>>> Not being able to know how big the data would become after inflation, >>>> it should be a quite common problem. So I'm curious if I missed something >>>> very simple and nice. Is there anything like that? >>>> >>>> -- >>>> Park, Sungjin >>>> >>>> ------------------------------------------------------------------------------------------------------------------- >>>> Peculiar travel suggestions are dancing lessons from god. >>>> -- The Books of Bokonon >>>> >>>> ------------------------------------------------------------------------------------------------------------------- >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> -- >>>> Guilherme >>>> https://www.gandrade.net/ >>>> PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >>> >>> -- >>> Christopher Vance >>> >> >> >> >> -- >> Park, Sungjin >> >> ------------------------------------------------------------------------------------------------------------------- >> Peculiar travel suggestions are dancing lessons from god. >> -- The Books of Bokonon >> >> ------------------------------------------------------------------------------------------------------------------- >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Christopher Vance > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Sep 24 10:09:19 2014 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 24 Sep 2014 20:09:19 +1200 Subject: [erlang-questions] zlib design flaw? In-Reply-To: References: <54223768.7010906@gandrade.net> Message-ID: I'm a little bit puzzled here. According to zlib.h, the underlying C code works with input and output buffers. Decompression is done in chunks, and one reason for a chunk stopping is that the output buffer has filled up. I would expect a zlib interface to say "give me the decompressed information in chunks no bigger than this, as and when I ask for them". Looking at RFC 1950 and RFC 1951, since reflecting on the fact that you are supposed to be able to create zlib compressed stuff by streaming blocks through the library without announcing the total size in advance, it really doesn't look as though there can be a way to get the size without actually decompressing. Is there any possibility of adjusting the overall protocol to transmit the size in advance? From jinni.park@REDACTED Wed Sep 24 10:26:02 2014 From: jinni.park@REDACTED (Park, Sungjin) Date: Wed, 24 Sep 2014 17:26:02 +0900 Subject: [erlang-questions] zlib design flaw? In-Reply-To: References: <54223768.7010906@gandrade.net> Message-ID: Guess not. Personally, I the method 2 among my suggestion before is the most consistent and generic API design in this sense - streaming chunks in inflating too. Actually, zlib.erl is implemented in streaming chunks from a port driver to the calling process. The problem is that the calling process collects all the chunks indefinitely and there is no way to intervene. On Wed, Sep 24, 2014 at 5:09 PM, Richard A. O'Keefe wrote: > I'm a little bit puzzled here. > According to zlib.h, the underlying C code works with > input and output buffers. Decompression is done in > chunks, and one reason for a chunk stopping is that the > output buffer has filled up. I would expect a zlib > interface to say "give me the decompressed information > in chunks no bigger than this, as and when I ask for them". > > Looking at RFC 1950 and RFC 1951, since reflecting on the fact > that you are supposed to be able to create zlib compressed > stuff by streaming blocks through the library without > announcing the total size in advance, it really doesn't look > as though there can be a way to get the size without > actually decompressing. > > Is there any possibility of adjusting the overall protocol to > transmit the size in advance? > > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Wed Sep 24 10:33:55 2014 From: jinni.park@REDACTED (Park, Sungjin) Date: Wed, 24 Sep 2014 17:33:55 +0900 Subject: [erlang-questions] zlib design flaw? In-Reply-To: References: <54223768.7010906@gandrade.net> Message-ID: Thank you Robert. It's so common that has the cool name! On Wed, Sep 24, 2014 at 5:26 PM, Park, Sungjin wrote: > Guess not. > Personally, I the method 2 among my suggestion before is the most > consistent and generic API design in this sense - streaming chunks in > inflating too. > Actually, zlib.erl is implemented in streaming chunks from a port driver > to the calling process. The problem is that the calling process collects > all the chunks indefinitely and there is no way to intervene. > > > On Wed, Sep 24, 2014 at 5:09 PM, Richard A. O'Keefe > wrote: > >> I'm a little bit puzzled here. >> According to zlib.h, the underlying C code works with >> input and output buffers. Decompression is done in >> chunks, and one reason for a chunk stopping is that the >> output buffer has filled up. I would expect a zlib >> interface to say "give me the decompressed information >> in chunks no bigger than this, as and when I ask for them". >> >> Looking at RFC 1950 and RFC 1951, since reflecting on the fact >> that you are supposed to be able to create zlib compressed >> stuff by streaming blocks through the library without >> announcing the total size in advance, it really doesn't look >> as though there can be a way to get the size without >> actually decompressing. >> >> Is there any possibility of adjusting the overall protocol to >> transmit the size in advance? >> >> > > > -- > Park, Sungjin > > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > > ------------------------------------------------------------------------------------------------------------------- > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitryme@REDACTED Wed Sep 24 10:40:34 2014 From: dmitryme@REDACTED (Dmitry Melnikov) Date: Wed, 24 Sep 2014 12:40:34 +0400 Subject: [erlang-questions] C node and erl_send crash Message-ID: Hi, I am trying to use Erlang C node feature and use erl_interface (not ei). My C node sends periodically data to its native Erlang subscribers. And it works perfectly but, if I stop subscriber (e.g. just pressing Ctrl+C twice from console) erl_send makes my C node silently died w/o crash dump. Is it expected behaviour? BTW, I am using erl_interface library sockets in non-blocking mode with epoll for processing events from another sockets, descriptors etc. -------------- next part -------------- An HTML attachment was scrubbed... URL: From huss01@REDACTED Wed Sep 24 10:52:57 2014 From: huss01@REDACTED (=?ISO-8859-1?Q?H=E5kan_Huss?=) Date: Wed, 24 Sep 2014 10:52:57 +0200 Subject: [erlang-questions] zlib design flaw? In-Reply-To: References: <54223768.7010906@gandrade.net> Message-ID: 2014-09-24 10:26 GMT+02:00 Park, Sungjin : > Guess not. > Personally, I the method 2 among my suggestion before is the most > consistent and generic API design in this sense - streaming chunks in > inflating too. > Actually, zlib.erl is implemented in streaming chunks from a port driver > to the calling process. The problem is that the calling process collects > all the chunks indefinitely and there is no way to intervene. > > Of course, you could feed zlib:inflate one small chunk of the input at a time. Assuming the previously quoted bound of compression is correct, taking say 1kB chunks would limit the inflated chunk size to ~1MB. If you are really careful you can do one byte of input at a time. > On Wed, Sep 24, 2014 at 5:09 PM, Richard A. O'Keefe > wrote: > >> I'm a little bit puzzled here. >> According to zlib.h, the underlying C code works with >> input and output buffers. Decompression is done in >> chunks, and one reason for a chunk stopping is that the >> output buffer has filled up. I would expect a zlib >> interface to say "give me the decompressed information >> in chunks no bigger than this, as and when I ask for them". >> >> I suspect that the idea is to avoid getting "partially inflated" bytes. Limiting the inflated size would require a more elaborate internal state in zlib, and also more corner cases in the API. Looking at RFC 1950 and RFC 1951, since reflecting on the fact >> that you are supposed to be able to create zlib compressed >> stuff by streaming blocks through the library without >> announcing the total size in advance, it really doesn't look >> as though there can be a way to get the size without >> actually decompressing. >> >> Is there any possibility of adjusting the overall protocol to >> transmit the size in advance? >> >> If the compressed data is actually zip or gzip, size information is present in the metadata. However, a malicious user can fake this, as would probably be the case with any solution transmitting the size. I guess you could create a cryptographic protocol which guarantees that the transmitted size is correct, but that may be overkill... > > > > -- > Park, Sungjin > > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > > ------------------------------------------------------------------------------------------------------------------- > > /H?kan Huss -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Wed Sep 24 10:17:42 2014 From: bob@REDACTED (Robert Wilkinson) Date: Wed, 24 Sep 2014 10:17:42 +0200 Subject: [erlang-questions] zlib design flaw? In-Reply-To: References: Message-ID: <20140924081742.GA23308@fourtheye.org> On Wed, Sep 24, 2014 at 11:55:18AM +0900, Park, Sungjin wrote: > > We recently had some malicious packets which were not very big in the first > place but inflated to really big ones - hundreds of megabytes each. As a > result, the server crashed with out-of-memory by the processes calling > zlib:inflate/2. Urgency forced us to make a custom NIF library with > inflation size limit. We also studied erlang reference manual but couldn't > find anything useful. The zlib library source code shows even > zlib:setBufSize/2 does not prevent producing very big binaries. > > Not being able to know how big the data would become after inflation, it > should be a quite common problem. So I'm curious if I missed something > very simple and nice. Is there anything like that? Hi Sungjin The articles referenced at http://en.wikipedia.org/wiki/Zip_bomb should give you some insight into the problem, in general. Bob From daniel.abrahamsson@REDACTED Wed Sep 24 11:43:13 2014 From: daniel.abrahamsson@REDACTED (Daniel Abrahamsson) Date: Wed, 24 Sep 2014 11:43:13 +0200 Subject: [erlang-questions] SSL handshake fails In-Reply-To: References: Message-ID: Got exactly the same error for some of our connections after upgrading from R16B03 to 17.3. Will this patch be included with the next stable release on the 17.x branch? //Daniel On Tue, Sep 23, 2014 at 2:29 PM, Ingela Andin wrote: > Hi! > > After some investigation I have concluded that the server may send an > SNI-extension, and that if it does so, it shall be empty. > > "In this event, the > server SHALL include an extension of type "server_name" in the > (extended) server hello. The "extension_data" field of this > extension SHALL be empty." > > > I do not really see the point in include an empty SNI-extension on the > server side, but as the RFC says so here comes a patch to handle it. > > diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl > index 22673e4..eee33ef 100644 > --- a/lib/ssl/src/ssl_handshake.erl > +++ b/lib/ssl/src/ssl_handshake.erl > @@ -1732,6 +1732,9 @@ > dec_hello_extensions(< #ec_point_formats{ec_point_format_list = > ECPointFormats}}); > > +dec_hello_extensions(<>, > Acc) when Len == 0 -> > + dec_hello_extensions(Rest, Acc#hello_extensions{sni = ""}); %% Server > may send an empy SNI > + > dec_hello_extensions(< ExtData:Len/binary, Rest/binary>>, Acc) -> > <> = ExtData, > > > Regards Ingela Erlang/OTP Team - Ericsson AB > > > > 2014-09-19 11:00 GMT+02:00 Iv?n Mart?nez : > >> Hello all, >> I just hired a CentOS 7 server that came with very little software >> installed. I installed Erlang 17.3 from sources, attached is output of the >> configure step. Now I'm trying to install zotonic but it fails when trying >> to do a SSL handshake with github, see below: >> >> [ivan@REDACTED zotonic]$ make >> erl -noshell -s inets -s ssl \ >> -eval '{ok, saved_to_file} = httpc:request(get, {" >> https://github.com/rebar/rebar/wiki/rebar", []}, [], [{stream, >> "./rebar"}])' \ >> -s init stop >> {"init terminating in >> do_boot",{{badmatch,{error,{failed_connect,[{to_address,{"github.com",443}},{inet,[inet],{eoptions,{{{badmatch,<<0 >> bytes>>},[{ssl_handshake,dec_hello_extensions,2,[{file,"ssl_handshake.erl"},{line,1737}]},{ssl_handshake,decode_handshake,3,[{file,"ssl_handshake.erl"},{line,926}]},{tls_handshake,get_tls_handshake_aux,3,[{file,"tls_handshake.erl"},{line,155}]},{tls_connection,next_state,4,[{file,"tls_connection.erl"},{line,433}]},{gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,503}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]},{gen_fsm,sync_send_all_state_event,[<0.54.0>,{start,infinity},infinity]}}}}]}}},[{erl_eval,expr,3,[]}]}} >> >> Crash dump was written to: erl_crash.dump >> init terminating in do_boot () >> make: *** [rebar] Error 1 >> >> I tried to do the handshake with openssl and apparently it works: >> >> [ivan@REDACTED zotonic]$ openssl s_client -host github.com -port 443 >> CONNECTED(00000003) >> depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert >> High Assurance EV Root CA >> verify return:1 >> depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert >> SHA2 Extended Validation Server CA >> verify return:1 >> depth=0 businessCategory = Private Organization, 1.3.6.1.4.1.311.60.2.1.3 >> = US, 1.3.6.1.4.1.311.60.2.1.2 = Delaware, serialNumber = 5157550, street = >> 548 4th Street, postalCode = 94107, C = US, ST = California, L = San >> Francisco, O = "GitHub, Inc.", CN = github.com >> verify return:1 >> --- >> Certificate chain >> 0 s:/businessCategory=Private >> Organization/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/serialNumber=5157550/street=548 >> 4th Street/postalCode=94107/C=US/ST=California/L=San Francisco/O=GitHub, >> Inc./CN=github.com >> i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended >> Validation Server CA >> 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended >> Validation Server CA >> i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance >> EV Root CA >> --- >> Server certificate >> -----BEGIN CERTIFICATE----- >> MIIF4DCCBMigAwIBAgIQDACTENIG2+M3VTWAEY3chzANBgkqhkiG9w0BAQsFADB1 >> ... >> XX4C2NesiZcLYbc2n7B9O+63M2k= >> -----END CERTIFICATE----- >> subject=/businessCategory=Private >> Organization/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/serialNumber=5157550/street=548 >> 4th Street/postalCode=94107/C=US/ST=California/L=San Francisco/O=GitHub, >> Inc./CN=github.com >> issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 >> Extended Validation Server CA >> --- >> No client certificate CA names sent >> Server Temp Key: ECDH, prime256v1, 256 bits >> --- >> SSL handshake has read 3233 bytes and written 375 bytes >> --- >> New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256 >> Server public key is 2048 bit >> Secure Renegotiation IS supported >> Compression: NONE >> Expansion: NONE >> SSL-Session: >> Protocol : TLSv1.2 >> Cipher : ECDHE-RSA-AES128-GCM-SHA256 >> Session-ID: >> DDEF6E78852287351EC5B20FFDD2578F8996E7226CB883A5F1A94325048B79C6 >> Session-ID-ctx: >> Master-Key: >> D6C6283F463BFCD5A160E0CCE0CC8962CF944E5C98153040E4BC20466981B1622A5327C1E6BBED5F1751A049782908E5 >> Key-Arg : None >> Krb5 Principal: None >> PSK identity: None >> PSK identity hint: None >> Start Time: 1411113552 >> Timeout : 300 (sec) >> Verify return code: 0 (ok) >> --- >> closed >> >> What can be wrong?. Thank you. >> Ivan >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Sep 24 12:07:28 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 24 Sep 2014 13:07:28 +0300 Subject: [erlang-questions] zlib design flaw? In-Reply-To: References: <54223768.7010906@gandrade.net> Message-ID: <542297E0.4030107@ninenines.eu> On 09/24/2014 11:09 AM, Richard A. O'Keefe wrote: > I'm a little bit puzzled here. > According to zlib.h, the underlying C code works with > input and output buffers. Decompression is done in > chunks, and one reason for a chunk stopping is that the > output buffer has filled up. I would expect a zlib > interface to say "give me the decompressed information > in chunks no bigger than this, as and when I ask for them". I believe it does. From http://www.zlib.net/manual.html#Usage > The application must update next_in and avail_in when avail_in has dropped to zero. It must update next_out and avail_out when avail_out has dropped to zero. So it should be possible to check if we are above a certain size when avail_out drops to 0, and return the data decompressed so far. Looking at implementations I can see that PHP is doing just that. On the other hand Python has no max length parameter and instead relies on process-wide limits. -- Lo?c Hoguin http://ninenines.eu From n.oxyde@REDACTED Wed Sep 24 12:40:12 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 24 Sep 2014 12:40:12 +0200 Subject: [erlang-questions] Question about the VM In-Reply-To: References: Message-ID: Did you try targeting Core Erlang first? Which primitives do you want to access in BEAM? There are 1024 X registers, and Y registers are used for locals in functions, to save them when calling other functions. You can find such information by reading the source code or various talk slides made by the OTP team these last years, but I can't help you find the links right now. Regards, Le 24 sept. 2014 ? 04:40, Samuel Barney a ?crit : > I'm developing a language on top of the Erlang VM, and there are a couple of flow-control constructs that I can't implement without skipping the Erlang AST and using the erlang assembly code. > > Is there a specific amount of X and Y registers? Or is there a way that I can find that out via Erlang? > > Also, what is the purpose of the Y registers? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From seriy.pr@REDACTED Wed Sep 24 15:17:12 2014 From: seriy.pr@REDACTED (=?UTF-8?B?0KHQtdGA0LPQtdC5INCf0YDQvtGF0L7RgNC+0LI=?=) Date: Wed, 24 Sep 2014 17:17:12 +0400 Subject: [erlang-questions] zlib design flaw? Message-ID: I've discussed exactly the same issue in #Erlang IRC ~year ago. So, we discovered, that zlib C API allows to limit decompressed data size (by storing some internal state and data in zlib internals), but this option isn't exported in Erlang API. The ad-hoc solution suggested was "decompress your data by smaller chunks". And, yes, I think that this option should be exported to Erlang's zlib. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Sep 24 15:53:42 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 24 Sep 2014 15:53:42 +0200 Subject: [erlang-questions] Erlang Memory Question In-Reply-To: References: <9A5764B7-9252-4560-9AD6-BEC0DAE0305E@cs.otago.ac.nz> Message-ID: On Wed, Sep 24, 2014 at 3:13 AM, Eranga Udesh wrote: > However my suggestion is to, instead of doing a full sweep by the garbage > collector (GC) to identify data going out of scope and reclaim, can the > program (or rather I) deliberately say I (the calling process) is finished > using the said data, so the GC may free that part. You probably want to read the paper "A unified theory of Garbage collection", Baker et.al[0]. What you are proposing is certainly possible. You can identify data which is dead after you leave a scope and quickly reclaim such data. Very simple data already has this property and goes onto the Y stack in he VM. It has to be weighed against a number of factors though. It is not a priori clear it would provide a much better space profile of the program. The Erlang GC is generational to avoid storing data which is short-lived in size, so usually it is able to take care of dead data quite efficiently. The paper describes the duality between quick release of data weighed against the advantage of batching up garbage collection in larger chunks. Good GCs tend to gravitate toward the dual point. So a collector relying on reference counting has to adapt itself as to not pause the system when freeing up a large structure. And systems relying on batch collection gains what you propose to free memory faster for certain known corner cases. That said, I am interested in your use case, because I have never really hit Erlangs GC as a limiting factor. Maybe because I'm concious about how a GC works and tries to pet it nicely :) [0] http://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Wed Sep 24 16:28:22 2014 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 24 Sep 2014 15:28:22 +0100 Subject: [erlang-questions] Erlang Memory Question In-Reply-To: References: <9A5764B7-9252-4560-9AD6-BEC0DAE0305E@cs.otago.ac.nz> Message-ID: I always thought that is one of the reasons to have processes. If you've got something big you want to throw away quickly, make a process for it. $ erl Erlang R16B03-1 (erts-5.10.4) [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V5.10.4 (abort with ^G) 1> erlang:memory(). [{total,14148416}, {processes,4091608}, {processes_used,4091488}, {system,10056808}, {atom,194289}, {atom_used,172614}, {binary,1514552}, {code,4026600}, {ets,262688}] 2> Pid = spawn( 2> fun () -> 2> X = binary:copy(<<1,2,3,4,5,6,7,8>>, 1024), 2> Y = binary:copy(X, 1024), 2> receive stop -> ok end 2> end 2> ). <0.35.0> 3> erlang:memory(). [{total,22643832}, {processes,4203448}, {processes_used,4203448}, {system,18440384}, {atom,194289}, {atom_used,175110}, {binary,9685320}, {code,4221791}, {ets,267056}] 4> Pid ! stop. stop 5> erlang:memory(). [{total,13587776}, {processes,4084496}, {processes_used,4084384}, {system,9503280}, {atom,194289}, {atom_used,175110}, {binary,748144}, {code,4221791}, {ets,267056}] 6> Regards, Robby On 24 September 2014 02:13, Eranga Udesh wrote: > Well, yes I may deliberately lie. > > However my suggestion is to, instead of doing a full sweep by the garbage > collector (GC) to identify data going out of scope and reclaim, can the > program (or rather I) deliberately say I (the calling process) is finished > using the said data, so the GC may free that part. > > Then the GC may carry out it's own logic, which it currently does to > verify if the same data is referenced by any other processes, etc., and > decide if to GC or not. > > = Eranga > > On Tue, Sep 23, 2014 at 12:42 PM, Richard A. O'Keefe > wrote: > >> >> On 23/09/2014, at 2:24 PM, Eranga Udesh wrote: >> >> > Thanks for the information received so far. >> > >> > Wouldn't it be good for Erlang to have a single object garbage >> collection function/bif? >> >> NO. In C it's called free() and it's a major cause of errors. >> >> > For example, when I no longer require a large object, I force to >> garbage collect it, without making a full sweep? >> >> Why should the garbage collector *believe* you that the "object" >> is free? You could be deliberately lying. You could (and >> probably are) be mistaken. How is it to know which bits you >> want to keep without doing its usual thing? In a shared heap >> implementation (which Erlang has had and may have again) the >> fact that *you've* finished with the object doesn't mean >> everyone *else* has. A meaningful operation should not depend >> on implementation details like that. >> > >> > As mentioned in the document, a full sweep may degrade the performance. >> >> Not half as much as freeing too much would! >> >> This is micro-optimisation. Avoid passing around large >> objects in the first place. >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Sep 24 18:52:27 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 24 Sep 2014 19:52:27 +0300 Subject: [erlang-questions] Specs for optional callbacks (try 2) Message-ID: <5422F6CB.6080403@ninenines.eu> Hi, I didn't get any reply on my previous question almost 2 years ago here: http://erlang.org/pipermail/erlang-questions/2013-January/071763.html So let me ask again, is there any plans for adding something like -optional_callback that would do the same as -callback except it would not require the callback to be defined? Basically Dialyzer would use it, and the compiler wouldn't fail if the callback is missing. As it stands now I am about to remove most -behavior in Cowboy because they're starting to make little sense because most callbacks become optional. The simplest handler type will have 3 possible callbacks, 2 being optional. The most complex handler type already has 20+ callbacks with only 1 required. Thanks for any reply. -- Lo?c Hoguin http://ninenines.eu From gumm@REDACTED Wed Sep 24 19:17:37 2014 From: gumm@REDACTED (Jesse Gumm) Date: Wed, 24 Sep 2014 12:17:37 -0500 Subject: [erlang-questions] Specs for optional callbacks (try 2) In-Reply-To: <5422F6CB.6080403@ninenines.eu> References: <5422F6CB.6080403@ninenines.eu> Message-ID: I would really like to see this as well. I think it's a solid idea. +1 from me. -Jesse -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm On Sep 24, 2014 11:52 AM, "Lo?c Hoguin" wrote: > Hi, > > I didn't get any reply on my previous question almost 2 years ago here: > http://erlang.org/pipermail/erlang-questions/2013-January/071763.html > > So let me ask again, is there any plans for adding something like > -optional_callback that would do the same as -callback except it would not > require the callback to be defined? Basically Dialyzer would use it, and > the compiler wouldn't fail if the callback is missing. > > As it stands now I am about to remove most -behavior in Cowboy because > they're starting to make little sense because most callbacks become > optional. The simplest handler type will have 3 possible callbacks, 2 being > optional. The most complex handler type already has 20+ callbacks with only > 1 required. > > Thanks for any reply. > > -- > Lo?c Hoguin > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Wed Sep 24 19:46:18 2014 From: tony@REDACTED (Tony Rogvall) Date: Wed, 24 Sep 2014 19:46:18 +0200 Subject: [erlang-questions] zlib design flaw? In-Reply-To: <20140924081742.GA23308@fourtheye.org> References: <20140924081742.GA23308@fourtheye.org> Message-ID: <67F3FD59-9943-4DE7-86EB-0C503DBA350D@rogvall.se> On 24 sep 2014, at 10:17, Robert Wilkinson wrote: > On Wed, Sep 24, 2014 at 11:55:18AM +0900, Park, Sungjin wrote: >> >> We recently had some malicious packets which were not very big in the first >> place but inflated to really big ones - hundreds of megabytes each. As a >> result, the server crashed with out-of-memory by the processes calling >> zlib:inflate/2. Urgency forced us to make a custom NIF library with >> inflation size limit. We also studied erlang reference manual but couldn't >> find anything useful. The zlib library source code shows even >> zlib:setBufSize/2 does not prevent producing very big binaries. >> >> Not being able to know how big the data would become after inflation, it >> should be a quite common problem. So I'm curious if I missed something >> very simple and nice. Is there anything like that? > > Hi Sungjin > > The articles referenced at http://en.wikipedia.org/wiki/Zip_bomb > should give you some insight into the problem, in general. > Thanks. Cool stuff :-) The following is also a fun version ( I am not the only one to blame for api design faults :-) > xmerl_scan:file("lol.xml"). where "lol.xml" contains: ]> &lol9; The following code could be used to try out various zlib chunk sizes. inflate(Date) -> inflate(Date,1024). inflate(Data,ChunkSize) when is_binary(Data) -> Z = zlib:open(), ok = zlib:inflateInit(Z), inflate_(Z, Data, ChunkSize, 0, []). inflate_(Z, Data, ChunkSize, InflatedSize, Acc) -> case Data of <> -> List = zlib:inflate(Z, Chunk), Size = erlang:iolist_size(List), InflatedSize1 = InflatedSize + Size, io:format("sz = ~w, inflated size = ~w\n", [Size, InflatedSize1]), inflate_(Z, Tail, ChunkSize, InflatedSize1, [List | Acc]); <> -> List = zlib:inflate(Z, LastChunk), zlib:inflateEnd(Z), zlib:close(Z), Size = erlang:iolist_size(List), InflatedSize1 = InflatedSize + Size, io:format("sz = ~w, inflated size = ~w\n", [Size, InflatedSize1]), list_to_binary(lists:reverse([List|Acc])) end. /Tony > Bob > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" From darach@REDACTED Wed Sep 24 20:10:02 2014 From: darach@REDACTED (Darach Ennis) Date: Wed, 24 Sep 2014 19:10:02 +0100 Subject: [erlang-questions] Specs for optional callbacks (try 2) In-Reply-To: References: <5422F6CB.6080403@ninenines.eu> Message-ID: +1. This would really help reduce boilerplate and increase documentation of intent. Cheers, Darach. On 24 Sep 2014 18:17, "Jesse Gumm" wrote: > I would really like to see this as well. I think it's a solid idea. > > +1 from me. > > -Jesse > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866 || sigma-star.com || @jessegumm > On Sep 24, 2014 11:52 AM, "Lo?c Hoguin" wrote: > >> Hi, >> >> I didn't get any reply on my previous question almost 2 years ago here: >> http://erlang.org/pipermail/erlang-questions/2013-January/071763.html >> >> So let me ask again, is there any plans for adding something like >> -optional_callback that would do the same as -callback except it would not >> require the callback to be defined? Basically Dialyzer would use it, and >> the compiler wouldn't fail if the callback is missing. >> >> As it stands now I am about to remove most -behavior in Cowboy because >> they're starting to make little sense because most callbacks become >> optional. The simplest handler type will have 3 possible callbacks, 2 being >> optional. The most complex handler type already has 20+ callbacks with only >> 1 required. >> >> Thanks for any reply. >> >> -- >> Lo?c Hoguin >> http://ninenines.eu >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roe.adrian@REDACTED Wed Sep 24 20:17:59 2014 From: roe.adrian@REDACTED (Adrian Roe) Date: Wed, 24 Sep 2014 19:17:59 +0100 Subject: [erlang-questions] Specs for optional callbacks (try 2) In-Reply-To: References: <5422F6CB.6080403@ninenines.eu> Message-ID: <29521538C75D4E27AEB8A151EFB9250C@gmail.com> Indeed - we have lots and lots of use cases where this would be a major help. -- Adrian Roe Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Wednesday, 24 September 2014 at 19:10, Darach Ennis wrote: > +1. This would really help reduce boilerplate and increase documentation of intent. > Cheers, > Darach. > On 24 Sep 2014 18:17, "Jesse Gumm" wrote: > > I would really like to see this as well. I think it's a solid idea. > > +1 from me. > > -Jesse > > -- > > Jesse Gumm > > Owner, Sigma Star Systems > > 414.940.4866 (tel:414.940.4866) || sigma-star.com (http://sigma-star.com) || @jessegumm > > On Sep 24, 2014 11:52 AM, "Lo?c Hoguin" wrote: > > > Hi, > > > > > > I didn't get any reply on my previous question almost 2 years ago here: http://erlang.org/pipermail/erlang-questions/2013-January/071763.html > > > > > > So let me ask again, is there any plans for adding something like -optional_callback that would do the same as -callback except it would not require the callback to be defined? Basically Dialyzer would use it, and the compiler wouldn't fail if the callback is missing. > > > > > > As it stands now I am about to remove most -behavior in Cowboy because they're starting to make little sense because most callbacks become optional. The simplest handler type will have 3 possible callbacks, 2 being optional. The most complex handler type already has 20+ callbacks with only 1 required. > > > > > > Thanks for any reply. > > > > > > -- > > > Lo?c Hoguin > > > http://ninenines.eu > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Wed Sep 24 22:34:12 2014 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 24 Sep 2014 22:34:12 +0200 Subject: [erlang-questions] Fwd: SSL handshake fails In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: Ingela Andin Date: 2014-09-24 14:14 GMT+02:00 Subject: Re: [erlang-questions] SSL handshake fails To: Daniel Abrahamsson Hi! It will of course be included in the next open source release, and merged to github maint shortly. Regards Ingela Erlang/OTP team - Ericsson AB 2014-09-24 11:43 GMT+02:00 Daniel Abrahamsson : > Got exactly the same error for some of our connections after > upgrading from R16B03 to 17.3. > > Will this patch be included with the next stable release on the 17.x > branch? > > //Daniel > > On Tue, Sep 23, 2014 at 2:29 PM, Ingela Andin > wrote: > >> Hi! >> >> After some investigation I have concluded that the server may send an >> SNI-extension, and that if it does so, it shall be empty. >> >> "In this event, the >> server SHALL include an extension of type "server_name" in the >> (extended) server hello. The "extension_data" field of this >> extension SHALL be empty." >> >> >> I do not really see the point in include an empty SNI-extension on the >> server side, but as the RFC says so here comes a patch to handle it. >> >> diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl >> index 22673e4..eee33ef 100644 >> --- a/lib/ssl/src/ssl_handshake.erl >> +++ b/lib/ssl/src/ssl_handshake.erl >> @@ -1732,6 +1732,9 @@ >> dec_hello_extensions(<> #ec_point_formats{ec_point_format_list = >> ECPointFormats}}); >> >> +dec_hello_extensions(<>, >> Acc) when Len == 0 -> >> + dec_hello_extensions(Rest, Acc#hello_extensions{sni = ""}); %% >> Server may send an empy SNI >> + >> dec_hello_extensions(<> ExtData:Len/binary, Rest/binary>>, Acc) -> >> <> = ExtData, >> >> >> Regards Ingela Erlang/OTP Team - Ericsson AB >> >> >> >> 2014-09-19 11:00 GMT+02:00 Iv?n Mart?nez : >> >>> Hello all, >>> I just hired a CentOS 7 server that came with very little software >>> installed. I installed Erlang 17.3 from sources, attached is output of the >>> configure step. Now I'm trying to install zotonic but it fails when trying >>> to do a SSL handshake with github, see below: >>> >>> [ivan@REDACTED zotonic]$ make >>> erl -noshell -s inets -s ssl \ >>> -eval '{ok, saved_to_file} = httpc:request(get, {" >>> https://github.com/rebar/rebar/wiki/rebar", []}, [], [{stream, >>> "./rebar"}])' \ >>> -s init stop >>> {"init terminating in >>> do_boot",{{badmatch,{error,{failed_connect,[{to_address,{"github.com",443}},{inet,[inet],{eoptions,{{{badmatch,<<0 >>> bytes>>},[{ssl_handshake,dec_hello_extensions,2,[{file,"ssl_handshake.erl"},{line,1737}]},{ssl_handshake,decode_handshake,3,[{file,"ssl_handshake.erl"},{line,926}]},{tls_handshake,get_tls_handshake_aux,3,[{file,"tls_handshake.erl"},{line,155}]},{tls_connection,next_state,4,[{file,"tls_connection.erl"},{line,433}]},{gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,503}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]},{gen_fsm,sync_send_all_state_event,[<0.54.0>,{start,infinity},infinity]}}}}]}}},[{erl_eval,expr,3,[]}]}} >>> >>> Crash dump was written to: erl_crash.dump >>> init terminating in do_boot () >>> make: *** [rebar] Error 1 >>> >>> I tried to do the handshake with openssl and apparently it works: >>> >>> [ivan@REDACTED zotonic]$ openssl s_client -host github.com -port 443 >>> CONNECTED(00000003) >>> depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert >>> High Assurance EV Root CA >>> verify return:1 >>> depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert >>> SHA2 Extended Validation Server CA >>> verify return:1 >>> depth=0 businessCategory = Private Organization, >>> 1.3.6.1.4.1.311.60.2.1.3 = US, 1.3.6.1.4.1.311.60.2.1.2 = Delaware, >>> serialNumber = 5157550, street = 548 4th Street, postalCode = 94107, C = >>> US, ST = California, L = San Francisco, O = "GitHub, Inc.", CN = >>> github.com >>> verify return:1 >>> --- >>> Certificate chain >>> 0 s:/businessCategory=Private >>> Organization/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/serialNumber=5157550/street=548 >>> 4th Street/postalCode=94107/C=US/ST=California/L=San Francisco/O=GitHub, >>> Inc./CN=github.com >>> i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended >>> Validation Server CA >>> 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended >>> Validation Server CA >>> i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High >>> Assurance EV Root CA >>> --- >>> Server certificate >>> -----BEGIN CERTIFICATE----- >>> MIIF4DCCBMigAwIBAgIQDACTENIG2+M3VTWAEY3chzANBgkqhkiG9w0BAQsFADB1 >>> ... >>> XX4C2NesiZcLYbc2n7B9O+63M2k= >>> -----END CERTIFICATE----- >>> subject=/businessCategory=Private >>> Organization/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/serialNumber=5157550/street=548 >>> 4th Street/postalCode=94107/C=US/ST=California/L=San Francisco/O=GitHub, >>> Inc./CN=github.com >>> issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 >>> Extended Validation Server CA >>> --- >>> No client certificate CA names sent >>> Server Temp Key: ECDH, prime256v1, 256 bits >>> --- >>> SSL handshake has read 3233 bytes and written 375 bytes >>> --- >>> New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256 >>> Server public key is 2048 bit >>> Secure Renegotiation IS supported >>> Compression: NONE >>> Expansion: NONE >>> SSL-Session: >>> Protocol : TLSv1.2 >>> Cipher : ECDHE-RSA-AES128-GCM-SHA256 >>> Session-ID: >>> DDEF6E78852287351EC5B20FFDD2578F8996E7226CB883A5F1A94325048B79C6 >>> Session-ID-ctx: >>> Master-Key: >>> D6C6283F463BFCD5A160E0CCE0CC8962CF944E5C98153040E4BC20466981B1622A5327C1E6BBED5F1751A049782908E5 >>> Key-Arg : None >>> Krb5 Principal: None >>> PSK identity: None >>> PSK identity hint: None >>> Start Time: 1411113552 >>> Timeout : 300 (sec) >>> Verify return code: 0 (ok) >>> --- >>> closed >>> >>> What can be wrong?. Thank you. >>> Ivan >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Wed Sep 24 23:04:15 2014 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 25 Sep 2014 00:04:15 +0300 Subject: [erlang-questions] Specs for optional callbacks (try 2) In-Reply-To: <5422F6CB.6080403@ninenines.eu> References: <5422F6CB.6080403@ninenines.eu> Message-ID: <542331CF.9040509@cs.ntua.gr> On 09/24/2014 07:52 PM, Lo?c Hoguin wrote: > So let me ask again, is there any plans for adding something like > -optional_callback that would do the same as -callback except it would > not require the callback to be defined? Basically Dialyzer would use it, > and the compiler wouldn't fail if the callback is missing. I think this proposal makes sense, if this situation is indeed quite common in user-defined behaviors. One question, esp. if the numbers mentioned below are indicative of expected uses of this feature is whether it makes sense to introduce yet another spec-variant attribute named -optional_callback to specify the types of optional callbacks or uniformly keep the -callback attribute for specifying the types of all callbacks and introduce just a declaration of which callbacks are optional. If the previous sentence is too complicated, I am asking which of the two proposals below is preferable: %%----- option 1 ------- -callback foo(...) -> ... -callback bar(...) -> ... -optional_callback baz(...) -> ... -optional_callback boo(...) -> ... %%----- option 1 ------- -callback foo(...) -> ... -callback bar(...) -> ... -callback baz(...) -> ... -callback boo(...) -> ... -optional_callback([baz/3, boo/17]). The two possibilities sketched above have pros and cons, both technical and aesthetic ones. Any opinions? > As it stands now I am about to remove most -behavior in Cowboy because > they're starting to make little sense because most callbacks become > optional. The simplest handler type will have 3 possible callbacks, 2 > being optional. The most complex handler type already has 20+ callbacks > with only 1 required. Kostis From ivan.martinez@REDACTED Wed Sep 24 21:00:32 2014 From: ivan.martinez@REDACTED (=?UTF-8?B?SXbDoW4gTWFydMOtbmV6?=) Date: Wed, 24 Sep 2014 21:00:32 +0200 Subject: [erlang-questions] Could not open pty master Message-ID: Hello all, Does anyone know why I'm having the following issue with a CentOS 7 system with kernel 3.10.23?: $ run_erl priv/ log "erl" run_erl:187 [6505] Wed Sep 24 18:42:39 2014 errno=1 'Operation not permitted' Could not open pty master It works as super user. It also works in a Fedora 20 system with kernel 3.16. I couldn't find any difference in user groups or /dev file permits between both systems. Starting with a user with UID above or below UID_MIN doesn't make any difference. I don't think it matters, but the only difference I could find is that the CentOS has many /dev/pty* files already created, while the Fedora doesn't have any. Thank you, Ivan Martinez -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Wed Sep 24 23:47:48 2014 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 24 Sep 2014 14:47:48 -0700 Subject: [erlang-questions] Specs for optional callbacks (try 2) In-Reply-To: <542331CF.9040509@cs.ntua.gr> References: <5422F6CB.6080403@ninenines.eu> <542331CF.9040509@cs.ntua.gr> Message-ID: <54233C04.7020205@gmail.com> On 09/24/2014 02:04 PM, Kostis Sagonas wrote: > On 09/24/2014 07:52 PM, Lo?c Hoguin wrote: >> So let me ask again, is there any plans for adding something like >> -optional_callback that would do the same as -callback except it would >> not require the callback to be defined? Basically Dialyzer would use it, >> and the compiler wouldn't fail if the callback is missing. > > I think this proposal makes sense, if this situation is indeed quite common in user-defined behaviors. > > One question, esp. if the numbers mentioned below are indicative of expected uses of this feature is whether it makes sense to introduce yet another spec-variant attribute named -optional_callback to specify the types of optional callbacks or uniformly keep the -callback attribute for specifying the types of all callbacks and introduce just a declaration of which callbacks are optional. > > If the previous sentence is too complicated, I am asking which of the two proposals below is preferable: > > %%----- option 1 ------- > > -callback foo(...) -> ... > -callback bar(...) -> ... > -optional_callback baz(...) -> ... > -optional_callback boo(...) -> ... I prefer option 1 because I find it more immediately descriptive. However, I think the ordering is better as "callback_optional" since it is definitely a callback (like the other callbacks), "optional" is just a adjective to modify what type of callback is being defined. So, having "callback" first provides a contextual scope and an aid for the human eye as it tracks from the upper-left to the lower-right. > > > %%----- option 1 ------- > > -callback foo(...) -> ... > -callback bar(...) -> ... > -callback baz(...) -> ... > -callback boo(...) -> ... > > -optional_callback([baz/3, boo/17]). > > > The two possibilities sketched above have pros and cons, both technical and aesthetic ones. Any opinions? > >> As it stands now I am about to remove most -behavior in Cowboy because >> they're starting to make little sense because most callbacks become >> optional. The simplest handler type will have 3 possible callbacks, 2 >> being optional. The most complex handler type already has 20+ callbacks >> with only 1 required. > > Kostis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Thu Sep 25 00:18:54 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Thu, 25 Sep 2014 01:18:54 +0300 Subject: [erlang-questions] Specs for optional callbacks (try 2) In-Reply-To: <54233C04.7020205@gmail.com> References: <5422F6CB.6080403@ninenines.eu> <542331CF.9040509@cs.ntua.gr> <54233C04.7020205@gmail.com> Message-ID: <5423434E.90409@ninenines.eu> On 09/25/2014 12:47 AM, Michael Truog wrote: > On 09/24/2014 02:04 PM, Kostis Sagonas wrote: >> On 09/24/2014 07:52 PM, Lo?c Hoguin wrote: >>> So let me ask again, is there any plans for adding something like >>> -optional_callback that would do the same as -callback except it would >>> not require the callback to be defined? Basically Dialyzer would use it, >>> and the compiler wouldn't fail if the callback is missing. >> >> I think this proposal makes sense, if this situation is indeed quite >> common in user-defined behaviors. >> >> One question, esp. if the numbers mentioned below are indicative of >> expected uses of this feature is whether it makes sense to introduce >> yet another spec-variant attribute named -optional_callback to specify >> the types of optional callbacks or uniformly keep the -callback >> attribute for specifying the types of all callbacks and introduce just >> a declaration of which callbacks are optional. >> >> If the previous sentence is too complicated, I am asking which of the >> two proposals below is preferable: >> >> %%----- option 1 ------- >> >> -callback foo(...) -> ... >> -callback bar(...) -> ... >> -optional_callback baz(...) -> ... >> -optional_callback boo(...) -> ... > > I prefer option 1 because I find it more immediately descriptive. > However, I think the ordering is better as "callback_optional" since it > is definitely a callback (like the other callbacks), "optional" is just > a adjective to modify what type of callback is being defined. So, > having "callback" first provides a contextual scope and an aid for the > human eye as it tracks from the upper-left to the lower-right. I think this makes sense, though "callback_optional" is a bit weird to read. Regardless of the actual name chosen I find the one liner easier to read than separated like option 2. I wish I could define and export types in one line too. -- Lo?c Hoguin http://ninenines.eu From zxq9@REDACTED Thu Sep 25 03:21:12 2014 From: zxq9@REDACTED (zxq9) Date: Thu, 25 Sep 2014 10:21:12 +0900 Subject: [erlang-questions] Could not open pty master In-Reply-To: References: Message-ID: <4676551.iEBOCzXdtb@burrito> On Wednesday 24 September 2014 21:00:32 Iv?n Mart?nez wrote: > Hello all, > Does anyone know why I'm having the following issue with a CentOS 7 system > with kernel 3.10.23?: > > $ run_erl priv/ log "erl" > run_erl:187 [6505] Wed Sep 24 18:42:39 2014 > errno=1 'Operation not permitted' > Could not open pty master > > It works as super user. It also works in a Fedora 20 system with kernel > 3.16. I couldn't find any difference in user groups or /dev file permits > between both systems. Starting with a user with UID above or below UID_MIN > doesn't make any difference. I don't think it matters, but the only > difference I could find is that the CentOS has many /dev/pty* files already > created, while the Fedora doesn't have any. You might be running into SELinux permission issues. To find out try doing "setenforce 0" and then running it again. If that works, use a tool like audit2allow or audit2why to create a policy that will permit the actions you require to run your program. I haven't kept up with the Fedora/RHEL world since 7 came out, but Dan Walsh's blog and Red Hat's SELinux docs have been good resources on this in the past. Of course, you might have a totally different issue, but SELinux booleans and audit logs are the first thing I check on a Fedora-type distro when something doesn't work but looks like it should. From eranga.erl@REDACTED Thu Sep 25 04:37:07 2014 From: eranga.erl@REDACTED (Eranga Udesh) Date: Thu, 25 Sep 2014 08:07:07 +0530 Subject: [erlang-questions] Erlang Memory Question In-Reply-To: References: <9A5764B7-9252-4560-9AD6-BEC0DAE0305E@cs.otago.ac.nz> Message-ID: Thanks all for your advice. Let me see how I can apply them to my program. It looks like my code/logic is going to get ugly and induce a performance penalty, in order to save memory. Cheers, - Eranga On Wed, Sep 24, 2014 at 7:58 PM, Robert Raschke wrote: > I always thought that is one of the reasons to have processes. > If you've got something big you want to throw away quickly, make a process > for it. > > $ erl > Erlang R16B03-1 (erts-5.10.4) [source] [64-bit] [smp:8:8] > [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V5.10.4 (abort with ^G) > 1> erlang:memory(). > [{total,14148416}, > {processes,4091608}, > {processes_used,4091488}, > {system,10056808}, > {atom,194289}, > {atom_used,172614}, > {binary,1514552}, > {code,4026600}, > {ets,262688}] > 2> Pid = spawn( > 2> fun () -> > 2> X = binary:copy(<<1,2,3,4,5,6,7,8>>, 1024), > 2> Y = binary:copy(X, 1024), > 2> receive stop -> ok end > 2> end > 2> ). > <0.35.0> > 3> erlang:memory(). > [{total,22643832}, > {processes,4203448}, > {processes_used,4203448}, > {system,18440384}, > {atom,194289}, > {atom_used,175110}, > {binary,9685320}, > {code,4221791}, > {ets,267056}] > 4> Pid ! stop. > stop > 5> erlang:memory(). > [{total,13587776}, > {processes,4084496}, > {processes_used,4084384}, > {system,9503280}, > {atom,194289}, > {atom_used,175110}, > {binary,748144}, > {code,4221791}, > {ets,267056}] > 6> > > > Regards, > Robby > > > On 24 September 2014 02:13, Eranga Udesh wrote: > >> Well, yes I may deliberately lie. >> >> However my suggestion is to, instead of doing a full sweep by the garbage >> collector (GC) to identify data going out of scope and reclaim, can the >> program (or rather I) deliberately say I (the calling process) is finished >> using the said data, so the GC may free that part. >> >> Then the GC may carry out it's own logic, which it currently does to >> verify if the same data is referenced by any other processes, etc., and >> decide if to GC or not. >> >> = Eranga >> >> On Tue, Sep 23, 2014 at 12:42 PM, Richard A. O'Keefe >> wrote: >> >>> >>> On 23/09/2014, at 2:24 PM, Eranga Udesh wrote: >>> >>> > Thanks for the information received so far. >>> > >>> > Wouldn't it be good for Erlang to have a single object garbage >>> collection function/bif? >>> >>> NO. In C it's called free() and it's a major cause of errors. >>> >>> > For example, when I no longer require a large object, I force to >>> garbage collect it, without making a full sweep? >>> >>> Why should the garbage collector *believe* you that the "object" >>> is free? You could be deliberately lying. You could (and >>> probably are) be mistaken. How is it to know which bits you >>> want to keep without doing its usual thing? In a shared heap >>> implementation (which Erlang has had and may have again) the >>> fact that *you've* finished with the object doesn't mean >>> everyone *else* has. A meaningful operation should not depend >>> on implementation details like that. >>> > >>> > As mentioned in the document, a full sweep may degrade the performance. >>> >>> Not half as much as freeing too much would! >>> >>> This is micro-optimisation. Avoid passing around large >>> objects in the first place. >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From samjbarney@REDACTED Wed Sep 24 23:36:53 2014 From: samjbarney@REDACTED (Samuel Barney) Date: Wed, 24 Sep 2014 15:36:53 -0600 Subject: [erlang-questions] Question about the VM In-Reply-To: References: Message-ID: I did look at targeting Core Erlang first, but it doesn't allow for native loops. I could emulate them with function calls, however, if I move down to the assembly level I can build loops with what is provided there. Thank you for the information, I'll do some more searching to see what I can find. Respectfully, Samuel Barney On Wed, Sep 24, 2014 at 4:40 AM, Anthony Ramine wrote: > Did you try targeting Core Erlang first? Which primitives do you want to > access in BEAM? > > There are 1024 X registers, and Y registers are used for locals in > functions, to save them when calling other functions. > > You can find such information by reading the source code or various talk > slides made by the OTP team these last years, but I can't help you find the > links right now. > > Regards, > > Le 24 sept. 2014 ? 04:40, Samuel Barney a ?crit : > > > I'm developing a language on top of the Erlang VM, and there are a > couple of flow-control constructs that I can't implement without skipping > the Erlang AST and using the erlang assembly code. > > > > Is there a specific amount of X and Y registers? Or is there a way that > I can find that out via Erlang? > > > > Also, what is the purpose of the Y registers? > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Sep 25 06:11:14 2014 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 25 Sep 2014 16:11:14 +1200 Subject: [erlang-questions] zlib design flaw? In-Reply-To: <67F3FD59-9943-4DE7-86EB-0C503DBA350D@rogvall.se> References: <20140924081742.GA23308@fourtheye.org> <67F3FD59-9943-4DE7-86EB-0C503DBA350D@rogvall.se> Message-ID: <3544A67E-2805-4E41-A82D-D2ABD429F372@cs.otago.ac.nz> On 25/09/2014, at 5:46 AM, Tony Rogvall wrote: > > Thanks. Cool stuff :-) > > The following is also a fun version ( I am not the only one to blame for api design faults :-) [[billion laughs]] Wikimedia markup has the same problem. For that matter, templates make the C++ type language a Turing-complete (but seriously ugly) functional programming language, and the various features that have been added to the Haskell type system since 2010 make that a Turing-complete logic programming language, so you can write a fairly short C++ or Haskell program that takes as long to type-check as you please. The Erlang binary term representation is just on the safe side of this kind of attack. The fact that backreferences can only be used to refer to *atoms* keeps it safe. I'm a little bit sad that this seems like a good reason not to make the binary representation "smarter". Oh heck. The binary format I use to provide persistence for Smalltalk has precisely this kind of problem, but if I *didn't* allow backreferences to arbitrary objects I would not be able to handle cyclic object graphs. I console myself: UBF(A) is on the wrong side of the safety line. The term you get by decoding n UBF(a) bytes will be O(n) in size, but it may take O(2**n) time to traverse it. From jinni.park@REDACTED Thu Sep 25 09:45:53 2014 From: jinni.park@REDACTED (Park, Sungjin) Date: Thu, 25 Sep 2014 16:45:53 +0900 Subject: [erlang-questions] zlib design flaw? In-Reply-To: <3544A67E-2805-4E41-A82D-D2ABD429F372@cs.otago.ac.nz> References: <20140924081742.GA23308@fourtheye.org> <67F3FD59-9943-4DE7-86EB-0C503DBA350D@rogvall.se> <3544A67E-2805-4E41-A82D-D2ABD429F372@cs.otago.ac.nz> Message-ID: Related with this problem, I tried attached simple patches to define zlib:inflate/3. The last parameter, MaxSize is handed over to the port driver. And changed the port driver to stop streaming at this point. Patches compile good but at runtime, I get undefined function exception even it's exported by zlib.erl 1> Data=<<"0123456789">>. 2> Compressed=zlib:compress(Data). 3> Z=zlib:open(). 4> zlib:inflateInit(Z). 5> catch zlib:inflate(Z,Compressed,9). {'EXIT',{undef,[{zlib,inflate, [#Port<0.582>, <<120,156,51,48,52,50,54,49,53,51,183,0,0,8,241,1,213>>, 9], []}, {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,661}]}, {erl_eval,expr,5,[{file,"erl_eval.erl"},{line,434}]}, {shell,exprs,7,[{file,"shell.erl"},{line,676}]}, {shell,eval_exprs,7,[{file,"shell.erl"},{line,631}]}, {shell,eval_loop,3,[{file,"shell.erl"},{line,616}]}]}} ... And also I get einval error (maybe from the port driver) when I call zlib:inflate/2. 6> catch zlib:inflate(Z,Compressed). {'EXIT',{einval,[{zlib,call,3,[]}, {zlib,inflate,2,[]}, {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,661}]}, {erl_eval,expr,5,[{file,"erl_eval.erl"},{line,434}]}, {shell,exprs,7,[{file,"shell.erl"},{line,676}]}, {shell,eval_exprs,7,[{file,"shell.erl"},{line,631}]}, {shell,eval_loop,3,[{file,"shell.erl"},{line,616}]}]}} I suspect that the port driver patch is applied but zlib.erl is not. Is there any other place that I have to define zlib:inflate/3? Or did I make other mistake? On Thu, Sep 25, 2014 at 1:11 PM, Richard A. O'Keefe wrote: > > On 25/09/2014, at 5:46 AM, Tony Rogvall wrote: > > > > Thanks. Cool stuff :-) > > > > The following is also a fun version ( I am not the only one to blame for > api design faults :-) > > [[billion laughs]] > > Wikimedia markup has the same problem. > > For that matter, templates make the C++ type language > a Turing-complete (but seriously ugly) functional programming > language, and the various features that have been added to > the Haskell type system since 2010 make that a Turing-complete > logic programming language, so you can write a fairly short > C++ or Haskell program that takes as long to type-check as > you please. > > The Erlang binary term representation is just on the safe side > of this kind of attack. The fact that backreferences can only > be used to refer to *atoms* keeps it safe. > > I'm a little bit sad that this seems like a good reason not > to make the binary representation "smarter". > > Oh heck. The binary format I use to provide persistence for > Smalltalk has precisely this kind of problem, but if I > *didn't* allow backreferences to arbitrary objects I would not > be able to handle cyclic object graphs. > > I console myself: UBF(A) is on the wrong side of the safety > line. The term you get by decoding n UBF(a) bytes will be > O(n) in size, but it may take O(2**n) time to traverse it. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: zlib_drv.patch Type: application/octet-stream Size: 1954 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: zlib_SUITE.patch Type: application/octet-stream Size: 651 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: zlib.patch Type: application/octet-stream Size: 1081 bytes Desc: not available URL: From jinni.park@REDACTED Thu Sep 25 10:48:50 2014 From: jinni.park@REDACTED (Park, Sungjin) Date: Thu, 25 Sep 2014 17:48:50 +0900 Subject: [erlang-questions] zlib design flaw? In-Reply-To: References: <20140924081742.GA23308@fourtheye.org> <67F3FD59-9943-4DE7-86EB-0C503DBA350D@rogvall.se> <3544A67E-2805-4E41-A82D-D2ABD429F372@cs.otago.ac.nz> Message-ID: Organized the patch files again that you can just patch -p1 < zlib-patch.txt On Thu, Sep 25, 2014 at 4:45 PM, Park, Sungjin wrote: > Related with this problem, I tried attached simple patches to define > zlib:inflate/3. The last parameter, MaxSize is handed over to the port > driver. And changed the port driver to stop streaming at this point. > > Patches compile good but at runtime, I get undefined function exception > even it's exported by zlib.erl > > 1> Data=<<"0123456789">>. > 2> Compressed=zlib:compress(Data). > 3> Z=zlib:open(). > 4> zlib:inflateInit(Z). > 5> catch zlib:inflate(Z,Compressed,9). > {'EXIT',{undef,[{zlib,inflate, > [#Port<0.582>, > > <<120,156,51,48,52,50,54,49,53,51,183,0,0,8,241,1,213>>, > 9], > []}, > {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,661}]}, > {erl_eval,expr,5,[{file,"erl_eval.erl"},{line,434}]}, > {shell,exprs,7,[{file,"shell.erl"},{line,676}]}, > {shell,eval_exprs,7,[{file,"shell.erl"},{line,631}]}, > {shell,eval_loop,3,[{file,"shell.erl"},{line,616}]}]}} > ... > > And also I get einval error (maybe from the port driver) when I call > zlib:inflate/2. > > 6> catch zlib:inflate(Z,Compressed). > {'EXIT',{einval,[{zlib,call,3,[]}, > {zlib,inflate,2,[]}, > {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,661}]}, > {erl_eval,expr,5,[{file,"erl_eval.erl"},{line,434}]}, > {shell,exprs,7,[{file,"shell.erl"},{line,676}]}, > {shell,eval_exprs,7,[{file,"shell.erl"},{line,631}]}, > {shell,eval_loop,3,[{file,"shell.erl"},{line,616}]}]}} > > I suspect that the port driver patch is applied but zlib.erl is not. > Is there any other place that I have to define zlib:inflate/3? Or did I > make other mistake? > > > On Thu, Sep 25, 2014 at 1:11 PM, Richard A. O'Keefe > wrote: > >> >> On 25/09/2014, at 5:46 AM, Tony Rogvall wrote: >> > >> > Thanks. Cool stuff :-) >> > >> > The following is also a fun version ( I am not the only one to blame >> for api design faults :-) >> >> [[billion laughs]] >> >> Wikimedia markup has the same problem. >> >> For that matter, templates make the C++ type language >> a Turing-complete (but seriously ugly) functional programming >> language, and the various features that have been added to >> the Haskell type system since 2010 make that a Turing-complete >> logic programming language, so you can write a fairly short >> C++ or Haskell program that takes as long to type-check as >> you please. >> >> The Erlang binary term representation is just on the safe side >> of this kind of attack. The fact that backreferences can only >> be used to refer to *atoms* keeps it safe. >> >> I'm a little bit sad that this seems like a good reason not >> to make the binary representation "smarter". >> >> Oh heck. The binary format I use to provide persistence for >> Smalltalk has precisely this kind of problem, but if I >> *didn't* allow backreferences to arbitrary objects I would not >> be able to handle cyclic object graphs. >> >> I console myself: UBF(A) is on the wrong side of the safety >> line. The term you get by decoding n UBF(a) bytes will be >> O(n) in size, but it may take O(2**n) time to traverse it. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Park, Sungjin > > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > > ------------------------------------------------------------------------------------------------------------------- > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- diff --git a/erts/emulator/drivers/common/zlib_drv.c b/erts/emulator/drivers/common/zlib_drv.c index 3143e45..bf2f63c 100644 --- a/erts/emulator/drivers/common/zlib_drv.c +++ b/erts/emulator/drivers/common/zlib_drv.c @@ -64,6 +64,8 @@ #define DEFAULT_BUFSZ 4000 +#define ZLIB_PORT_ERROR 90000 + static int zlib_init(void); static ErlDrvData zlib_start(ErlDrvPort port, char* buf); static void zlib_stop(ErlDrvData e); @@ -113,10 +115,12 @@ typedef struct { uLong crc; int inflate_eos_seen; int want_crc; /* 1 if crc is calculated on clear text */ + int sentsz; + int sentsz_max; ErlDrvPort port; /* the associcated port */ } ZLibData; -static int zlib_inflate(ZLibData* d, int flush); +static int zlib_inflate(ZLibData* d, int flush, int sentsz_max); static int zlib_deflate(ZLibData* d, int flush); #if defined(__WIN32__) @@ -159,6 +163,9 @@ static char* zlib_reason(int code, int* err) case Z_VERSION_ERROR: *err = 1; return "version_error"; + case ZLIB_PORT_ERROR: + *err = 1; + return "port_error"; default: *err = 1; return "unknown_error"; @@ -226,19 +233,32 @@ static int zlib_output_init(ZLibData* d) static int zlib_output(ZLibData* d) { if (d->bin != NULL) { - int len = d->binsz - d->s.avail_out; - if (len > 0) { - if (driver_output_binary(d->port, NULL, 0, d->bin, 0, len) < 0) - return -1; - } - driver_free_binary(d->bin); - d->bin = NULL; - d->binsz = 0; + int len = d->binsz - d->s.avail_out; + if (len > 0) { + int will_be_sentsz = d->sentsz+len; + if (d->sentsz_max > 0 && will_be_sentsz > d->sentsz_max) { + driver_output_binary(d->port, NULL, 0, NULL, 0, 0); + return -1; + } else { + if (driver_output_binary(d->port, NULL, 0, d->bin, 0, len) < 0) + return -1; + d->sentsz = will_be_sentsz; + } + } + driver_free_binary(d->bin); + d->bin = NULL; + d->binsz = 0; } return zlib_output_init(d); } -static int zlib_inflate(ZLibData* d, int flush) +/** + * Start inflating data. + * sentsz_max : max number of bytes that can be sent to the emulator + * Emulator receives an empty binary on overflow. + * sentsz_max = 0 means infinity. + */ +static int zlib_inflate(ZLibData* d, int flush, int sentsz_max) { int res = Z_OK; @@ -247,6 +267,9 @@ static int zlib_inflate(ZLibData* d, int flush) return Z_ERRNO; } + d->sentsz_max = sentsz_max; + d->sentsz = 0; + while ((driver_sizeq(d->port) > 0) && (res != Z_STREAM_END)) { int vlen; SysIOVec* iov = driver_peekq(d->port, &vlen); @@ -255,7 +278,7 @@ static int zlib_inflate(ZLibData* d, int flush) d->s.next_in = iov[0].iov_base; d->s.avail_in = iov[0].iov_len; - while((possibly_more_output || (d->s.avail_in > 0)) && (res != Z_STREAM_END)) { + while((possibly_more_output || (d->s.avail_in > 0)) && (res != Z_STREAM_END) && (res != ZLIB_PORT_ERROR)) { res = inflate(&d->s, Z_NO_FLUSH); if (res == Z_NEED_DICT) { /* Essential to eat the header bytes that zlib has looked at */ @@ -276,7 +299,8 @@ static int zlib_inflate(ZLibData* d, int flush) if (d->want_crc) d->crc = crc32(d->crc, (unsigned char*)d->bin->orig_bytes, d->binsz - d->s.avail_out); - zlib_output(d); + if (zlib_output(d) < 0) + res = ZLIB_PORT_ERROR; possibly_more_output = 1; } } @@ -288,7 +312,8 @@ static int zlib_inflate(ZLibData* d, int flush) d->crc = crc32(d->crc, (unsigned char*) d->bin->orig_bytes, d->binsz - d->s.avail_out); } - zlib_output(d); + if (res != ZLIB_PORT_ERROR) + zlib_output(d); if (res == Z_STREAM_END) { d->inflate_eos_seen = 1; } @@ -304,6 +329,9 @@ static int zlib_deflate(ZLibData* d, int flush) return Z_ERRNO; } + d->sentsz_max = 0; + d->sentsz = 0; + while ((driver_sizeq(d->port) > 0) && (res != Z_STREAM_END)) { int vlen; SysIOVec* iov = driver_peekq(d->port, &vlen); @@ -560,8 +588,8 @@ static ErlDrvSSizeT zlib_ctl(ErlDrvData drv_data, unsigned int command, char *bu case INFLATE: if (d->state != ST_INFLATE) goto badarg; - if (len != 4) goto badarg; - res = zlib_inflate(d, i32(buf)); + if (len != 8) goto badarg; + res = zlib_inflate(d, i32(buf), i32(buf+4)); if (res == Z_NEED_DICT) { return zlib_value2(3, d->s.adler, rbuf, rlen); } else { diff --git a/erts/preloaded/src/zlib.erl b/erts/preloaded/src/zlib.erl index df7b2e6..1a1ee0e 100644 --- a/erts/preloaded/src/zlib.erl +++ b/erts/preloaded/src/zlib.erl @@ -23,7 +23,7 @@ deflateSetDictionary/2,deflateReset/1,deflateParams/3, deflate/2,deflate/3,deflateEnd/1, inflateInit/1,inflateInit/2,inflateSetDictionary/2, - inflateSync/1,inflateReset/1,inflate/2,inflateEnd/1, + inflateSync/1,inflateReset/1,inflate/2,inflate/3,inflateEnd/1, setBufSize/2,getBufSize/1, crc32/1,crc32/2,crc32/3,adler32/2,adler32/3,getQSize/1, crc32_combine/4,adler32_combine/4, @@ -212,6 +212,9 @@ deflate(Z, Data, Flush) -> _ = call(Z, ?DEFLATE, <<(arg_flush(Flush)):32>>), collect(Z) catch + throw:Reason -> + flush(Z), + erlang:exit(Reason); error:_Err -> flush(Z), erlang:error(badarg) @@ -253,11 +256,19 @@ inflateReset(Z) -> Data :: iodata(), Decompressed :: iolist(). inflate(Z, Data) -> + inflate(Z, Data, infinity). + +inflate(Z, Data, infinity) -> + inflate(Z, Data, 0); +inflate(Z, Data, MaxSize) when erlang:is_integer(MaxSize) -> try port_command(Z, Data) of true -> - _ = call(Z, ?INFLATE, <>), + _ = call(Z, ?INFLATE, <>), collect(Z) catch + throw:Reason -> + flush(Z), + erlang:exit(Reason); error:_Err -> flush(Z), erlang:error(badarg) @@ -456,8 +467,10 @@ collect(Z) -> -spec collect(zstream(), iolist()) -> iolist(). collect(Z, Acc) -> receive + {Z, {data, <<>>}} -> + erlang:throw(overflow); {Z, {data, Bin}} -> - collect(Z, [Bin|Acc]) + collect(Z, [Bin|Acc]) after 0 -> reverse(Acc) end. diff --git a/lib/kernel/test/zlib_SUITE.erl b/lib/kernel/test/zlib_SUITE.erl index 3be6f39..92c6e5f 100644 --- a/lib/kernel/test/zlib_SUITE.erl +++ b/lib/kernel/test/zlib_SUITE.erl @@ -82,7 +82,7 @@ groups() -> api_deflateSetDictionary, api_deflateReset, api_deflateParams, api_deflate, api_deflateEnd, api_inflateInit, api_inflateSetDictionary, - api_inflateSync, api_inflateReset, api_inflate, + api_inflateSync, api_inflateReset, api_inflate, api_inflateUpto, api_inflateEnd, api_setBufsz, api_getBufsz, api_crc32, api_adler32, api_getQSize, api_un_compress, api_un_zip, api_g_un_zip]}, @@ -357,6 +357,18 @@ api_inflate(Config) when is_list(Config) -> ?m({'EXIT',{data_error,_}}, zlib:inflate(Z1, <<2,1,2,1,2>>)), ?m(ok, zlib:close(Z1)). +api_inflateUpto(doc) -> "Test inflate up to given max size limit"; +api_inflateUpto(suite) -> []; +api_inflateUpto(Config) when is_list(Config) -> + Data = [<<1,2,2,3,3,3,4,4,4,4>>], + ?line Compressed = zlib:compress(Data), + ?line Z1 = zlib:open(), + ?m(ok, zlib:inflateInit(Z1)), + Options = [{max_size, erlang:iolist_size(Data)-1}], + ?m({'EXIT', overflow}, zlib:inflate(Z1, Compressed, Options)), + ?m(ok, zlib:inflateEnd(Z1)), + ?m(ok, zlib:close(Z1)). + api_inflateEnd(doc) -> "Test inflateEnd"; api_inflateEnd(suite) -> []; api_inflateEnd(Config) when is_list(Config) -> From n.oxyde@REDACTED Thu Sep 25 11:07:57 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 25 Sep 2014 11:07:57 +0200 Subject: [erlang-questions] Question about the VM In-Reply-To: References: Message-ID: What do you call native loops? Are you planning to generate BEAM code that loops using plain jump instructions? What do you do in that loop? Can you be sure your process will still yield in a cooperative fashion to not hang the VM? Did you measure that you actually need such a thing? Did you check that you will still be able to feed that assembly to the erlc compiler to benefit from the other usual instructions? In the end couldn't you just use HiPE to take care of these loops? Regards, -- Anthony Ramine > Le 24 sept. 2014 ? 23:36, Samuel Barney a ?crit : > > I did look at targeting Core Erlang first, but it doesn't allow for native loops. I could emulate them with function calls, however, if I move down to the assembly level I can build loops with what is provided there. > > Thank you for the information, I'll do some more searching to see what I can find. > > Respectfully, > Samuel Barney > >> On Wed, Sep 24, 2014 at 4:40 AM, Anthony Ramine wrote: >> Did you try targeting Core Erlang first? Which primitives do you want to access in BEAM? >> >> There are 1024 X registers, and Y registers are used for locals in functions, to save them when calling other functions. >> >> You can find such information by reading the source code or various talk slides made by the OTP team these last years, but I can't help you find the links right now. >> >> Regards, >> >> Le 24 sept. 2014 ? 04:40, Samuel Barney a ?crit : >> >> > I'm developing a language on top of the Erlang VM, and there are a couple of flow-control constructs that I can't implement without skipping the Erlang AST and using the erlang assembly code. >> > >> > Is there a specific amount of X and Y registers? Or is there a way that I can find that out via Erlang? >> > >> > Also, what is the purpose of the Y registers? >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan.martinez@REDACTED Thu Sep 25 10:41:49 2014 From: ivan.martinez@REDACTED (=?UTF-8?B?SXbDoW4gTWFydMOtbmV6?=) Date: Thu, 25 Sep 2014 10:41:49 +0200 Subject: [erlang-questions] Could not open pty master In-Reply-To: <4676551.iEBOCzXdtb@burrito> References: <4676551.iEBOCzXdtb@burrito> Message-ID: Thank you. Looks like SELinux has nothing to do with the issue: $ setenforce 0 setenforce: SELinux is disabled 2014-09-25 3:21 GMT+02:00 zxq9 : > On Wednesday 24 September 2014 21:00:32 Iv?n Mart?nez wrote: > > Hello all, > > Does anyone know why I'm having the following issue with a CentOS 7 > system > > with kernel 3.10.23?: > > > > $ run_erl priv/ log "erl" > > run_erl:187 [6505] Wed Sep 24 18:42:39 2014 > > errno=1 'Operation not permitted' > > Could not open pty master > > > > It works as super user. It also works in a Fedora 20 system with kernel > > 3.16. I couldn't find any difference in user groups or /dev file permits > > between both systems. Starting with a user with UID above or below > UID_MIN > > doesn't make any difference. I don't think it matters, but the only > > difference I could find is that the CentOS has many /dev/pty* files > already > > created, while the Fedora doesn't have any. > > You might be running into SELinux permission issues. To find out try doing > "setenforce 0" and then running it again. If that works, use a tool like > audit2allow or audit2why to create a policy that will permit the actions > you > require to run your program. I haven't kept up with the Fedora/RHEL world > since 7 came out, but Dan Walsh's blog and Red Hat's SELinux docs have been > good resources on this in the past. > > Of course, you might have a totally different issue, but SELinux booleans > and > audit logs are the first thing I check on a Fedora-type distro when > something > doesn't work but looks like it should. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Thu Sep 25 12:26:29 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 25 Sep 2014 12:26:29 +0200 Subject: [erlang-questions] Specs for optional callbacks (try 2) In-Reply-To: <542331CF.9040509@cs.ntua.gr> References: <5422F6CB.6080403@ninenines.eu> <542331CF.9040509@cs.ntua.gr> Message-ID: <305E93B8-4372-4FC1-AA64-57285F378FDB@gmail.com> I prefer the second option 1, because an optional_callback attribute with special syntax means that we need yet again to increase the surface of the hack between erl_scan and erl_parse: https://github.com/erlang/otp/blob/maint/lib/stdlib/src/erl_parse.yrl#L575-L588 Le 24 sept. 2014 ? 23:04, Kostis Sagonas a ?crit : > %%----- option 1 ------- > > -callback foo(...) -> ... > -callback bar(...) -> ... > -callback baz(...) -> ... > -callback boo(...) -> ... > > -optional_callback([baz/3, boo/17]). From hans.bolinder@REDACTED Thu Sep 25 14:01:43 2014 From: hans.bolinder@REDACTED (Hans Bolinder) Date: Thu, 25 Sep 2014 12:01:43 +0000 Subject: [erlang-questions] Specs for optional callbacks (try 2) In-Reply-To: <5422F6CB.6080403@ninenines.eu> References: <5422F6CB.6080403@ninenines.eu> Message-ID: <56466BD70414EA48969B4064696CF28C21DF8EE5@ESESSMB204.ericsson.se> Hi, [Lo?c Hoguin:] > I didn't get any reply on my previous question almost 2 years ago here: > http://erlang.org/pipermail/erlang-questions/2013-January/071763.html I missed your original question, otherwise I would have replied to it. There is an implementation in Erlang/OTP 18, see commit 683a293, using the second syntax suggested by Kostis, -optional_callbacks/1. Best regards, Hans Bolinder, Erlang OTP/team, Ericsson From essen@REDACTED Thu Sep 25 14:38:43 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Thu, 25 Sep 2014 15:38:43 +0300 Subject: [erlang-questions] Specs for optional callbacks (try 2) In-Reply-To: <56466BD70414EA48969B4064696CF28C21DF8EE5@ESESSMB204.ericsson.se> References: <5422F6CB.6080403@ninenines.eu> <56466BD70414EA48969B4064696CF28C21DF8EE5@ESESSMB204.ericsson.se> Message-ID: <54240CD3.1080107@ninenines.eu> On 09/25/2014 03:01 PM, Hans Bolinder wrote: > [Lo?c Hoguin:] >> I didn't get any reply on my previous question almost 2 years ago here: >> http://erlang.org/pipermail/erlang-questions/2013-January/071763.html > > I missed your original question, otherwise I would have replied to it. No worries. :) > There is an implementation in Erlang/OTP 18, see commit 683a293, using > the second syntax suggested by Kostis, -optional_callbacks/1. Woot! This looks great. I guess I'll just have to wait now and play with it a few weeks before 18 gets released. Thanks for the heads up! -- Lo?c Hoguin http://ninenines.eu From james.hague@REDACTED Thu Sep 25 15:23:41 2014 From: james.hague@REDACTED (James Hague) Date: Thu, 25 Sep 2014 08:23:41 -0500 Subject: [erlang-questions] Starting EShell via code In-Reply-To: References: <20140920205428.GK4681@ferdmbp.local> Message-ID: Thanks everyone! I know this is trickier than it seems, because the shell is more than just a shell. (Fred, I read your excellent write-up at http://ferd.ca/repl-a-bit-more-and-less-than-that.html earlier this year.) Now I've got some new things to try. James -------------- next part -------------- An HTML attachment was scrubbed... URL: From madabusi.govindarajan@REDACTED Thu Sep 25 16:35:11 2014 From: madabusi.govindarajan@REDACTED (Madabusi Govindarajan) Date: Thu, 25 Sep 2014 20:05:11 +0530 Subject: [erlang-questions] [ANN] 'Intro to Erlang' @ Bangalore, 15 November 2014 Message-ID: Hello all, i'm teaching 'Intro to Erlang' @ Bangalore, 15 November 2014 This is an affordable 1-day event for any technical individual in Bangalore who may be interested in learning a bit of Erlang. Requires some hands-on coding. If the response is good, then we plan a follow-up event - bring in a renowned expert for advanced topics. Full details at: http://www.keyfunda.com/events/ [If you are an experienced Erlang programmer then this event is not for you. However if you are in Bangalore and would like to show up to check out the scene, just send me a note & i'll get you a free ticket.] The content is quite challenging for 1 day, but we'll try. Thank you, Madabusi Govindarajan CONTENT 0. Installing Erlang (offline) 1. Executive summary of Erlang and applications 2. Basics Shell and expressions Modules, forms, funs Tuples and Lists, Recursion Code: Simple prime sieve 3. Processes and messages Spawning processes Code: Parallel prime sieve Code: Comparison with node.js and vert.x 4. Distributed Erlang 5. Data storage: ETS and Mnesia 6. A simpler server in Erlang 7. A roadmap for learning OTP (see note below) 8. A sip of Elixir 9. Homework: Modeling an epidemic with Erlang -------------- next part -------------- An HTML attachment was scrubbed... URL: From schneider@REDACTED Thu Sep 25 17:48:23 2014 From: schneider@REDACTED (Frans Schneider) Date: Thu, 25 Sep 2014 17:48:23 +0200 Subject: [erlang-questions] macro definition map for both assignment and matching Message-ID: <54243947.2010601@xs4all.nl> Hi list, I am using maps to represent attributes of an entity and use macro's to simplify the code. As an example: -define(MEDIA_URL_ATTRS(Media_type, Url, Name, Size), #{'type' => Media_type, 'url' => Url, 'file' => Name, 'size' => Size}). This is great for creating the maps, but now I also want to use the macro's for pattern matching which requires the => to become :=. as in handle_call(?MEDIA_URL_ATTRS(Media_type, Url, Name, Size), ....) -> ... Of course, I could just copy / paste / global replace / rename the macro's to a another set of macro's, but that seems rather silly. There are approximately 200 macro's, many of them nested, which makes maintaining the code an issue. Any suggestions on making this work? Frans PS I started out using lists for the attributes, but matching only works if the order of the attributes can be guaranteed, which is not the case. Also, partial matching works rather nice with maps. From mononcqc@REDACTED Thu Sep 25 18:42:43 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 25 Sep 2014 12:42:43 -0400 Subject: [erlang-questions] macro definition map for both assignment and matching In-Reply-To: <54243947.2010601@xs4all.nl> References: <54243947.2010601@xs4all.nl> Message-ID: <20140925164242.GC19325@ferdair.local> On 09/25, Frans Schneider wrote: > Hi list, > > I am using maps to represent attributes of an entity and use macro's to > simplify the code. > As an example: > > [...] > > This is great for creating the maps, but now I also want to use the macro's > for pattern matching which requires the => to become :=. as in > > handle_call(?MEDIA_URL_ATTRS(Media_type, Url, Name, Size), ....) -> > ... > > Of course, I could just copy / paste / global replace / rename the macro's > to a another set of macro's, but that seems rather silly. There are > approximately 200 macro's, many of them nested, which makes maintaining the > code an issue. > > Any suggestions on making this work? > > Frans > > PS > I started out using lists for the attributes, but matching only works if the > order of the attributes can be guaranteed, which is not the case. Also, > partial matching works rather nice with maps. Here's a pretty cool set of options if what you need is something of the form LABEL(Pos1, Pos2, Pos3, Pos4). 1. Use tuples: {media_url_attrs, MediaType, URL, Name, Size} Nice aspects: - forces you to define them properly in messages, rather than any kind of random map that someone can pass your server. Explicit protocols are the best protocols - Lowest memory overhead possible for this while remaining readable - Can be used both for pattern matching and assignment, although it requires an explicit set of all values (setelement is otherwise possible to use!) Downsides: - Not practical for frequent updates to only a subset of elements - Cannot pattern match on a subset of the elements 2. Use a list in a tagged tuple: {media_url_attrs, [MediaType, URL, Name, Size]} Nice aspects: - Allows matching on variable element numbers Downsides: - Lists tend to have no 'ordered' meaning semantically, proplists help with that but you didn't like them 3. Records Nice aspects: - Meant for static number of elements - As efficient as tuples, but allow partial matching and assignment Downsides: - they require shared definitions across modules (don't do that) - Cumbersome syntax - No access to variable keys (that's on purpose) 4. Maps Nice aspects: - Made for variable number of elements (like dict, etc.) - Can be used more like lists, without the proplists interface to be used Downsides: - No variable variable names (yet) 5. Functions: media_url_attributes(Key, Term) -> Val Nice aspects: - Internal representation can remain hidden Downsides: - Doesn't allow pattern matching 6. Macros: Nice aspects: - Same advantages as function Downsides: - Same downsides as tuples - Also why would you do that when everything else mentioned here may work? In conclusion: - You're handling messages. Please define your protocols explicitly and stick that stuff in a tuple - You're moving things inside your state. Please consider using a record for your state, it's what they were meant for - The format of a message being passed and a piece of state you keep around do not need to be the same, and picking the right data structures will likely help you get nicer programs rather than shoving it all in a map. - I am unfortunately not surprised you're having maintenance issues with ~200 such macros (you're getting all the advantages of function calls with all the downsides of records). Why do you even need them in the first place? If all you need are default values, please consider using records, or using a default map that you merge with a user-submitted map. Regards, Fred. From lloyd@REDACTED Thu Sep 25 19:00:13 2014 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Thu, 25 Sep 2014 13:00:13 -0400 Subject: [erlang-questions] Chicago Erlang Conference --- a dabbler's take Message-ID: <2FE7D7C5-E895-4486-9F71-F41DD7ECE76F@writersglen.com> This post may be a stretch for this list but I humbly post it on grounds that it deals with Erlang questions in the large rather than nitty gritty "how do I..." First off, I can no longer refer to myself as a newbie since I've been hanging around too long. But I can't pass off as a wizard since I'm not. I do strive to plunge deeply into Erlang, but dabbler sounds about right. So, I spent a brain-frying day in Chicago last Monday breathing the fumes of exciting ideas and shaking hands with an impressive posse of Erlang leading lights. You missed much if you missed out. Since, no doubt, the videos of the presentations will soon pop up on the web, I'll just say that, for the most part, they dove deeply into the arcana of keeping complex Erlang systems and clusters running in top form. I learned much--- mostly how much I have yet to learn. Random personal highlights: Reid Draper --- Building Fault Tolerant Teams at Basho: A jaw-droppingly impressive overview of how to test and turn out gold-standard software. Irons Guberman -- Maximizing Throughput on Multicore; How to get more bang for the buck from Erlang. Super slides. Garrett Smith --- Writing Quality Code in Erlang; As an old Forth guy I felt right at home. Factor, factor, factor until you distill out the essence of your solution in all its crystal clarity. Garrett was definitely channeling Chuck Moore. Brian Troutwine --- Monitoring Complex Systems; I hope I'm never in Brian's shoes. But if I ever have a complex system to wrangle, Brian would be the kind of guy I'd want by my side. Jesse Gumm --- Building Web Scale Apps with Nitrogen; Jesse exploded on the stage and wrote a non-trivial web app before our eyes in less than 40 minutes. Yeow! Other speakers were equally engaging, but not as close to my personal needs and interests. The big questions rumbled to the surface during the after-conference party. - Several Erlang newbies took on Joe Armstrong: "Why are libraries so inconsistent, documentation so sparse, and Erlang so hard to learn?" Joe held his own, but no doubt these questions will persist. - A group of Elixir groupies made a strong case for their new puppy. Thinking about learning one more language makes my head hurt. But the converging?/diverging? paths of old and new will be fun to watch. - And, finally, during waning minutes of the party, Garrett Smith told me that Erlang should be completely rewritten from the ground up. He's tried with e2, but there's no interest. Maybe Garrett was just worn out following weeks on a job well done. Best to all, LRP From flw@REDACTED Thu Sep 25 19:11:01 2014 From: flw@REDACTED (Florian Waas) Date: Thu, 25 Sep 2014 17:11:01 +0000 Subject: [erlang-questions] Erlang for enterprise applications -- best practices Message-ID: Based on mailing list traffic and conversations with Erlangers over the years, I?ve come to believe that most systems written in Erlang are either hosted services (e.g. web servers, messaging) or embedded systems shipping as an appliance, often on custom hardware configurations (e.g. networking, switching). It seems, few systems, if any(?), are ?classic? enterprise software that is installed, operated, and troubleshot by IT staff that have absolutely no Erlang background. Gearing up to ship my own Erlang-based enterprise app, I?m curious to hear from others about their experience in this space: do-s/don?t-s, best practices, experience, war stories, gotchas, customer expectations etc. Happy to compile this into a best practices doc if there?s a significant number of replies. Thanks, -fl. From g@REDACTED Thu Sep 25 20:37:19 2014 From: g@REDACTED (Garrett Smith) Date: Thu, 25 Sep 2014 13:37:19 -0500 Subject: [erlang-questions] Chicago Erlang Conference --- a dabbler's take In-Reply-To: <2FE7D7C5-E895-4486-9F71-F41DD7ECE76F@writersglen.com> References: <2FE7D7C5-E895-4486-9F71-F41DD7ECE76F@writersglen.com> Message-ID: On Thu, Sep 25, 2014 at 12:00 PM, Lloyd R. Prentice wrote: > This post may be a stretch for this list but I humbly post it on grounds that it deals with Erlang questions in the large rather than nitty gritty "how do I..." > > First off, I can no longer refer to myself as a newbie since I've been hanging around too long. But I can't pass off as a wizard since I'm not. I do strive to plunge deeply into Erlang, but dabbler sounds about right. > > So, I spent a brain-frying day in Chicago last Monday breathing the fumes of exciting ideas and shaking hands with an impressive posse of Erlang leading lights. You missed much if you missed out. Very kind words - thank you! > Since, no doubt, the videos of the presentations will soon pop up on the web, I'll just say that, for the most part, they dove deeply into the arcana of keeping complex Erlang systems and clusters running in top form. I learned much--- mostly how much I have yet to learn. > > Irons Guberman -- Maximizing Throughput on Multicore; How to get more bang for the buck from Erlang. Super slides. Irons Guberman is the toughest, most ruthless Erlang warrior to step onto the stage. She delivered a high impact, Iron-Fisted talk that was brilliant - even more so given it was her first conference presentation! Irina Guberman, Irons' soft spoken alter ego, was equally excellent, engaging, warm and thoroughly entertaining. > Garrett Smith --- Writing Quality Code in Erlang; As an old Forth guy I felt right at home. Factor, factor, factor until you distill out the essence of your solution in all its crystal clarity. Garrett was definitely channeling Chuck Moore. > > Brian Troutwine --- Monitoring Complex Systems; I hope I'm never in Brian's shoes. But if I ever have a complex system to wrangle, Brian would be the kind of guy I'd want by my side. > > Jesse Gumm --- Building Web Scale Apps with Nitrogen; Jesse exploded on the stage and wrote a non-trivial web app before our eyes in less than 40 minutes. Yeow! > > Other speakers were equally engaging, but not as close to my personal needs and interests. > > The big questions rumbled to the surface during the after-conference party. > > - Several Erlang newbies took on Joe Armstrong: "Why are libraries so inconsistent, documentation so sparse, and Erlang so hard to learn?" Joe held his own, but no doubt these questions will persist. > > - A group of Elixir groupies made a strong case for their new puppy. Thinking about learning one more language makes my head hurt. But the converging?/diverging? paths of old and new will be fun to watch. > > - And, finally, during waning minutes of the party, Garrett Smith told me that Erlang should be completely rewritten from the ground up. He's tried with e2, but there's no interest. You forgot to mention that Garrett was stammering drunk and shouting for Vinoski, saying he wanted to "teach that dirty scheduler writing scoundrel a lesson he'd not soon forget!" But to clarify, I did not say that Erlang should be rewritten, but that some other new distribution, built atop the Erlang VM and toolchain (similar to what Elixir has done), could be created that cleaned up the core libraries and was free from legacy restrictions. I personally like Erlang as it is -- I don't want a new language (like Elixir). But I recognize that the legacy of Erlang prevents it from moving aggressively into a more newbie friendly and accessible form. That's a matter of physics, not politics, religion, or personal taste. Erlang will continue to improve incrementally, but it will never enjoy the freedom to build-from-scratch that Elixir has. But something new *could*. The problem, obviously I think, is the breathtaking cost of "rewriting Erlang". This is a vision that is not likely possible. But one can drink and ponder. I have proposed a new language called Lamprey: https://github.com/gar1t/lamprey This doesn't exist. It's mostly a joke and I don't even know if it's possible. But if it came to be, it would be epic in the history of languages. The idea is to take the stunning and amazing work of the Elixir community and bring it back to Erlang syntax as a new language -- Lamprey. Lamprey would enjoy clean libraries, documentation, tools, etc. without doing any work. When a new version of Elixir is released, Lamprey would just run its bash scripts to convert everything back to Erlang, and whamo - a new release of Lamprey! Parasitism at the computer language level. Robert Virding btw gets credit for the name :) Thanks Lloyd for attending on Monday. It was a pleasure to finally meet you! Garrett From samjbarney@REDACTED Thu Sep 25 21:17:44 2014 From: samjbarney@REDACTED (Samuel Barney) Date: Thu, 25 Sep 2014 13:17:44 -0600 Subject: [erlang-questions] Question about the VM In-Reply-To: References: Message-ID: 1. A native loop is a loop that is not emulated via function calls. 2. I am planning on generating loops using plain jump instructions. 3. It shouldn't hang the VM any since, if I understand correctly, the thread can yield after any instruction is called. 4. What do you mean by, 'Did you measure that you actuall need such a thing?' 5. I have checked that the erlc compiler can handle it, and still benefit. 6. If I wanted to use HiPE, I would use it. Where's the fun in having HiPE do all my dirty work? I might integrate it into my language later on, but for right now I want pure BEAM. On Thu, Sep 25, 2014 at 3:07 AM, Anthony Ramine wrote: > What do you call native loops? Are you planning to generate BEAM code that > loops using plain jump instructions? What do you do in that loop? Can you > be sure your process will still yield in a cooperative fashion to not hang > the VM? Did you measure that you actually need such a thing? Did you check > that you will still be able to feed that assembly to the erlc compiler to > benefit from the other usual instructions? In the end couldn't you just use > HiPE to take care of these loops? > > Regards, > > -- > Anthony Ramine > > Le 24 sept. 2014 ? 23:36, Samuel Barney a ?crit : > > I did look at targeting Core Erlang first, but it doesn't allow for native > loops. I could emulate them with function calls, however, if I move down to > the assembly level I can build loops with what is provided there. > > Thank you for the information, I'll do some more searching to see what I > can find. > > Respectfully, > Samuel Barney > > On Wed, Sep 24, 2014 at 4:40 AM, Anthony Ramine wrote: > >> Did you try targeting Core Erlang first? Which primitives do you want to >> access in BEAM? >> >> There are 1024 X registers, and Y registers are used for locals in >> functions, to save them when calling other functions. >> >> You can find such information by reading the source code or various talk >> slides made by the OTP team these last years, but I can't help you find the >> links right now. >> >> Regards, >> >> Le 24 sept. 2014 ? 04:40, Samuel Barney a ?crit : >> >> > I'm developing a language on top of the Erlang VM, and there are a >> couple of flow-control constructs that I can't implement without skipping >> the Erlang AST and using the erlang assembly code. >> > >> > Is there a specific amount of X and Y registers? Or is there a way that >> I can find that out via Erlang? >> > >> > Also, what is the purpose of the Y registers? >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Thu Sep 25 21:40:38 2014 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Thu, 25 Sep 2014 21:40:38 +0200 Subject: [erlang-questions] Question about the VM In-Reply-To: References: Message-ID: 2014-09-25 21:17 GMT+02:00 Samuel Barney : > > 3. It shouldn't hang the VM any since, if I understand correctly, the > thread can yield after any instruction is called. > no Try this: file t.S {module, t}. %% version = 0 {exports, [{go,0},{module_info,0},{module_info,1}]}. {attributes, []}. {labels, 3}. {function, go, 0, 2}. {label,1}. {line,[{location,"t.erl",14}]}. {func_info,{atom,t},{atom,go},0}. {label,2}. {jump,{f,2}}. Compile with erlc +no_postopt t.S And spawn some processes in the shell like this: spawn(fun() -> t:go() end). Let me know how far you get. It's not necessarily a bad idea to target beam-assembler but the assembler is highly tuned for the Erlang language .. or rather the Core language. -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Thu Sep 25 21:52:10 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 25 Sep 2014 21:52:10 +0200 Subject: [erlang-questions] Question about the VM In-Reply-To: References: Message-ID: <515095CA-7070-4B80-9187-B12973AB3D2C@gmail.com> Last time I tried manually changing a map/2 function to a "native loop" with simple jumps, I couldn't compile using BEAM optimisation passes because liveness analysis currently cannot cope with a backwards jump (e.g. {label,2}. ... {jump,{f,2}}): $ bin/erlc foo.S Function: map/3 foo.S: internal error in beam_block; crash reason: {case_clause, {'EXIT', {function_clause, [{gb_trees,get_1,[10,nil],[{file,"gb_trees.erl"},{line,239}]}, {beam_utils,live_opt,4,[{file,"beam_utils.erl"},{line,687}]}, {beam_utils,live_opt,1,[{file,"beam_utils.erl"},{line,207}]}, {beam_block,function,2,[{file,"beam_block.erl"},{line,38}]}, {lists,mapfoldl,3,[{file,"lists.erl"},{line,1352}]}, {lists,mapfoldl,3,[{file,"lists.erl"},{line,1353}]}, {beam_block,module,2,[{file,"beam_block.erl"},{line,29}]}, {compile,'-select_passes/2-anonymous-2-',2, [{file,"compile.erl"},{line,495}]}]}}} in function compile:'-select_passes/2-anonymous-2-'/2 (compile.erl, line 495) in call from compile:'-internal_comp/4-anonymous-1-'/2 (compile.erl, line 292) in call from compile:fold_comp/3 (compile.erl, line 310) in call from compile:internal_comp/4 (compile.erl, line 294) in call from compile:'-do_compile/2-anonymous-0-'/2 (compile.erl, line 153) Without the BEAM optimisations (+no_postopt), compilation succeeds. foo.S, for reference: https://gist.githubusercontent.com/nox/264ca631ef2d8b9d8ce1/raw/1421204b0f360af0f3abc372675d0b85dd455d1b/foo.S For the other points, see what Bj?rn-Egil said. AFAIK only calls and gc-inducing instructions can yield. Le 25 sept. 2014 ? 21:17, Samuel Barney a ?crit : > 1. A native loop is a loop that is not emulated via function calls. > 2. I am planning on generating loops using plain jump instructions. > 3. It shouldn't hang the VM any since, if I understand correctly, the thread can yield after any instruction is called. > 4. What do you mean by, 'Did you measure that you actuall need such a thing?' > 5. I have checked that the erlc compiler can handle it, and still benefit. > > 6. If I wanted to use HiPE, I would use it. Where's the fun in having HiPE do all my dirty work? I might integrate it into my language later on, but for right now I want pure BEAM. > > On Thu, Sep 25, 2014 at 3:07 AM, Anthony Ramine wrote: > What do you call native loops? Are you planning to generate BEAM code that loops using plain jump instructions? What do you do in that loop? Can you be sure your process will still yield in a cooperative fashion to not hang the VM? Did you measure that you actually need such a thing? Did you check that you will still be able to feed that assembly to the erlc compiler to benefit from the other usual instructions? In the end couldn't you just use HiPE to take care of these loops? > > Regards, > > -- > Anthony Ramine > > Le 24 sept. 2014 ? 23:36, Samuel Barney a ?crit : > >> I did look at targeting Core Erlang first, but it doesn't allow for native loops. I could emulate them with function calls, however, if I move down to the assembly level I can build loops with what is provided there. >> >> Thank you for the information, I'll do some more searching to see what I can find. >> >> Respectfully, >> Samuel Barney >> >> On Wed, Sep 24, 2014 at 4:40 AM, Anthony Ramine wrote: >> Did you try targeting Core Erlang first? Which primitives do you want to access in BEAM? >> >> There are 1024 X registers, and Y registers are used for locals in functions, to save them when calling other functions. >> >> You can find such information by reading the source code or various talk slides made by the OTP team these last years, but I can't help you find the links right now. >> >> Regards, >> >> Le 24 sept. 2014 ? 04:40, Samuel Barney a ?crit : >> >> > I'm developing a language on top of the Erlang VM, and there are a couple of flow-control constructs that I can't implement without skipping the Erlang AST and using the erlang assembly code. >> > >> > Is there a specific amount of X and Y registers? Or is there a way that I can find that out via Erlang? >> > >> > Also, what is the purpose of the Y registers? >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> > From samjbarney@REDACTED Thu Sep 25 22:25:28 2014 From: samjbarney@REDACTED (Samuel Barney) Date: Thu, 25 Sep 2014 14:25:28 -0600 Subject: [erlang-questions] Question about the VM In-Reply-To: <515095CA-7070-4B80-9187-B12973AB3D2C@gmail.com> References: <515095CA-7070-4B80-9187-B12973AB3D2C@gmail.com> Message-ID: Hmmmm..... Thank you for pointing that out. I guess my tests should have included non-function calls. ;P Well, I suppose I could get around it by ending each iteration of the loop with a call to erlang:yield(), at least if the loop doesn't contain any calls. Does anyone know if anything other than calls and gc instructions can yield? On Thu, Sep 25, 2014 at 1:52 PM, Anthony Ramine wrote: > Last time I tried manually changing a map/2 function to a "native loop" > with simple jumps, I couldn't compile using BEAM optimisation passes > because liveness analysis currently cannot cope with a backwards jump (e.g. > {label,2}. ... {jump,{f,2}}): > > $ bin/erlc foo.S > Function: map/3 > foo.S: internal error in beam_block; > crash reason: {case_clause, > {'EXIT', > {function_clause, > [{gb_trees,get_1,[10,nil],[{file,"gb_trees.erl"},{line,239}]}, > {beam_utils,live_opt,4,[{file,"beam_utils.erl"},{line,687}]}, > {beam_utils,live_opt,1,[{file,"beam_utils.erl"},{line,207}]}, > {beam_block,function,2,[{file,"beam_block.erl"},{line,38}]}, > {lists,mapfoldl,3,[{file,"lists.erl"},{line,1352}]}, > {lists,mapfoldl,3,[{file,"lists.erl"},{line,1353}]}, > {beam_block,module,2,[{file,"beam_block.erl"},{line,29}]}, > {compile,'-select_passes/2-anonymous-2-',2, > [{file,"compile.erl"},{line,495}]}]}}} > > in function compile:'-select_passes/2-anonymous-2-'/2 (compile.erl, > line 495) > in call from compile:'-internal_comp/4-anonymous-1-'/2 (compile.erl, > line 292) > in call from compile:fold_comp/3 (compile.erl, line 310) > in call from compile:internal_comp/4 (compile.erl, line 294) > in call from compile:'-do_compile/2-anonymous-0-'/2 (compile.erl, line > 153) > > Without the BEAM optimisations (+no_postopt), compilation succeeds. > > foo.S, for reference: > > > https://gist.githubusercontent.com/nox/264ca631ef2d8b9d8ce1/raw/1421204b0f360af0f3abc372675d0b85dd455d1b/foo.S > > For the other points, see what Bj?rn-Egil said. AFAIK only calls and > gc-inducing instructions can yield. > > Le 25 sept. 2014 ? 21:17, Samuel Barney a ?crit : > > > 1. A native loop is a loop that is not emulated via function calls. > > 2. I am planning on generating loops using plain jump instructions. > > 3. It shouldn't hang the VM any since, if I understand correctly, the > thread can yield after any instruction is called. > > 4. What do you mean by, 'Did you measure that you actuall need such a > thing?' > > 5. I have checked that the erlc compiler can handle it, and still > benefit. > > > > 6. If I wanted to use HiPE, I would use it. Where's the fun in having > HiPE do all my dirty work? I might integrate it into my language later on, > but for right now I want pure BEAM. > > > > On Thu, Sep 25, 2014 at 3:07 AM, Anthony Ramine > wrote: > > What do you call native loops? Are you planning to generate BEAM code > that loops using plain jump instructions? What do you do in that loop? Can > you be sure your process will still yield in a cooperative fashion to not > hang the VM? Did you measure that you actually need such a thing? Did you > check that you will still be able to feed that assembly to the erlc > compiler to benefit from the other usual instructions? In the end couldn't > you just use HiPE to take care of these loops? > > > > Regards, > > > > -- > > Anthony Ramine > > > > Le 24 sept. 2014 ? 23:36, Samuel Barney a ?crit : > > > >> I did look at targeting Core Erlang first, but it doesn't allow for > native loops. I could emulate them with function calls, however, if I move > down to the assembly level I can build loops with what is provided there. > >> > >> Thank you for the information, I'll do some more searching to see what > I can find. > >> > >> Respectfully, > >> Samuel Barney > >> > >> On Wed, Sep 24, 2014 at 4:40 AM, Anthony Ramine > wrote: > >> Did you try targeting Core Erlang first? Which primitives do you want > to access in BEAM? > >> > >> There are 1024 X registers, and Y registers are used for locals in > functions, to save them when calling other functions. > >> > >> You can find such information by reading the source code or various > talk slides made by the OTP team these last years, but I can't help you > find the links right now. > >> > >> Regards, > >> > >> Le 24 sept. 2014 ? 04:40, Samuel Barney a ?crit > : > >> > >> > I'm developing a language on top of the Erlang VM, and there are a > couple of flow-control constructs that I can't implement without skipping > the Erlang AST and using the erlang assembly code. > >> > > >> > Is there a specific amount of X and Y registers? Or is there a way > that I can find that out via Erlang? > >> > > >> > Also, what is the purpose of the Y registers? > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Thu Sep 25 22:51:52 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 25 Sep 2014 22:51:52 +0200 Subject: [erlang-questions] Question about the VM In-Reply-To: References: <515095CA-7070-4B80-9187-B12973AB3D2C@gmail.com> Message-ID: How would that be better than just looping with function calls? Le 25 sept. 2014 ? 22:25, Samuel Barney a ?crit : > Well, I suppose I could get around it by ending each iteration of the loop with a call to erlang:yield(), at least if the loop doesn't contain any calls. From daniel.goertzen@REDACTED Thu Sep 25 21:34:30 2014 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Thu, 25 Sep 2014 14:34:30 -0500 Subject: [erlang-questions] Chicago Erlang Conference --- a dabbler's take In-Reply-To: References: <2FE7D7C5-E895-4486-9F71-F41DD7ECE76F@writersglen.com> Message-ID: On Thu, Sep 25, 2014 at 1:37 PM, Garrett Smith wrote: > On Thu, Sep 25, 2014 at 12:00 PM, Lloyd R. Prentice > wrote: > > > > - And, finally, during waning minutes of the party, Garrett Smith told > me that Erlang should be completely rewritten from the ground up. He's > tried with e2, but there's no interest. > > You forgot to mention that Garrett was stammering drunk and shouting > for Vinoski, saying he wanted to "teach that dirty scheduler writing > scoundrel a lesson he'd not soon forget!" > > But to clarify, I did not say that Erlang should be rewritten, but > that some other new distribution, built atop the Erlang VM and > toolchain (similar to what Elixir has done), could be created that > cleaned up the core libraries and was free from legacy restrictions. > > I personally like Erlang as it is -- I don't want a new language (like > Elixir). But I recognize that the legacy of Erlang prevents it from > moving aggressively into a more newbie friendly and accessible form. > That's a matter of physics, not politics, religion, or personal taste. > Erlang will continue to improve incrementally, but it will never enjoy > the freedom to build-from-scratch that Elixir has. But something new > *could*. > > The problem, obviously I think, is the breathtaking cost of "rewriting > Erlang". This is a vision that is not likely possible. But one can > drink and ponder. > > I have proposed a new language called Lamprey: > > https://github.com/gar1t/lamprey > > This doesn't exist. It's mostly a joke and I don't even know if it's > possible. But if it came to be, it would be epic in the history of > languages. > > The idea is to take the stunning and amazing work of the Elixir > community and bring it back to Erlang syntax as a new language -- > Lamprey. Lamprey would enjoy clean libraries, documentation, tools, > etc. without doing any work. When a new version of Elixir is released, > Lamprey would just run its bash scripts to convert everything back to > Erlang, and whamo - a new release of Lamprey! > > Parasitism at the computer language level. > > Robert Virding btw gets credit for the name :) > > Thanks Lloyd for attending on Monday. It was a pleasure to finally meet > you! > > Garrett > Funny you should mention all that; I've been thinking about something similar lately. Port Elixir back to Erlang syntax so that it can compile .erl files but bring along interesting Elixir features (interesting to me is metaprogramming, pipe operator, module system.) I named it "Still". It's "still" just erlang with alchemical homage to elixir. I was also at the conference; thank you for all the hard work that went into this fantastic event! -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Thu Sep 25 22:34:25 2014 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Thu, 25 Sep 2014 16:34:25 -0400 Subject: [erlang-questions] Chicago Erlang Conference --- a dabbler's take In-Reply-To: References: <2FE7D7C5-E895-4486-9F71-F41DD7ECE76F@writersglen.com> Message-ID: Hi Garrett, Well, mildly tipsy maybe, but you were 100% articulate and expressing interesting ideas. Incidentally, I did have the pleasure of meeting Steve. Since we're both in the Boston area, sharing war stories and lies over lunch may be a prospect. As I attempted to express in my response to Alex, the creative tensions stimulated by new ideas like Elixir, e2, and the cryptozoological Lamprey keep progress advancing apace and life exiting. Centripetal and centrifugal influences challenge every community. The question is how how we harness them to better achieve our goals and foster an ever more vibrant community. All the best, LRP P.S. Big apology "Irons" for the typo. Your presentation was indeed terrific. Sent from my iPadSent from my iPad > On Sep 25, 2014, at 2:37 PM, Garrett Smith wrote: > > On Thu, Sep 25, 2014 at 12:00 PM, Lloyd R. Prentice > wrote: >> This post may be a stretch for this list but I humbly post it on grounds that it deals with Erlang questions in the large rather than nitty gritty "how do I..." >> >> First off, I can no longer refer to myself as a newbie since I've been hanging around too long. But I can't pass off as a wizard since I'm not. I do strive to plunge deeply into Erlang, but dabbler sounds about right. >> >> So, I spent a brain-frying day in Chicago last Monday breathing the fumes of exciting ideas and shaking hands with an impressive posse of Erlang leading lights. You missed much if you missed out. > > Very kind words - thank you! > >> Since, no doubt, the videos of the presentations will soon pop up on the web, I'll just say that, for the most part, they dove deeply into the arcana of keeping complex Erlang systems and clusters running in top form. I learned much--- mostly how much I have yet to learn. >> >> Irons Guberman -- Maximizing Throughput on Multicore; How to get more bang for the buck from Erlang. Super slides. > > Irons Guberman is the toughest, most ruthless Erlang warrior to step > onto the stage. She delivered a high impact, Iron-Fisted talk that was > brilliant - even more so given it was her first conference > presentation! > > Irina Guberman, Irons' soft spoken alter ego, was equally excellent, > engaging, warm and thoroughly entertaining. > >> Garrett Smith --- Writing Quality Code in Erlang; As an old Forth guy I felt right at home. Factor, factor, factor until you distill out the essence of your solution in all its crystal clarity. Garrett was definitely channeling Chuck Moore. >> >> Brian Troutwine --- Monitoring Complex Systems; I hope I'm never in Brian's shoes. But if I ever have a complex system to wrangle, Brian would be the kind of guy I'd want by my side. >> >> Jesse Gumm --- Building Web Scale Apps with Nitrogen; Jesse exploded on the stage and wrote a non-trivial web app before our eyes in less than 40 minutes. Yeow! >> >> Other speakers were equally engaging, but not as close to my personal needs and interests. >> >> The big questions rumbled to the surface during the after-conference party. >> >> - Several Erlang newbies took on Joe Armstrong: "Why are libraries so inconsistent, documentation so sparse, and Erlang so hard to learn?" Joe held his own, but no doubt these questions will persist. >> >> - A group of Elixir groupies made a strong case for their new puppy. Thinking about learning one more language makes my head hurt. But the converging?/diverging? paths of old and new will be fun to watch. >> >> - And, finally, during waning minutes of the party, Garrett Smith told me that Erlang should be completely rewritten from the ground up. He's tried with e2, but there's no interest. > > You forgot to mention that Garrett was stammering drunk and shouting > for Vinoski, saying he wanted to "teach that dirty scheduler writing > scoundrel a lesson he'd not soon forget!" > > But to clarify, I did not say that Erlang should be rewritten, but > that some other new distribution, built atop the Erlang VM and > toolchain (similar to what Elixir has done), could be created that > cleaned up the core libraries and was free from legacy restrictions. > > I personally like Erlang as it is -- I don't want a new language (like > Elixir). But I recognize that the legacy of Erlang prevents it from > moving aggressively into a more newbie friendly and accessible form. > That's a matter of physics, not politics, religion, or personal taste. > Erlang will continue to improve incrementally, but it will never enjoy > the freedom to build-from-scratch that Elixir has. But something new > *could*. > > The problem, obviously I think, is the breathtaking cost of "rewriting > Erlang". This is a vision that is not likely possible. But one can > drink and ponder. > > I have proposed a new language called Lamprey: > > https://github.com/gar1t/lamprey > > This doesn't exist. It's mostly a joke and I don't even know if it's > possible. But if it came to be, it would be epic in the history of > languages. > > The idea is to take the stunning and amazing work of the Elixir > community and bring it back to Erlang syntax as a new language -- > Lamprey. Lamprey would enjoy clean libraries, documentation, tools, > etc. without doing any work. When a new version of Elixir is released, > Lamprey would just run its bash scripts to convert everything back to > Erlang, and whamo - a new release of Lamprey! > > Parasitism at the computer language level. > > Robert Virding btw gets credit for the name :) > > Thanks Lloyd for attending on Monday. It was a pleasure to finally meet you! > > Garrett From joakim@REDACTED Thu Sep 25 22:44:49 2014 From: joakim@REDACTED (Joakim Hirsch) Date: Thu, 25 Sep 2014 22:44:49 +0200 Subject: [erlang-questions] wxErlang wxCommand type missing? Message-ID: <54247EC1.2070507@snookerhallen.se> Hi everyone, I've added a wxToolBar to my gui, but I can't seem to find the proper command type to connect to. I've found 2 types that are related to toolbars: command_tool_rclicked and command_tool_enter. I'm able to receive both, but there doesn't seem to be a type for left clicking. My best guess is that it should be named command_tool_clicked, but that causes an error when trying to connect. Is it missing, or is it called something else and I'm just not able to find it? Regards, Joakim -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Fri Sep 26 04:39:43 2014 From: jinni.park@REDACTED (Park, Sungjin) Date: Fri, 26 Sep 2014 11:39:43 +0900 Subject: [erlang-questions] Change suggestion in ranch/cowboy Message-ID: Hi, this post might better be specifically targeted to Loic. But anyway, collective knowledge is better, most of the time right? So to begin with, items below are taken by practical necessity. I also want to have other people's opinions before trying. 1) Reducing Transport:controlling_process/2 Currently, ranch_acceptor calls the function to give accepted socket to the ranch_conns_sup and the ranch_conns_sup calls the function again after spawning a ranch_protocol. I have a naive idea of doing it in peer-to-peer fashion between ranch_acceptor and ranch_protocol. I don't know of course if there is any deep consideration in doing things that way though, and how much improvement there would be. 2) Dynamic parameters Currently, parameters required to run ranch_protocol are stored in the ranch_conns_sup and given to the ranch_protocol when spawning. Combined with cowboy, cascaded parameters like cowboy_protocol -> cowboy_router, cowboy_handler -> dispatch env are constant once started. Changing any of the above requires ranch_conns_sup restart. Even if we could restart ranch_conns_sup without disrupting connections, the protocols running with the connections will still use old parameters. This is especially annoying if the connections have long running nature handling thousands of requests in their lifetime. At this point, I think that there could be kind of parameter repository service which provide event notification service to whatever entity that requires the parameters and let cowboy_router subscribe to it. Would the above worth trying? -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Fri Sep 26 07:12:31 2014 From: dangud@REDACTED (Dan Gudmundsson) Date: Fri, 26 Sep 2014 07:12:31 +0200 Subject: [erlang-questions] wxErlang wxCommand type missing? In-Reply-To: <54247EC1.2070507@snookerhallen.se> References: <54247EC1.2070507@snookerhallen.se> Message-ID: Hej Joakim If you take a look at http://docs.wxwidgets.org/2.8.12/wx_wxtoolbar.html EVT_TOOL(id, func) Process a wxEVT_COMMAND_TOOL_CLICKED event (a synonym for wxEVT_COMMAND_MENU_SELECTED). Pass the id of the tool. So command_menu_selected should work.. I have not find a way to generate this into the documentation, which is kind of stupid now that I I think about it. If I get some time I will try to do it. /Dan On Thu, Sep 25, 2014 at 10:44 PM, Joakim Hirsch wrote: > Hi everyone, > > I've added a wxToolBar to my gui, but I can't seem to find the proper > command type to connect to. I've found 2 types that are related to > toolbars: command_tool_rclicked and command_tool_enter. I'm able to > receive both, but there doesn't seem to be a type for left clicking. My > best guess is that it should be named command_tool_clicked, but that > causes an error when trying to connect. Is it missing, or is it called > something else and I'm just not able to find it? > > Regards, > Joakim > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From schneider@REDACTED Fri Sep 26 08:36:30 2014 From: schneider@REDACTED (Frans Schneider) Date: Fri, 26 Sep 2014 08:36:30 +0200 Subject: [erlang-questions] macro definition map for both assignment and matching In-Reply-To: <20140925164242.GC19325@ferdair.local> References: <54243947.2010601@xs4all.nl> <20140925164242.GC19325@ferdair.local> Message-ID: <5425096E.6080401@xs4all.nl> Dear Fred, Thanks for the advice. Rethinking the attributes issue, I came to the conclusion that matching is overkill. One of the problems I have to deal with is that I don't have access to the protocol specs and have to do a lot of reverse engineering and use trial and error. About solution number 3, using records, and not sharing record definitions across modules. Is that because the compiled code gets out of sync? I found records to be very nice in partial matching and use them for that reason. The syntax is ok for me and with the macros, the complexity of nested records can be hidden. By the way, I don't use the state as one might conclude from the example I gave. Should have been more precise on that point. Frans On 09/25/2014 06:42 PM, Fred Hebert wrote: > On 09/25, Frans Schneider wrote: >> Hi list, >> >> I am using maps to represent attributes of an entity and use macro's to >> simplify the code. >> As an example: >> >> [...] >> >> This is great for creating the maps, but now I also want to use the macro's >> for pattern matching which requires the => to become :=. as in >> >> handle_call(?MEDIA_URL_ATTRS(Media_type, Url, Name, Size), ....) -> >> ... >> >> Of course, I could just copy / paste / global replace / rename the macro's >> to a another set of macro's, but that seems rather silly. There are >> approximately 200 macro's, many of them nested, which makes maintaining the >> code an issue. >> >> Any suggestions on making this work? >> >> Frans >> >> PS >> I started out using lists for the attributes, but matching only works if the >> order of the attributes can be guaranteed, which is not the case. Also, >> partial matching works rather nice with maps. > Here's a pretty cool set of options if what you need is something of the > form LABEL(Pos1, Pos2, Pos3, Pos4). > > 1. Use tuples: {media_url_attrs, MediaType, URL, Name, Size} > > Nice aspects: > - forces you to define them properly in messages, rather than any kind > of random map that someone can pass your server. Explicit protocols > are the best protocols > - Lowest memory overhead possible for this while remaining readable > - Can be used both for pattern matching and assignment, although it > requires an explicit set of all values (setelement is otherwise > possible to use!) > > Downsides: > - Not practical for frequent updates to only a subset of elements > - Cannot pattern match on a subset of the elements > > 2. Use a list in a tagged tuple: {media_url_attrs, [MediaType, URL, Name, Size]} > > Nice aspects: > - Allows matching on variable element numbers > > Downsides: > - Lists tend to have no 'ordered' meaning semantically, proplists help > with that but you didn't like them > > 3. Records > > Nice aspects: > - Meant for static number of elements > - As efficient as tuples, but allow partial matching and assignment > > Downsides: > - they require shared definitions across modules (don't do that) > - Cumbersome syntax > - No access to variable keys (that's on purpose) > > 4. Maps > > Nice aspects: > - Made for variable number of elements (like dict, etc.) > - Can be used more like lists, without the proplists interface to be > used > > Downsides: > - No variable variable names (yet) > > 5. Functions: media_url_attributes(Key, Term) -> Val > > Nice aspects: > - Internal representation can remain hidden > > Downsides: > - Doesn't allow pattern matching > > 6. Macros: > > Nice aspects: > - Same advantages as function > > Downsides: > - Same downsides as tuples > - Also why would you do that when everything else mentioned here may > work? > > > In conclusion: > > - You're handling messages. Please define your protocols explicitly and > stick that stuff in a tuple > - You're moving things inside your state. Please consider using a record > for your state, it's what they were meant for > - The format of a message being passed and a piece of state you keep > around do not need to be the same, and picking the right data > structures will likely help you get nicer programs rather than shoving > it all in a map. > - I am unfortunately not surprised you're having maintenance issues with > ~200 such macros (you're getting all the advantages of function calls > with all the downsides of records). Why do you even need them in the > first place? If all you need are default values, please consider using > records, or using a default map that you merge with a user-submitted > map. > > Regards, > Fred. From essen@REDACTED Fri Sep 26 09:26:39 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Fri, 26 Sep 2014 10:26:39 +0300 Subject: [erlang-questions] Change suggestion in ranch/cowboy In-Reply-To: References: Message-ID: <5425152F.6080105@ninenines.eu> On 09/26/2014 05:39 AM, Park, Sungjin wrote: > 2) Dynamic parameters > Currently, parameters required to run ranch_protocol are stored in the > ranch_conns_sup and given to the ranch_protocol when spawning. Combined > with cowboy, cascaded parameters like cowboy_protocol -> cowboy_router, > cowboy_handler -> dispatch env are constant once started. Changing any > of the above requires ranch_conns_sup restart. Hmm no? http://ninenines.eu/docs/en/ranch/1.0/manual/ranch/index.html#set_protocol_options http://ninenines.eu/docs/en/cowboy/1.0/manual/cowboy/index.html#set_env Changing for all existing connections is up to the application though. -- Lo?c Hoguin http://ninenines.eu From joakim@REDACTED Fri Sep 26 10:22:56 2014 From: joakim@REDACTED (Joakim Hirsch) Date: Fri, 26 Sep 2014 10:22:56 +0200 Subject: [erlang-questions] wxErlang wxCommand type missing? In-Reply-To: References: <54247EC1.2070507@snookerhallen.se> Message-ID: <54252260.5010100@snookerhallen.se> Thanks Dan, that's it! /Joakim Dan Gudmundsson skrev 2014-09-26 07:12: > Hej Joakim > > If you take a look at http://docs.wxwidgets.org/2.8.12/wx_wxtoolbar.html > > EVT_TOOL(id, func)Process a wxEVT_COMMAND_TOOL_CLICKED event (a > synonym for wxEVT_COMMAND_MENU_SELECTED). Pass the id of the tool. > > So command_menu_selected should work.. > > I have not find a way to generate this into the documentation, which > is kind of stupid now that I I think about it. If I get some time I > will try to do it. > > /Dan > > > On Thu, Sep 25, 2014 at 10:44 PM, Joakim Hirsch > > wrote: > > Hi everyone, > > I've added a wxToolBar to my gui, but I can't seem to find the > proper command type to connect to. I've found 2 types that are > related to toolbars: command_tool_rclicked and command_tool_enter. > I'm able to receive both, but there doesn't seem to be a type for > left clicking. My best guess is that it should be named > command_tool_clicked, but that causes an error when trying to > connect. Is it missing, or is it called something else and I'm > just not able to find it? > > Regards, > Joakim > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan.martinez@REDACTED Fri Sep 26 12:12:05 2014 From: ivan.martinez@REDACTED (=?UTF-8?B?SXbDoW4gTWFydMOtbmV6?=) Date: Fri, 26 Sep 2014 12:12:05 +0200 Subject: [erlang-questions] Could not open pty master In-Reply-To: References: <4676551.iEBOCzXdtb@burrito> Message-ID: I have done chmod 777 to /dev, /dev/pty*, /dev/pts and /dev/pts/* and it didn't help. Must be something different from file access permits. Anybody has a clue of what run_erl does and requires?. Thank you, Ivan 2014-09-25 10:41 GMT+02:00 Iv?n Mart?nez : > Thank you. Looks like SELinux has nothing to do with the issue: > > $ setenforce 0 > setenforce: SELinux is disabled > > > 2014-09-25 3:21 GMT+02:00 zxq9 : > >> On Wednesday 24 September 2014 21:00:32 Iv?n Mart?nez wrote: >> > Hello all, >> > Does anyone know why I'm having the following issue with a CentOS 7 >> system >> > with kernel 3.10.23?: >> > >> > $ run_erl priv/ log "erl" >> > run_erl:187 [6505] Wed Sep 24 18:42:39 2014 >> > errno=1 'Operation not permitted' >> > Could not open pty master >> > >> > It works as super user. It also works in a Fedora 20 system with kernel >> > 3.16. I couldn't find any difference in user groups or /dev file permits >> > between both systems. Starting with a user with UID above or below >> UID_MIN >> > doesn't make any difference. I don't think it matters, but the only >> > difference I could find is that the CentOS has many /dev/pty* files >> already >> > created, while the Fedora doesn't have any. >> >> You might be running into SELinux permission issues. To find out try doing >> "setenforce 0" and then running it again. If that works, use a tool like >> audit2allow or audit2why to create a policy that will permit the actions >> you >> require to run your program. I haven't kept up with the Fedora/RHEL world >> since 7 came out, but Dan Walsh's blog and Red Hat's SELinux docs have >> been >> good resources on this in the past. >> >> Of course, you might have a totally different issue, but SELinux booleans >> and >> audit logs are the first thing I check on a Fedora-type distro when >> something >> doesn't work but looks like it should. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ates@REDACTED Fri Sep 26 12:25:38 2014 From: ates@REDACTED (Artem Teslenko) Date: Fri, 26 Sep 2014 13:25:38 +0300 Subject: [erlang-questions] [ANN] eavro - Apache Avro binding Message-ID: <54253F22.7050307@ipv6.dp.ua> Hello Community, I would like to announce an Erlang application 'eavro' for Apache Avro binding: https://github.com/SIfoxDevTeam/eavro Features: * Encode/decode are fully supported for all Avro types * Read/write Object Container Files * deflate - supported * snappy - not yet * Avro RPC protocol * Client * Server Feedback and patches are welcome. From nbellerophon@REDACTED Fri Sep 26 17:43:20 2014 From: nbellerophon@REDACTED (nbellerophon@REDACTED) Date: Fri, 26 Sep 2014 16:43:20 +0100 Subject: [erlang-questions] Can't start observer on MacOSX 10.9.5 in ESL Erlang 17.1 Message-ID: Sorry if this appears as a repost. Some signup confusion. Anyway... Trying to start observer off of clean install. ESL Erlang 17.1, brew install wxmac. OS X 10.9.5. Erlang/OTP 17 [erts-6.1] [source-d2a4c20] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.1 (abort with ^G) 1> observer:start(). =ERROR REPORT==== 26-Sep-2014::16:31:42 === WX Failed loading "wxe_driver"@"/usr/local/lib/erlang/lib/wx-1.3/priv" {error,{{load_driver,"dlopen(/usr/local/lib/erlang/lib/wx-1.3/priv/wxe_driver.so, 2): Symbol not found: __ZThn1152_N16wxStyledTextCtrl10AppendTextERK8wxString\n Referenced from: /usr/local/lib/erlang/lib/wx-1.3/priv/wxe_driver.so\n Expected in: flat namespace\n"}, [{wxe_server,start,1,[{file,"wxe_server.erl"},{line,64}]}, {wx,new,1,[{file,"wx.erl"},{line,114}]}, {observer_wx,init,1,[{file,"observer_wx.erl"},{line,91}]}, {wx_object,init_it,6,[{file,"wx_object.erl"},{line,299}]}, {proc_lib,init_p_do_apply,3, [{file,"proc_lib.erl"},{line,239}]}]}} Thanks, Nicholas Bellerophon nbellerophon@REDACTED From antoine.koener@REDACTED Fri Sep 26 22:39:37 2014 From: antoine.koener@REDACTED (Antoine Koener) Date: Sat, 27 Sep 2014 04:39:37 +0800 Subject: [erlang-questions] Could not open pty master In-Reply-To: References: <4676551.iEBOCzXdtb@burrito> Message-ID: <1A8AF49E-8E57-46DE-8EFB-045487D436B2@gmail.com> Hello, why do you not just run strace to see ? > On 26 Sep 2014, at 18:12, Iv?n Mart?nez wrote: > > I have done chmod 777 to /dev, /dev/pty*, /dev/pts and /dev/pts/* and it didn't help. Must be something different from file access permits. Anybody has a clue of what run_erl does and requires?. > Thank you, > Ivan > > 2014-09-25 10:41 GMT+02:00 Iv?n Mart?nez : >> Thank you. Looks like SELinux has nothing to do with the issue: >> >> $ setenforce 0 >> setenforce: SELinux is disabled >> >> >> 2014-09-25 3:21 GMT+02:00 zxq9 : >>> On Wednesday 24 September 2014 21:00:32 Iv?n Mart?nez wrote: >>> > Hello all, >>> > Does anyone know why I'm having the following issue with a CentOS 7 system >>> > with kernel 3.10.23?: >>> > >>> > $ run_erl priv/ log "erl" >>> > run_erl:187 [6505] Wed Sep 24 18:42:39 2014 >>> > errno=1 'Operation not permitted' >>> > Could not open pty master >>> > >>> > It works as super user. It also works in a Fedora 20 system with kernel >>> > 3.16. I couldn't find any difference in user groups or /dev file permits >>> > between both systems. Starting with a user with UID above or below UID_MIN >>> > doesn't make any difference. I don't think it matters, but the only >>> > difference I could find is that the CentOS has many /dev/pty* files already >>> > created, while the Fedora doesn't have any. >>> >>> You might be running into SELinux permission issues. To find out try doing >>> "setenforce 0" and then running it again. If that works, use a tool like >>> audit2allow or audit2why to create a policy that will permit the actions you >>> require to run your program. I haven't kept up with the Fedora/RHEL world >>> since 7 came out, but Dan Walsh's blog and Red Hat's SELinux docs have been >>> good resources on this in the past. >>> >>> Of course, you might have a totally different issue, but SELinux booleans and >>> audit logs are the first thing I check on a Fedora-type distro when something >>> doesn't work but looks like it should. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard.youngkin@REDACTED Sat Sep 27 00:31:12 2014 From: richard.youngkin@REDACTED (Youngkin, Rich) Date: Fri, 26 Sep 2014 16:31:12 -0600 Subject: [erlang-questions] relx tutorial, usage information? Message-ID: Hi all, I'm using relx in an application and I'm able to build a working release, including a tarball version of the release. I think I understand the basics after reading the relx readme on github, Fred Hebert's "How I Start - Erlang", Brian Troutwine's "Making trivial ... with relx", and Eric Merritt's presentation at Erlang Factory 2013, but I don't have a good understanding of the broader capabilities of relx. There are several areas I'd like to get more information such as: 1. I'm using "relx release tar", but I can't find any documentation that documents the "release" and "tar" command line options. 2. relx providers 3. Other capabilities that are useful? Are there any more advanced and/or detailed sources of information on these kinds of topics? Thanks, Rich -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Sat Sep 27 00:36:34 2014 From: t@REDACTED (Tristan Sloughter) Date: Fri, 26 Sep 2014 17:36:34 -0500 Subject: [erlang-questions] relx tutorial, usage information? In-Reply-To: References: Message-ID: <1411770994.72935.172254937.66CC1B86@webmail.messagingengine.com> Sorry, no, we are terrible about doing documentation :(. The only documentation is the wiki and README: https://github.com/erlware/relx/wiki/configuration [1]https://github.com/erlware/relx/blob/master/README.md Any specific questions you can bring to #erlware on freenode. -- Tristan Sloughter t@REDACTED On Fri, Sep 26, 2014, at 05:31 PM, Youngkin, Rich wrote: Hi all, I'm using relx in an application and I'm able to build a working release, including a tarball version of the release. I think I understand the basics after reading the relx readme on github, Fred Hebert's "How I Start - Erlang", Brian Troutwine's "Making trivial ... with relx", and Eric Merritt's presentation at Erlang Factory 2013, but I don't have a good understanding of the broader capabilities of relx. There are several areas I'd like to get more information such as: 1. I'm using "relx release tar", but I can't find any documentation that documents the "release" and "tar" command line options. 2. relx providers 3. Other capabilities that are useful? Are there any more advanced and/or detailed sources of information on these kinds of topics? Thanks, Rich _______________________________________________ erlang-questions mailing list [2]erlang-questions@REDACTED [3]http://erlang.org/mailman/listinfo/erlang-questions References Visible links 1. https://github.com/erlware/relx/blob/master/README.md 2. mailto:erlang-questions@REDACTED 3. http://erlang.org/mailman/listinfo/erlang-questions Hidden links: 5. https://github.com/erlware/relx/wiki/configuration 6. https://github.com/erlware/relx/wiki/configuration 7. https://github.com/erlware/relx/blob/master/README.md -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan.martinez@REDACTED Sat Sep 27 11:27:42 2014 From: ivan.martinez@REDACTED (=?UTF-8?B?SXbDoW4gTWFydMOtbmV6?=) Date: Sat, 27 Sep 2014 11:27:42 +0200 Subject: [erlang-questions] Could not open pty master In-Reply-To: <1A8AF49E-8E57-46DE-8EFB-045487D436B2@gmail.com> References: <4676551.iEBOCzXdtb@burrito> <1A8AF49E-8E57-46DE-8EFB-045487D436B2@gmail.com> Message-ID: Thank you Antoine. I'm a developer rather than a systems administrator, I didn't know strace existed. I have tried it but I don't know what to make from the output: $ strace run_erl priv/ log "erl" execve("/usr/local/bin/run_erl", ["run_erl", "priv/", "log", "erl"], [/* 23 vars */]) = 0 brk(0) = 0x60c000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f794a62f000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=32636, ...}) = 0 mmap(NULL, 32636, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f794a627000 close(3) = 0 open("/lib64/libutil.so.1", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\17\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=14608, ...}) = 0 mmap(NULL, 2105616, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f794a20d000 mprotect(0x7f794a20f000, 2093056, PROT_NONE) = 0 mmap(0x7f794a40e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7f794a40e000 close(3) = 0 open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\16\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=19512, ...}) = 0 mmap(NULL, 2109744, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f794a009000 mprotect(0x7f794a00c000, 2093056, PROT_NONE) = 0 mmap(0x7f794a20b000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f794a20b000 close(3) = 0 open("/lib64/libm.so.6", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260T\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=1141552, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f794a626000 mmap(NULL, 3150168, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f7949d07000 mprotect(0x7f7949e08000, 2093056, PROT_NONE) = 0 mmap(0x7f794a007000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x100000) = 0x7f794a007000 close(3) = 0 open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\34\2\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=2107600, ...}) = 0 mmap(NULL, 3932736, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f7949946000 mprotect(0x7f7949afc000, 2097152, PROT_NONE) = 0 mmap(0x7f7949cfc000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b6000) = 0x7f7949cfc000 mmap(0x7f7949d02000, 16960, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f7949d02000 close(3) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f794a625000 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f794a623000 arch_prctl(ARCH_SET_FS, 0x7f794a623740) = 0 mprotect(0x7f7949cfc000, 16384, PROT_READ) = 0 mprotect(0x7f794a007000, 4096, PROT_READ) = 0 mprotect(0x7f794a20b000, 4096, PROT_READ) = 0 mprotect(0x7f794a40e000, 4096, PROT_READ) = 0 mprotect(0x604000, 4096, PROT_READ) = 0 mprotect(0x7f794a630000, 4096, PROT_READ) = 0 munmap(0x7f794a627000, 32636) = 0 brk(0) = 0x60c000 brk(0x62f000) = 0x62f000 brk(0) = 0x62f000 openat(AT_FDCWD, "priv/", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3 getdents(3, /* 6 entries */, 32768) = 200 getdents(3, /* 0 entries */, 32768) = 0 close(3) = 0 mknod("priv/erlang.pipe.2.r", S_IFIFO|0666) = 0 open("priv/erlang.pipe.2.w", O_WRONLY|O_NONBLOCK) = -1 ENOENT (No such file or directory) mknod("priv/erlang.pipe.2.w", S_IFIFO|0666) = 0 open("/dev/ptmx", O_RDWR) = 3 statfs("/dev/pts", {f_type="DEVPTS_SUPER_MAGIC", f_bsize=4096, f_blocks=0, f_bfree=0, f_bavail=0, f_files=0, f_ffree=0, f_fsid={0, 0}, f_namelen=255, f_frsize=4096}) = 0 ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(3, TIOCGPTN, [1]) = 0 stat("/dev/pts/1", {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 1), ...}) = 0 ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(3, TIOCGPTN, [1]) = 0 stat("/dev/pts/1", {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 1), ...}) = 0 getuid() = 998 socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 4 connect(4, {sa_family=AF_LOCAL, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory) close(4) = 0 socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 4 connect(4, {sa_family=AF_LOCAL, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory) close(4) = 0 open("/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 4 fstat(4, {st_mode=S_IFREG|0644, st_size=1728, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f794a62e000 read(4, "#\n# /etc/nsswitch.conf\n#\n# An ex"..., 4096) = 1728 read(4, "", 4096) = 0 close(4) = 0 munmap(0x7f794a62e000, 4096) = 0 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 4 fstat(4, {st_mode=S_IFREG|0644, st_size=32636, ...}) = 0 mmap(NULL, 32636, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7f794a627000 close(4) = 0 open("/lib64/libnss_files.so.2", O_RDONLY|O_CLOEXEC) = 4 read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\"\0\0\0\0\0\0"..., 832) = 832 fstat(4, {st_mode=S_IFREG|0755, st_size=58288, ...}) = 0 mmap(NULL, 2144360, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7f794973a000 mprotect(0x7f7949745000, 2093056, PROT_NONE) = 0 mmap(0x7f7949944000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0xa000) = 0x7f7949944000 close(4) = 0 mprotect(0x7f7949944000, 4096, PROT_READ) = 0 munmap(0x7f794a627000, 32636) = 0 open("/etc/group", O_RDONLY|O_CLOEXEC) = 4 fstat(4, {st_mode=S_IFREG|0644, st_size=591, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f794a62e000 read(4, "root:x:0:\nbin:x:1:\ndaemon:x:2:\ns"..., 4096) = 591 close(4) = 0 munmap(0x7f794a62e000, 4096) = 0 chown("/dev/pts/1", 998, 5) = -1 EPERM (Operation not permitted) close(3) = 0 open("/dev/ptmx", O_RDWR) = 3 ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(3, TIOCGPTN, [1]) = 0 stat("/dev/pts/1", {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 1), ...}) = 0 getuid() = 998 chown("/dev/pts/1", 998, 5) = -1 EPERM (Operation not permitted) close(3) = 0 open("/etc/localtime", O_RDONLY|O_CLOEXEC) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=118, ...}) = 0 fstat(3, {st_mode=S_IFREG|0644, st_size=118, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f794a62e000 read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., 4096) = 118 lseek(3, -62, SEEK_CUR) = 56 read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., 4096) = 62 close(3) = 0 munmap(0x7f794a62e000, 4096) = 0 getpid() = 5161 write(2, "run_erl:187 [5161] Sat Sep 27 09"..., 44run_erl:187 [5161] Sat Sep 27 09:19:59 2014 ) = 44 write(2, "errno=1 'Operation not permitted"..., 60errno=1 'Operation not permitted' Could not open pty master ) = 60 exit_group(1) = ? +++ exited with 1 +++ 2014-09-26 22:39 GMT+02:00 Antoine Koener : > Hello, why do you not just run strace to see ? > > On 26 Sep 2014, at 18:12, Iv?n Mart?nez > wrote: > > I have done chmod 777 to /dev, /dev/pty*, /dev/pts and /dev/pts/* and it > didn't help. Must be something different from file access permits. Anybody > has a clue of what run_erl does and requires?. > Thank you, > Ivan > > 2014-09-25 10:41 GMT+02:00 Iv?n Mart?nez : > >> Thank you. Looks like SELinux has nothing to do with the issue: >> >> $ setenforce 0 >> setenforce: SELinux is disabled >> >> >> 2014-09-25 3:21 GMT+02:00 zxq9 : >> >>> On Wednesday 24 September 2014 21:00:32 Iv?n Mart?nez wrote: >>> > Hello all, >>> > Does anyone know why I'm having the following issue with a CentOS 7 >>> system >>> > with kernel 3.10.23?: >>> > >>> > $ run_erl priv/ log "erl" >>> > run_erl:187 [6505] Wed Sep 24 18:42:39 2014 >>> > errno=1 'Operation not permitted' >>> > Could not open pty master >>> > >>> > It works as super user. It also works in a Fedora 20 system with kernel >>> > 3.16. I couldn't find any difference in user groups or /dev file >>> permits >>> > between both systems. Starting with a user with UID above or below >>> UID_MIN >>> > doesn't make any difference. I don't think it matters, but the only >>> > difference I could find is that the CentOS has many /dev/pty* files >>> already >>> > created, while the Fedora doesn't have any. >>> >>> You might be running into SELinux permission issues. To find out try >>> doing >>> "setenforce 0" and then running it again. If that works, use a tool like >>> audit2allow or audit2why to create a policy that will permit the actions >>> you >>> require to run your program. I haven't kept up with the Fedora/RHEL world >>> since 7 came out, but Dan Walsh's blog and Red Hat's SELinux docs have >>> been >>> good resources on this in the past. >>> >>> Of course, you might have a totally different issue, but SELinux >>> booleans and >>> audit logs are the first thing I check on a Fedora-type distro when >>> something >>> doesn't work but looks like it should. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From grahamrhay@REDACTED Sat Sep 27 13:22:41 2014 From: grahamrhay@REDACTED (Graham Hay) Date: Sat, 27 Sep 2014 12:22:41 +0100 Subject: [erlang-questions] [ANN] RSS Feedparser 0.1.0 In-Reply-To: References: Message-ID: There's also this: https://code.google.com/p/atomizer/ (and a multitude of forks on github) On 22 September 2014 17:54, T Ty wrote: > Hello, > > https://github.com/ttyerlsol/rss > > A basic RSS v2.0 feedparser. There doesn't seem to be one for Erlang and I > needed it to pull popular Delicious tags. > > Enjoy, > > tty > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From semmitmondo@REDACTED Sat Sep 27 21:13:30 2014 From: semmitmondo@REDACTED (semmit mondo) Date: Sat, 27 Sep 2014 21:13:30 +0200 (CEST) Subject: [erlang-questions] HMP Message-ID: Is the ERTS able to handle a configuration like this? # grep Bogo /proc/cpuinfoBogoMIPS : 78.00 BogoMIPS : 78.00 BogoMIPS : 78.00 BogoMIPS : 78.00 BogoMIPS : 114.00 BogoMIPS : 114.00 BogoMIPS : 114.00 BogoMIPS : 114.00# cat /proc/cpuinfoprocessor : 0model name : ARMv7 Processor rev 3 (v7l)BogoMIPS : 78.00Features : swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt CPU implementer : 0x41CPU architecture: 7CPU variant : 0x0CPU part : 0xc07CPU revision : 3 processor : 1model name : ARMv7 Processor rev 3 (v7l)BogoMIPS : 78.00Features : swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt CPU implementer : 0x41CPU architecture: 7CPU variant : 0x0CPU part : 0xc07CPU revision : 3 processor : 2model name : ARMv7 Processor rev 3 (v7l)BogoMIPS : 78.00Features : swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt CPU implementer : 0x41CPU architecture: 7CPU variant : 0x0CPU part : 0xc07CPU revision : 3 processor : 3model name : ARMv7 Processor rev 3 (v7l)BogoMIPS : 78.00Features : swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt CPU implementer : 0x41CPU architecture: 7CPU variant : 0x0CPU part : 0xc07CPU revision : 3 processor : 4m odel name : ARMv7 Processor rev 3 (v7l)BogoMIPS : 114.00Features : swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt CPU implementer : 0x41CPU architecture: 7CPU variant : 0x2CPU part : 0xc0fCPU revision : 3 processor : 5model name : ARMv7 Processor rev 3 (v7l)BogoMIPS : 114.00Features : swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt CPU implementer : 0x41CPU architecture: 7CPU variant : 0x2CPU part : 0xc0fCPU revision : 3 processor : 6model name : ARMv7 Processor rev 3 (v7l)BogoMIPS : 114.00Features : swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt CPU implementer : 0x41CPU architecture: 7CPU variant : 0x2CPU part : 0xc0fCPU revision : 3 processor : 7model name : ARMv7 Processor rev 3 (v7l)BogoMIPS : 114.00Features : swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt CPU implementer : 0x41CPU architecture: 7CPU variant : 0x2CPU part : 0xc0fCPU revision : 3 ... -------------- next part -------------- An HTML attachment was scrubbed... URL: From tty.erlang@REDACTED Sun Sep 28 00:55:01 2014 From: tty.erlang@REDACTED (T Ty) Date: Sat, 27 Sep 2014 18:55:01 -0400 Subject: [erlang-questions] [ANN] RSS Feedparser 0.1.0 In-Reply-To: References: Message-ID: That was the first link I found, unfortunately it and the 2 forks I found on github only handle Atom and no parsing for RSS. Mine does RSS and no Atom heh. On Sat, Sep 27, 2014 at 7:22 AM, Graham Hay wrote: > There's also this: > > https://code.google.com/p/atomizer/ > > (and a multitude of forks on github) > > On 22 September 2014 17:54, T Ty wrote: > >> Hello, >> >> https://github.com/ttyerlsol/rss >> >> A basic RSS v2.0 feedparser. There doesn't seem to be one for Erlang and >> I needed it to pull popular Delicious tags. >> >> Enjoy, >> >> tty >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sun Sep 28 01:57:31 2014 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sun, 28 Sep 2014 01:57:31 +0200 Subject: [erlang-questions] [ANN] debbie 1.0.0 : .DEB Built In Erlang Message-ID: <54274EEB.1070502@wanadoo.fr> Hello community, debbie 1.0.0 has been released. debbie is an Erlang module that creates DEBIAN packages from an usual DEBIAN package directory structure on disk. An Erlang equivalent to dpkg-deb. debbie is FULL Erlang, and no external command is done, allowing to create DEBIAN packages on any platform where Erlang can run. No need to have root privilege, neither fakeroot command, UID/GID are modified in embedded TAR files. I hope you will enjoy this module ! Regards From eric.pailleau@REDACTED Sun Sep 28 02:00:22 2014 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sun, 28 Sep 2014 02:00:22 +0200 Subject: [erlang-questions] [ANN] debbie 1.0.0 : .DEB Built In Erlang. In-Reply-To: <54274EEB.1070502@wanadoo.fr> References: <54274EEB.1070502@wanadoo.fr> Message-ID: <54274F96.2080405@wanadoo.fr> Sorry, URL is missing in former mail : https://github.com/crownedgrouse/debbie Regards. From rvirding@REDACTED Sun Sep 28 05:46:05 2014 From: rvirding@REDACTED (Robert Virding) Date: Sat, 27 Sep 2014 20:46:05 -0700 Subject: [erlang-questions] Erlang Memory Question In-Reply-To: References: <9A5764B7-9252-4560-9AD6-BEC0DAE0305E@cs.otago.ac.nz> Message-ID: The obvious question is whether you are sure you actually need to optimise to save memory? Premature optimisation and all that. (Actually sensible advice) Maybe reviewing algorithms and datastructures will do the trick for you. Robert >From my Nexus On Sep 24, 2014 7:37 PM, "Eranga Udesh" wrote: > Thanks all for your advice. Let me see how I can apply them to my program. > It looks like my code/logic is going to get ugly and induce a performance > penalty, in order to save memory. > > Cheers, > - Eranga > > On Wed, Sep 24, 2014 at 7:58 PM, Robert Raschke > wrote: > >> I always thought that is one of the reasons to have processes. >> If you've got something big you want to throw away quickly, make a >> process for it. >> >> $ erl >> Erlang R16B03-1 (erts-5.10.4) [source] [64-bit] [smp:8:8] >> [async-threads:10] [hipe] [kernel-poll:false] >> >> Eshell V5.10.4 (abort with ^G) >> 1> erlang:memory(). >> [{total,14148416}, >> {processes,4091608}, >> {processes_used,4091488}, >> {system,10056808}, >> {atom,194289}, >> {atom_used,172614}, >> {binary,1514552}, >> {code,4026600}, >> {ets,262688}] >> 2> Pid = spawn( >> 2> fun () -> >> 2> X = binary:copy(<<1,2,3,4,5,6,7,8>>, 1024), >> 2> Y = binary:copy(X, 1024), >> 2> receive stop -> ok end >> 2> end >> 2> ). >> <0.35.0> >> 3> erlang:memory(). >> [{total,22643832}, >> {processes,4203448}, >> {processes_used,4203448}, >> {system,18440384}, >> {atom,194289}, >> {atom_used,175110}, >> {binary,9685320}, >> {code,4221791}, >> {ets,267056}] >> 4> Pid ! stop. >> stop >> 5> erlang:memory(). >> [{total,13587776}, >> {processes,4084496}, >> {processes_used,4084384}, >> {system,9503280}, >> {atom,194289}, >> {atom_used,175110}, >> {binary,748144}, >> {code,4221791}, >> {ets,267056}] >> 6> >> >> >> Regards, >> Robby >> >> >> On 24 September 2014 02:13, Eranga Udesh wrote: >> >>> Well, yes I may deliberately lie. >>> >>> However my suggestion is to, instead of doing a full sweep by the >>> garbage collector (GC) to identify data going out of scope and reclaim, can >>> the program (or rather I) deliberately say I (the calling process) is >>> finished using the said data, so the GC may free that part. >>> >>> Then the GC may carry out it's own logic, which it currently does to >>> verify if the same data is referenced by any other processes, etc., and >>> decide if to GC or not. >>> >>> = Eranga >>> >>> On Tue, Sep 23, 2014 at 12:42 PM, Richard A. O'Keefe >>> wrote: >>> >>>> >>>> On 23/09/2014, at 2:24 PM, Eranga Udesh wrote: >>>> >>>> > Thanks for the information received so far. >>>> > >>>> > Wouldn't it be good for Erlang to have a single object garbage >>>> collection function/bif? >>>> >>>> NO. In C it's called free() and it's a major cause of errors. >>>> >>>> > For example, when I no longer require a large object, I force to >>>> garbage collect it, without making a full sweep? >>>> >>>> Why should the garbage collector *believe* you that the "object" >>>> is free? You could be deliberately lying. You could (and >>>> probably are) be mistaken. How is it to know which bits you >>>> want to keep without doing its usual thing? In a shared heap >>>> implementation (which Erlang has had and may have again) the >>>> fact that *you've* finished with the object doesn't mean >>>> everyone *else* has. A meaningful operation should not depend >>>> on implementation details like that. >>>> > >>>> > As mentioned in the document, a full sweep may degrade the >>>> performance. >>>> >>>> Not half as much as freeing too much would! >>>> >>>> This is micro-optimisation. Avoid passing around large >>>> objects in the first place. >>>> >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Sep 28 08:56:26 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 28 Sep 2014 10:56:26 +0400 Subject: [erlang-questions] [ANN] debbie 1.0.0 : .DEB Built In Erlang In-Reply-To: <54274EEB.1070502@wanadoo.fr> References: <54274EEB.1070502@wanadoo.fr> Message-ID: Something like: https://github.com/flussonic/epm but without any ability to make rpm -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sun Sep 28 09:07:27 2014 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Sun, 28 Sep 2014 09:07:27 +0200 Subject: [erlang-questions] [ANN] debbie 1.0.0 : .DEB Built In Erlang Message-ID: <23rpm2qkcju7vbhjbo0jmm6x.1411888047073@email.android.com> Hi, not exacly, epm use external commands, like 'ar' to build a .deb... debbie is totally full Erlang. By the way I plan to do the same for RPM. Regards ? Envoy? depuis mon mobile ? Eric Max Lapshin a ?crit?: >Something like: https://github.com/flussonic/epm but without any ability >to make rpm -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Sep 28 09:10:45 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 28 Sep 2014 11:10:45 +0400 Subject: [erlang-questions] [ANN] debbie 1.0.0 : .DEB Built In Erlang In-Reply-To: <23rpm2qkcju7vbhjbo0jmm6x.1411888047073@email.android.com> References: <23rpm2qkcju7vbhjbo0jmm6x.1411888047073@email.android.com> Message-ID: On Sun, Sep 28, 2014 at 11:07 AM, Eric Pailleau wrote: > > By the way I plan to do the same for RPM. > Good luck! It is extremely easy to make deb, because it was designed by clever people, but you will fall into abyss of pain, when you will try to make rpm for centos and then you will find out that this rpm can make segfault and corrupt rpm database under Suse =)) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Sun Sep 28 09:11:46 2014 From: rvirding@REDACTED (Robert Virding) Date: Sun, 28 Sep 2014 00:11:46 -0700 Subject: [erlang-questions] Specs for optional callbacks (try 2) In-Reply-To: <54240CD3.1080107@ninenines.eu> References: <5422F6CB.6080403@ninenines.eu> <56466BD70414EA48969B4064696CF28C21DF8EE5@ESESSMB204.ericsson.se> <54240CD3.1080107@ninenines.eu> Message-ID: Just to throw something completely different and definitely not backwards compatible I would prefer something like: -callback([foo/2,bar/4]). -optional_callback([bar/3,boo/14]). foo(...) -> ... . bar(...) -> ... . baz(...) -> ... . boo(...) -> ... . This because it mirrors how -export(). behaves and does not push the information down into the actual definition. I think not seeing it at the start of the module is a bad option as it means I have to step down into the module to see what it exports and, in this case, what are the callbacks. You have to delve down into the module to find this "global" information. I find it very annoying with languages that do this, which is why I purposely followed erlang when doing LFE. Yes, I know it is too late to change it. Is it? Maybe a parse transform to get it right? Robert On 25 September 2014 05:38, Lo?c Hoguin wrote: > On 09/25/2014 03:01 PM, Hans Bolinder wrote: > >> [Lo?c Hoguin:] >> >>> I didn't get any reply on my previous question almost 2 years ago here: >>> http://erlang.org/pipermail/erlang-questions/2013-January/071763.html >>> >> >> I missed your original question, otherwise I would have replied to it. >> > > No worries. :) > > There is an implementation in Erlang/OTP 18, see commit 683a293, using >> the second syntax suggested by Kostis, -optional_callbacks/1. >> > > Woot! > > This looks great. I guess I'll just have to wait now and play with it a > few weeks before 18 gets released. > > Thanks for the heads up! > > -- > Lo?c Hoguin > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sun Sep 28 09:52:29 2014 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Sun, 28 Sep 2014 09:52:29 +0200 Subject: [erlang-questions] [ANN] debbie 1.0.0 : .DEB Built In Erlang Message-ID: Hi Max, I know this, I already did some tests for rpm. I understand, now, why I like Debian family OS... :-) debbie is not ?yet another external commands wrapper? . you can create a (multi-platform) .deb on a windows machine without cygwin. You are free to integrate it to epm, to get rid of external executable... regards ? Envoy? depuis mon mobile ? Eric Max Lapshin a ?crit?: >On Sun, Sep 28, 2014 at 11:07 AM, Eric Pailleau >wrote: > >> >> >By the way I plan to do the same for RPM. >> > > >Good luck! > >It is extremely easy to make deb, because it was designed by clever people, >but you will fall into abyss of pain, when you will try to make rpm for >centos and then you will find out that this rpm can make segfault and >corrupt rpm database under Suse =)) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Sun Sep 28 10:47:41 2014 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 28 Sep 2014 10:47:41 +0200 Subject: [erlang-questions] Specs for optional callbacks (try 2) In-Reply-To: References: <5422F6CB.6080403@ninenines.eu> <56466BD70414EA48969B4064696CF28C21DF8EE5@ESESSMB204.ericsson.se> <54240CD3.1080107@ninenines.eu> Message-ID: The most backwards-compatible way to do this would surely be: behaviour_info(callbacks) -> [{foo, 2}, {bar, 4}]; behaviour_info(optional_callbacks) -> [{bar, 3}, {boo, 14}]; behaviour_info(_) -> undefined. Presumably, having behaviour_info/1 together with -callback specs was disallowed because the compiler would have to check consistency between them, and to partial evaluation of the behaviour_info/1 code? In this respect, -callbacks/1 and -optional_callbacks/1 are better since they can be understood directly after parsing. But they introduce some uncertainties: - are multiple -callbacks/1 attributes allowed? If not explicitly disallowed, they would have to be unioned together. - should the compiler produce a default -callbacks/1 directive including all specified callbacks? (except for those represented in -optional_callbacks/1?) I think the most consistent and clear alternative is to introduce -optional_callback specs, given the path we?re already on. BR, Ulf W On 28 Sep 2014, at 09:11, Robert Virding wrote: > Just to throw something completely different and definitely not backwards compatible I would prefer something like: > > -callback([foo/2,bar/4]). > -optional_callback([bar/3,boo/14]). > > foo(...) -> ... . > bar(...) -> ... . > baz(...) -> ... . > boo(...) -> ... . > > This because it mirrors how -export(). behaves and does not push the information down into the actual definition. I think not seeing it at the start of the module is a bad option as it means I have to step down into the module to see what it exports and, in this case, what are the callbacks. You have to delve down into the module to find this "global" information. I find it very annoying with languages that do this, which is why I purposely followed erlang when doing LFE. > > Yes, I know it is too late to change it. Is it? Maybe a parse transform to get it right? > > Robert > > > On 25 September 2014 05:38, Lo?c Hoguin wrote: > On 09/25/2014 03:01 PM, Hans Bolinder wrote: > [Lo?c Hoguin:] > I didn't get any reply on my previous question almost 2 years ago here: > http://erlang.org/pipermail/erlang-questions/2013-January/071763.html > > I missed your original question, otherwise I would have replied to it. > > No worries. :) > > There is an implementation in Erlang/OTP 18, see commit 683a293, using > the second syntax suggested by Kostis, -optional_callbacks/1. > > Woot! > > This looks great. I guess I'll just have to wait now and play with it a few weeks before 18 gets released. > > Thanks for the heads up! > > -- > Lo?c Hoguin > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Sun Sep 28 11:28:16 2014 From: kostis@REDACTED (Kostis Sagonas) Date: Sun, 28 Sep 2014 12:28:16 +0300 Subject: [erlang-questions] Specs for optional callbacks (try 2) In-Reply-To: References: <5422F6CB.6080403@ninenines.eu> <56466BD70414EA48969B4064696CF28C21DF8EE5@ESESSMB204.ericsson.se> <54240CD3.1080107@ninenines.eu> Message-ID: <5427D4B0.9000406@cs.ntua.gr> On 09/28/2014 11:47 AM, Ulf Wiger wrote: > The most backwards-compatible way to do this would surely be: > > behaviour_info(callbacks) -> > [{foo, 2}, {bar, 4}]; > behaviour_info(optional_callbacks) -> > [{bar, 3}, {boo, 14}]; > behaviour_info(_) -> > undefined. > > Presumably, having behaviour_info/1 together with -callback specs was > disallowed because the compiler would have to check consistency between > them, and to partial evaluation of the behaviour_info/1 code? No. With -callback information, the whole behaviour_info/1 function is redundant since it can be (and is) generated automatically by the compiler from the -callback specifications. behaviour_info/1 is currently allowed to be present for backwards compatibility but its support should be discouraged and discontinued. Besides the {F,A} notation being butt ugly, behaviour modules should really document the expected types of their callbacks, not just their arity. Kostis From grahamrhay@REDACTED Sun Sep 28 13:06:05 2014 From: grahamrhay@REDACTED (Graham Hay) Date: Sun, 28 Sep 2014 12:06:05 +0100 Subject: [erlang-questions] [ANN] RSS Feedparser 0.1.0 In-Reply-To: References: Message-ID: Ah, yes, sorry. I thought it was the other way round (should have been obvious from the name!). I have a fork that supports both: https://github.com/grahamrhay/atomizer but there isn't really a canonical repo upstream to send a PR to. On 27 September 2014 23:55, T Ty wrote: > That was the first link I found, unfortunately it and the 2 forks I found > on github only handle Atom and no parsing for RSS. Mine does RSS and no > Atom heh. > > On Sat, Sep 27, 2014 at 7:22 AM, Graham Hay wrote: > >> There's also this: >> >> https://code.google.com/p/atomizer/ >> >> (and a multitude of forks on github) >> >> On 22 September 2014 17:54, T Ty wrote: >> >>> Hello, >>> >>> https://github.com/ttyerlsol/rss >>> >>> A basic RSS v2.0 feedparser. There doesn't seem to be one for Erlang and >>> I needed it to pull popular Delicious tags. >>> >>> Enjoy, >>> >>> tty >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan.martinez@REDACTED Sun Sep 28 19:27:39 2014 From: ivan.martinez@REDACTED (=?UTF-8?B?SXbDoW4gTWFydMOtbmV6?=) Date: Sun, 28 Sep 2014 19:27:39 +0200 Subject: [erlang-questions] Could not open pty master In-Reply-To: References: <4676551.iEBOCzXdtb@burrito> <1A8AF49E-8E57-46DE-8EFB-045487D436B2@gmail.com> Message-ID: $ ls -l crw--w---- 1 ivan tty 136, 0 Sep 28 17:23 0 c--------- 1 root root 5, 2 Sep 24 15:43 ptmx I'm running run_erl as user ivan. Running it as root works, yes. Thank you, Ivan 2014-09-27 15:24 GMT+02:00 Aidan Hobson Sayers : > chown("/dev/pts/1", 998, 5) = -1 EPERM (Operation not > permitted) > > What does ls -l in /dev/pts/ say about the owner of the files in > /dev/pts/? And what user are you running as (and, by extension, does > running as root work)? > > On 27 September 2014 10:27, Iv?n Mart?nez > wrote: > >> Thank you Antoine. I'm a developer rather than a systems administrator, I >> didn't know strace existed. I have tried it but I don't know what to make >> from the output: >> >> $ strace run_erl priv/ log "erl" >> execve("/usr/local/bin/run_erl", ["run_erl", "priv/", "log", "erl"], [/* >> 23 vars */]) = 0 >> brk(0) = 0x60c000 >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) >> = 0x7f794a62f000 >> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or >> directory) >> open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 >> fstat(3, {st_mode=S_IFREG|0644, st_size=32636, ...}) = 0 >> mmap(NULL, 32636, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f794a627000 >> close(3) = 0 >> open("/lib64/libutil.so.1", O_RDONLY|O_CLOEXEC) = 3 >> read(3, >> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\17\0\0\0\0\0\0"..., 832) >> = 832 >> fstat(3, {st_mode=S_IFREG|0755, st_size=14608, ...}) = 0 >> mmap(NULL, 2105616, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) >> = 0x7f794a20d000 >> mprotect(0x7f794a20f000, 2093056, PROT_NONE) = 0 >> mmap(0x7f794a40e000, 8192, PROT_READ|PROT_WRITE, >> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7f794a40e000 >> close(3) = 0 >> open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3 >> read(3, >> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\16\0\0\0\0\0\0"..., >> 832) = 832 >> fstat(3, {st_mode=S_IFREG|0755, st_size=19512, ...}) = 0 >> mmap(NULL, 2109744, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) >> = 0x7f794a009000 >> mprotect(0x7f794a00c000, 2093056, PROT_NONE) = 0 >> mmap(0x7f794a20b000, 8192, PROT_READ|PROT_WRITE, >> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f794a20b000 >> close(3) = 0 >> open("/lib64/libm.so.6", O_RDONLY|O_CLOEXEC) = 3 >> read(3, >> "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260T\0\0\0\0\0\0"..., 832) >> = 832 >> fstat(3, {st_mode=S_IFREG|0755, st_size=1141552, ...}) = 0 >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) >> = 0x7f794a626000 >> mmap(NULL, 3150168, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) >> = 0x7f7949d07000 >> mprotect(0x7f7949e08000, 2093056, PROT_NONE) = 0 >> mmap(0x7f794a007000, 8192, PROT_READ|PROT_WRITE, >> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x100000) = 0x7f794a007000 >> close(3) = 0 >> open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 >> read(3, >> "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\34\2\0\0\0\0\0"..., 832) >> = 832 >> fstat(3, {st_mode=S_IFREG|0755, st_size=2107600, ...}) = 0 >> mmap(NULL, 3932736, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) >> = 0x7f7949946000 >> mprotect(0x7f7949afc000, 2097152, PROT_NONE) = 0 >> mmap(0x7f7949cfc000, 24576, PROT_READ|PROT_WRITE, >> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b6000) = 0x7f7949cfc000 >> mmap(0x7f7949d02000, 16960, PROT_READ|PROT_WRITE, >> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f7949d02000 >> close(3) = 0 >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) >> = 0x7f794a625000 >> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) >> = 0x7f794a623000 >> arch_prctl(ARCH_SET_FS, 0x7f794a623740) = 0 >> mprotect(0x7f7949cfc000, 16384, PROT_READ) = 0 >> mprotect(0x7f794a007000, 4096, PROT_READ) = 0 >> mprotect(0x7f794a20b000, 4096, PROT_READ) = 0 >> mprotect(0x7f794a40e000, 4096, PROT_READ) = 0 >> mprotect(0x604000, 4096, PROT_READ) = 0 >> mprotect(0x7f794a630000, 4096, PROT_READ) = 0 >> munmap(0x7f794a627000, 32636) = 0 >> brk(0) = 0x60c000 >> brk(0x62f000) = 0x62f000 >> brk(0) = 0x62f000 >> openat(AT_FDCWD, "priv/", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3 >> getdents(3, /* 6 entries */, 32768) = 200 >> getdents(3, /* 0 entries */, 32768) = 0 >> close(3) = 0 >> mknod("priv/erlang.pipe.2.r", S_IFIFO|0666) = 0 >> open("priv/erlang.pipe.2.w", O_WRONLY|O_NONBLOCK) = -1 ENOENT (No such >> file or directory) >> mknod("priv/erlang.pipe.2.w", S_IFIFO|0666) = 0 >> open("/dev/ptmx", O_RDWR) = 3 >> statfs("/dev/pts", {f_type="DEVPTS_SUPER_MAGIC", f_bsize=4096, >> f_blocks=0, f_bfree=0, f_bavail=0, f_files=0, f_ffree=0, f_fsid={0, 0}, >> f_namelen=255, f_frsize=4096}) = 0 >> ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, >> {B38400 opost isig icanon echo ...}) = 0 >> ioctl(3, TIOCGPTN, [1]) = 0 >> stat("/dev/pts/1", {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 1), ...}) >> = 0 >> ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, >> {B38400 opost isig icanon echo ...}) = 0 >> ioctl(3, TIOCGPTN, [1]) = 0 >> stat("/dev/pts/1", {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 1), ...}) >> = 0 >> getuid() = 998 >> socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 4 >> connect(4, {sa_family=AF_LOCAL, sun_path="/var/run/nscd/socket"}, 110) = >> -1 ENOENT (No such file or directory) >> close(4) = 0 >> socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 4 >> connect(4, {sa_family=AF_LOCAL, sun_path="/var/run/nscd/socket"}, 110) = >> -1 ENOENT (No such file or directory) >> close(4) = 0 >> open("/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 4 >> fstat(4, {st_mode=S_IFREG|0644, st_size=1728, ...}) = 0 >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) >> = 0x7f794a62e000 >> read(4, "#\n# /etc/nsswitch.conf\n#\n# An ex"..., 4096) = 1728 >> read(4, "", 4096) = 0 >> close(4) = 0 >> munmap(0x7f794a62e000, 4096) = 0 >> open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 4 >> fstat(4, {st_mode=S_IFREG|0644, st_size=32636, ...}) = 0 >> mmap(NULL, 32636, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7f794a627000 >> close(4) = 0 >> open("/lib64/libnss_files.so.2", O_RDONLY|O_CLOEXEC) = 4 >> read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\"\0\0\0\0\0\0"..., >> 832) = 832 >> fstat(4, {st_mode=S_IFREG|0755, st_size=58288, ...}) = 0 >> mmap(NULL, 2144360, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) >> = 0x7f794973a000 >> mprotect(0x7f7949745000, 2093056, PROT_NONE) = 0 >> mmap(0x7f7949944000, 8192, PROT_READ|PROT_WRITE, >> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0xa000) = 0x7f7949944000 >> close(4) = 0 >> mprotect(0x7f7949944000, 4096, PROT_READ) = 0 >> munmap(0x7f794a627000, 32636) = 0 >> open("/etc/group", O_RDONLY|O_CLOEXEC) = 4 >> fstat(4, {st_mode=S_IFREG|0644, st_size=591, ...}) = 0 >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) >> = 0x7f794a62e000 >> read(4, "root:x:0:\nbin:x:1:\ndaemon:x:2:\ns"..., 4096) = 591 >> close(4) = 0 >> munmap(0x7f794a62e000, 4096) = 0 >> chown("/dev/pts/1", 998, 5) = -1 EPERM (Operation not >> permitted) >> close(3) = 0 >> open("/dev/ptmx", O_RDWR) = 3 >> ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, >> {B38400 opost isig icanon echo ...}) = 0 >> ioctl(3, TIOCGPTN, [1]) = 0 >> stat("/dev/pts/1", {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 1), ...}) >> = 0 >> getuid() = 998 >> chown("/dev/pts/1", 998, 5) = -1 EPERM (Operation not >> permitted) >> close(3) = 0 >> open("/etc/localtime", O_RDONLY|O_CLOEXEC) = 3 >> fstat(3, {st_mode=S_IFREG|0644, st_size=118, ...}) = 0 >> fstat(3, {st_mode=S_IFREG|0644, st_size=118, ...}) = 0 >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) >> = 0x7f794a62e000 >> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., >> 4096) = 118 >> lseek(3, -62, SEEK_CUR) = 56 >> read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., >> 4096) = 62 >> close(3) = 0 >> munmap(0x7f794a62e000, 4096) = 0 >> getpid() = 5161 >> write(2, "run_erl:187 [5161] Sat Sep 27 09"..., 44run_erl:187 [5161] Sat >> Sep 27 09:19:59 2014 >> ) = 44 >> write(2, "errno=1 'Operation not permitted"..., 60errno=1 'Operation not >> permitted' >> Could not open pty master >> ) = 60 >> exit_group(1) = ? >> +++ exited with 1 +++ >> >> 2014-09-26 22:39 GMT+02:00 Antoine Koener : >> >>> Hello, why do you not just run strace to see ? >>> >>> On 26 Sep 2014, at 18:12, Iv?n Mart?nez >>> wrote: >>> >>> I have done chmod 777 to /dev, /dev/pty*, /dev/pts and /dev/pts/* and it >>> didn't help. Must be something different from file access permits. Anybody >>> has a clue of what run_erl does and requires?. >>> Thank you, >>> Ivan >>> >>> 2014-09-25 10:41 GMT+02:00 Iv?n Mart?nez : >>> >>>> Thank you. Looks like SELinux has nothing to do with the issue: >>>> >>>> $ setenforce 0 >>>> setenforce: SELinux is disabled >>>> >>>> >>>> 2014-09-25 3:21 GMT+02:00 zxq9 : >>>> >>>>> On Wednesday 24 September 2014 21:00:32 Iv?n Mart?nez wrote: >>>>> > Hello all, >>>>> > Does anyone know why I'm having the following issue with a CentOS 7 >>>>> system >>>>> > with kernel 3.10.23?: >>>>> > >>>>> > $ run_erl priv/ log "erl" >>>>> > run_erl:187 [6505] Wed Sep 24 18:42:39 2014 >>>>> > errno=1 'Operation not permitted' >>>>> > Could not open pty master >>>>> > >>>>> > It works as super user. It also works in a Fedora 20 system with >>>>> kernel >>>>> > 3.16. I couldn't find any difference in user groups or /dev file >>>>> permits >>>>> > between both systems. Starting with a user with UID above or below >>>>> UID_MIN >>>>> > doesn't make any difference. I don't think it matters, but the only >>>>> > difference I could find is that the CentOS has many /dev/pty* files >>>>> already >>>>> > created, while the Fedora doesn't have any. >>>>> >>>>> You might be running into SELinux permission issues. To find out try >>>>> doing >>>>> "setenforce 0" and then running it again. If that works, use a tool >>>>> like >>>>> audit2allow or audit2why to create a policy that will permit the >>>>> actions you >>>>> require to run your program. I haven't kept up with the Fedora/RHEL >>>>> world >>>>> since 7 came out, but Dan Walsh's blog and Red Hat's SELinux docs have >>>>> been >>>>> good resources on this in the past. >>>>> >>>>> Of course, you might have a totally different issue, but SELinux >>>>> booleans and >>>>> audit logs are the first thing I check on a Fedora-type distro when >>>>> something >>>>> doesn't work but looks like it should. >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> >>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Mon Sep 29 04:59:31 2014 From: jinni.park@REDACTED (Park, Sungjin) Date: Mon, 29 Sep 2014 11:59:31 +0900 Subject: [erlang-questions] Change suggestion in ranch/cowboy In-Reply-To: <5425152F.6080105@ninenines.eu> References: <5425152F.6080105@ninenines.eu> Message-ID: Oh, I missed the function thanks. On Fri, Sep 26, 2014 at 4:26 PM, Lo?c Hoguin wrote: > On 09/26/2014 05:39 AM, Park, Sungjin wrote: > >> 2) Dynamic parameters >> Currently, parameters required to run ranch_protocol are stored in the >> ranch_conns_sup and given to the ranch_protocol when spawning. Combined >> with cowboy, cascaded parameters like cowboy_protocol -> cowboy_router, >> cowboy_handler -> dispatch env are constant once started. Changing any >> of the above requires ranch_conns_sup restart. >> > > Hmm no? > > http://ninenines.eu/docs/en/ranch/1.0/manual/ranch/index. > html#set_protocol_options > http://ninenines.eu/docs/en/cowboy/1.0/manual/cowboy/index.html#set_env > > Changing for all existing connections is up to the application though. > > -- > Lo?c Hoguin > http://ninenines.eu > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From tty.erlang@REDACTED Mon Sep 29 13:29:08 2014 From: tty.erlang@REDACTED (T Ty) Date: Mon, 29 Sep 2014 07:29:08 -0400 Subject: [erlang-questions] [ANN] Rockse: A Facebook Rocksdb Erlang binding Message-ID: A very initial non-production ready release of the Facebook Rocksdb Erlang binding. Currently it only supports put/get/delete https://github.com/ttyerlsol/rockse My motivation is to have some basics in-place which would encourage investigation into distributed databases. At this release rockse only deals with tables within a single node, later release will also do database replication with per table replication strategies. Enjoy -------------- next part -------------- An HTML attachment was scrubbed... URL: From z@REDACTED Mon Sep 29 17:34:46 2014 From: z@REDACTED (Danil Zagoskin) Date: Mon, 29 Sep 2014 19:34:46 +0400 Subject: [erlang-questions] Can't start observer on MacOSX 10.9.5 in ESL Erlang 17.1 In-Reply-To: References: Message-ID: Hi Nicholas! ESL Erlang has wxe_driver built against some not-known-by-everyone version of wxmac. You need to rebuild this driver from source by downloading otp src, minimal configuration (disable all options except wx) and quick build. Then you replace the .so distributed by ESL with your fresh one. Faster way ? find pre-built binary (I publish ones from time to time in this folder: https://yadi.sk/d/iAlQVrIeVMvHw) and replace ESL's one with it. Possibly the most proper way ? install Erlang using brew. On Fri, Sep 26, 2014 at 7:43 PM, wrote: > > Sorry if this appears as a repost. Some signup confusion. Anyway... > Trying to start observer off of clean install. ESL Erlang 17.1, brew > install wxmac. OS X 10.9.5. > > Erlang/OTP 17 [erts-6.1] [source-d2a4c20] [64-bit] [smp:4:4] > [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V6.1 (abort with ^G) > 1> observer:start(). > > =ERROR REPORT==== 26-Sep-2014::16:31:42 === > WX Failed loading "wxe_driver"@"/usr/local/lib/erlang/lib/wx-1.3/priv" > {error,{{load_driver,"dlopen(/usr/local/lib/erlang/lib/wx-1.3/priv/wxe_driver.so, > 2): Symbol not found: > __ZThn1152_N16wxStyledTextCtrl10AppendTextERK8wxString\n Referenced from: > /usr/local/lib/erlang/lib/wx-1.3/priv/wxe_driver.so\n Expected in: flat > namespace\n"}, > [{wxe_server,start,1,[{file,"wxe_server.erl"},{line,64}]}, > {wx,new,1,[{file,"wx.erl"},{line,114}]}, > {observer_wx,init,1,[{file,"observer_wx.erl"},{line,91}]}, > {wx_object,init_it,6,[{file,"wx_object.erl"},{line,299}]}, > {proc_lib,init_p_do_apply,3, > [{file,"proc_lib.erl"},{line,239}]}]}} > > > Thanks, > > Nicholas Bellerophon > nbellerophon@REDACTED > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Mon Sep 29 18:10:15 2014 From: roger@REDACTED (Roger Lipscombe) Date: Mon, 29 Sep 2014 17:10:15 +0100 Subject: [erlang-questions] Get grandchild pid when using nested supervisors? Message-ID: I'm attempting to use nested supervisors, as follows: Top -> A (simple_one_for_one) --temporary--> B (one_for_one) --transient--> W The point of this is so that, if W crashes, it gets restarted according to B's strategy, but that -- if the restart intensity is reached, A is _not_ killed (because it's supervising a large number of B->W pairs). There's only one W for each B. So, the first question: is this sane? Or is there a better way to implement this? Assuming it is sane, my second question: Calling supervisor:start_child(A, []) returns {ok, BPid}, where I really want WPid. What's a sensible way to get hold of it? Thanks, Roger. From mloftis@REDACTED Mon Sep 29 18:24:07 2014 From: mloftis@REDACTED (Michael Loftis) Date: Mon, 29 Sep 2014 09:24:07 -0700 Subject: [erlang-questions] Get grandchild pid when using nested supervisors? In-Reply-To: References: Message-ID: On Mon, Sep 29, 2014 at 9:10 AM, Roger Lipscombe wrote: > I'm attempting to use nested supervisors, as follows: > > Top -> A (simple_one_for_one) --temporary--> B (one_for_one) --transient--> W > > The point of this is so that, if W crashes, it gets restarted > according to B's strategy, but that -- if the restart intensity is > reached, A is _not_ killed (because it's supervising a large number of > B->W pairs). There's only one W for each B. > > So, the first question: is this sane? Or is there a better way to > implement this? > > Assuming it is sane, my second question: > > Calling supervisor:start_child(A, []) returns {ok, BPid}, where I > really want WPid. What's a sensible way to get hold of it? Typically I would personally have W "call back" to you with it's Pid. You might have a timeout on wait for the callback. You'd call the start with additional args so that your pid would get passed through the supervision tree to the eventual WPid process init(), and either there in init() have it gen_server:cast or do a usual erlang Pid ! {wpid, self()} or similar. As for if your supervision tree is normal like that, it doesn't really matter if it's what you need, right? However, it is normal. It's actually a lot more normal to have a little bit of a deeper supervision tree than just a single level, IE, to have one like yours. Fire up mnesia and connect observer to your node. OR any number of other applications, you'll see that their supervision trees are 2-3-4 levels deep depending on their needs. > > Thanks, > Roger. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- "Genius might be described as a supreme capacity for getting its possessors into trouble of all kinds." -- Samuel Butler From pabftk@REDACTED Mon Sep 29 11:14:14 2014 From: pabftk@REDACTED (Pavel Baturko) Date: Mon, 29 Sep 2014 13:14:14 +0400 Subject: [erlang-questions] Pid after node restart Message-ID: Hi, I have a question about internal structure of pid and possibility to compare pids in distributed system. I see strange (for me) behavior of erlang (version 17.1) and I need help to explain this. My case is: I have 2 connected nodes. 1. On node 1 I generate a pid P (<0.X.0>) 2. P is transfered to node 2 (on node 2 I receive it as ) 3. P is transfered back from node 2 to node 1 (I'm receiving it as P2) 4. P2 is recognized as local pid (<0.X.0>) 5. I can compare P and P2 on node 1 and they are identical 6. I'm restarting node 1 7. On node 1 I generate the same pid P as in step 1 (<0.X.0>) 8. I repeat step 3, again receive initial pid as P2 9. P2 is recognized as local pid (<0.X.0>) - as before 10. But now P == P2 returns false. 11. Also P == list_to_pid(pid_to_list(P2)) returns true Could someone explain this behavior? What is changing in internal pid representation after node restart so comparison in step 10 returns false? And why comparison in step 11 returns true? My code (first lines for node 1, after empty line code for node 2): ------------ erl -sname n1 -setcookie t erl -sname n2 -setcookie t ------------ net_kernel:connect_node('n2@REDACTED'). global:register_name(n1, self()). global:register_name(n2, self()). ------------ P = pid(0,1,0). global:send(n2, P). receive P -> P end. ------------ receive P2 -> P2 end. P == P2. % this will return true global:send(n1, P). ------------ %restart n1 net_kernel:connect_node('n2@REDACTED'). global:register_name(n1, self()). P = pid(0,1,0). receive P2 -> P2 end. P == P2. % this will return false global:send(n1, P). ------------ Thanks, Pavel -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Tue Sep 30 07:06:24 2014 From: vances@REDACTED (Vance Shipley) Date: Tue, 30 Sep 2014 10:36:24 +0530 Subject: [erlang-questions] Get grandchild pid when using nested supervisors? In-Reply-To: References: Message-ID: On Mon, Sep 29, 2014 at 9:40 PM, Roger Lipscombe wrote: > > Calling supervisor:start_child(A, []) returns {ok, BPid}, where I > really want WPid. What's a sensible way to get hold of it? > {ok, BPid} = supervisor:start_child(A, []), [{_, WPid, _, _}] = supervisor:which_children(BPid) -- -Vance -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Tue Sep 30 10:11:36 2014 From: tony@REDACTED (Tony Rogvall) Date: Tue, 30 Sep 2014 10:11:36 +0200 Subject: [erlang-questions] Pid after node restart In-Reply-To: References: Message-ID: The magic is called creation :-) You can check the creation number in a node by calling: erlang:system_info(creation). This number is either 1,2 or 3 in current implementation, and its purpose is to be able to make sure that code can tell new pids from old ones, as you discovered. The nodes creation number is handed out by the epmd (Erlang port mapper daemon) when a node starts: > erl -sname n1 > erlang:system_info(creation). 1 > halt(). > erl -sname n1 > erlang:system_info(creation). 2 > halt(). > erl -sname n1 > erlang:system_info(creation). 3 > halt(). > erl -sname n1 > erlang:system_info(creation). 1 > halt(). The creation is not stored together with the pid it self, but stored in a distribution entry that is populated and transfered with the distribution protocol. It is a bit strange (to say the least) the the print names of the two pids are identical but they totally different. The ert_debug:size reveal that one is a "local" pid and the other one is a "remote". Transferring the pids to P1 and P2 and run: > erts_debug:size(P1). 4 > erts_debug:size(P2). 0 Meaning that P2 is the local pid and P1 is the "remote"/old pid. The only (non-supported) way I found to print this info is an obscure (but nice) function called erts_debug:get_internal_state(node_and_dist_references). But before this can be called this function must be called first: erts_debug:set_internal_state(available_internal_state, true). The data in my case looks like: > erts_debug:get_internal_state(node_and_dist_references). {{node_references,[{{n1@REDACTED,2}, 1, [{{system,undefined},[{system,1}]}]}, {{n2@REDACTED,3}, 12, [{{process,<0.13.0>},[{heap,11}]}, {{process,<0.14.0>},[{heap,1}]}]}, {{n1@REDACTED,1}, 7, [{{process,<0.32.0>},[{heap,4}]}, {{process,<0.41.0>},[{heap,2}]}, {{process,<0.77.0>},[{heap,1}]}]}]}, {dist_references,[{n2@REDACTED,2, [{{port,#Port<0.2237>},[{control,1}]}, {{node,{n2@REDACTED,3}},[{node,1}]}]}, {n1@REDACTED,3, [{{process,<0.20.0>},[{control,1}]}, {{node,{n1@REDACTED,1}},[{node,1}]}, {{node,{n1@REDACTED,2}},[{node,1}]}]}]}} You can see the {n1@REDACTED,1}, {n1@REDACTED,2} in the node_reference part of the list. Do not ask about the other number, you have to read the source code :-) /Tony On 29 sep 2014, at 11:14, Pavel Baturko wrote: > Hi, > > I have a question about internal structure of pid and possibility to compare pids in distributed system. > I see strange (for me) behavior of erlang (version 17.1) and I need help to explain this. > My case is: > I have 2 connected nodes. > ? On node 1 I generate a pid P (<0.X.0>) > ? P is transfered to node 2 (on node 2 I receive it as ) > ? P is transfered back from node 2 to node 1 (I'm receiving it as P2) > ? P2 is recognized as local pid (<0.X.0>) > ? I can compare P and P2 on node 1 and they are identical > ? I'm restarting node 1 > ? On node 1 I generate the same pid P as in step 1 (<0.X.0>) > ? I repeat step 3, again receive initial pid as P2 > ? P2 is recognized as local pid (<0.X.0>) - as before > ? But now P == P2 returns false. > ? Also P == list_to_pid(pid_to_list(P2)) returns true > Could someone explain this behavior? > What is changing in internal pid representation after node restart so comparison in step 10 returns false? And why comparison in step 11 returns true? > > My code (first lines for node 1, after empty line code for node 2): > ------------ > erl -sname n1 -setcookie t > > erl -sname n2 -setcookie t > ------------ > net_kernel:connect_node('n2@REDACTED'). > global:register_name(n1, self()). > > global:register_name(n2, self()). > ------------ > P = pid(0,1,0). > global:send(n2, P). > > receive P -> P end. > ------------ > receive P2 -> P2 end. > P == P2. % this will return true > > global:send(n1, P). > ------------ > %restart n1 > net_kernel:connect_node('n2@REDACTED'). > global:register_name(n1, self()). > P = pid(0,1,0). > receive P2 -> P2 end. > P == P2. % this will return false > > global:send(n1, P). > ------------ > > Thanks, > Pavel > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" From joakim@REDACTED Tue Sep 30 11:29:20 2014 From: joakim@REDACTED (Joakim Hirsch) Date: Tue, 30 Sep 2014 11:29:20 +0200 Subject: [erlang-questions] Reading data from wxListCtrl Message-ID: <542A77F0.9060503@snookerhallen.se> Hi, I'm having some trouble using wxListCtrl. I've created a table and filled it with data. When a row is activated (2-click) I receive an index into the table: the row number. I want to read (some of) the data from that row, but I can't figure out how. To me it appears as if wxListCtrl:getItem/1 doesn't read the data into the Item. Below is a minimal example. Where do I go wrong? Regs, Joakim -module(test_gui). -behaviour(wx_object). %% Client API -export([start/0]). %% wx_object callbacks -export([init/1, terminate/2, code_change/3, handle_info/2, handle_call/3, handle_cast/2, handle_event/2, handle_sync_event/3]). -include_lib("wx/include/wx.hrl"). -record(state, { parent }). start() -> Wx = wx:new(), Size = {size, {500, 500}}, Frame = wxFrame:new(Wx, ?wxID_ANY, "Title", [Size]), Config = [{parent, Frame}], wx_object:start_link(?MODULE, Config, []). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% init(Config) -> wx:batch(fun() -> do_init(Config) end). do_init(Config) -> Parent = proplists:get_value(parent, Config), % Create panel and set up sizer. Panel = wxPanel:new(Parent, []), MainSizer = wxBoxSizer:new(?wxVERTICAL), wxPanel:setSizer(Panel, MainSizer), LC = wxListCtrl:new(Panel, [{style, ?wxLC_REPORT bor ?wxLC_SINGLE_SEL}]), wxSizer:add(MainSizer, LC, [{flag, ?wxEXPAND}]), % Create columns wxListCtrl:insertColumn(LC, 0, "name", []), wxListCtrl:insertColumn(LC, 1, "address", []), % Create the row wxListCtrl:insertItem(LC, 0, ""), % Write cell 0,0 wxListCtrl:setItem(LC, 0, 0, "Bob"), % Write cell 0,1 Item1 = wxListItem:new(), wxListItem:setId(Item1, 0), wxListItem:setColumn(Item1, 1), wxListItem:setText(Item1, "Street"), true = wxListCtrl:setItem(LC, Item1), wxFrame:show(Parent), % Read 0,0 It0 = wxListItem:new(), wxListItem:setId(It0, 0), wxListItem:setColumn(It0, 0), true = wxListCtrl:getItem(LC, It0), io:format("Row 0: ~p~n", [wxListItem:getId(It0)]), io:format("Col 0: ~p~n", [wxListItem:getColumn(It0)]), io:format("getText: ~p~n", [wxListItem:getText(It0)]), % Read 0,1 It1 = wxListItem:new(), wxListItem:setId(It1, 0), wxListItem:setColumn(It1, 1), true = wxListCtrl:getItem(LC, It1), io:format("Row 0: ~p~n", [wxListItem:getId(It1)]), io:format("Col 1: ~p~n", [wxListItem:getColumn(It1)]), io:format("getText: ~p~n", [wxListItem:getText(It1)]), wxWindow:connect(LC, command_list_item_activated), {Parent, #state{parent = Parent}}. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Sync events i.e. from callbacks must return ok, it can not return a %% new state. Do the redrawing here. handle_sync_event(#wx{event = #wxPaint{}},_, #state{}) -> ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Callbacks handled as normal gen_server callbacks %% handle_event(#wx{obj = LC, event = #wxList{type = command_list_item_activated, itemIndex = Index}}, State) -> It0 = wxListItem:new(), wxListItem:setId(It0, Index), wxListItem:setColumn(It0, 0), true = wxListCtrl:getItem(LC, It0), io:format("Row 0: ~p~n", [wxListItem:getId(It0)]), io:format("Col 0: ~p~n", [wxListItem:getColumn(It0)]), io:format("getText: ~p~n", [wxListItem:getText(It0)]), It1 = wxListItem:new(), wxListItem:setId(It1, Index), wxListItem:setColumn(It1, 1), true = wxListCtrl:getItem(LC, It1), io:format("Row 0: ~p~n", [wxListItem:getId(It1)]), io:format("Col 1: ~p~n", [wxListItem:getColumn(It1)]), io:format("getText: ~p~n", [wxListItem:getText(It1)]), {noreply, State}; %% Unknown event. %% handle_event(Ev = #wx{}, State) -> io:format("Got event:~p\n", [Ev]), {noreply, State}. handle_info(Msg, State) -> io:format("Got Info ~p\n", [Msg]), {noreply, State}. handle_call(shutdown, _From, State=#state{parent=Parent}) -> io:format("Shutdown~n", []), wxPanel:destroy(Parent), {stop, normal, ok, State}; handle_call(Msg, _From, State) -> io:format("Got Call ~p\n", [Msg]), {reply, {error, unknown_call}, State}. handle_cast(Msg, State) -> io:format("Got cast ~p~n",[Msg]), {noreply, State}. code_change(_, _, State) -> {stop, ignore, State}. terminate(_Reason, _State) -> io:format("Stopping~n", []), wx:destroy(), ok. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Sep 30 16:09:46 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 30 Sep 2014 16:09:46 +0200 Subject: [erlang-questions] Get grandchild pid when using nested supervisors? In-Reply-To: References: Message-ID: On Mon, Sep 29, 2014 at 6:10 PM, Roger Lipscombe wrote: > Calling supervisor:start_child(A, []) returns {ok, BPid}, where I > really want WPid. What's a sensible way to get hold of it? > One way, that is really useful is a process registry like gproc. That way, you can avoid a lot of this "digging for a Pid". It also is faster for lookup since it uses an ETS table, so you can avoid having the need to cache pids. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tauceti100@REDACTED Tue Sep 30 16:10:38 2014 From: tauceti100@REDACTED (a a) Date: Tue, 30 Sep 2014 18:10:38 +0400 Subject: [erlang-questions] The node (with release using ssl) does not shut down after calling init:stop/0 in Erlang 17.3 Message-ID: Hello, I use Erlang 17.3 downloaded from https://www.erlang-solutions.com/downloads/download-erlang-otp When I am trying to stop the ssl_hello_world example from cowboy I have encountered a strange behaviour. $ git clone https://github.com/ninenines/cowboy $ cd cowboy $ git checkout 1.0.0 $ cd examples/ssl_hello_world $ make $ ./_rel/ssl_hello_world_example/bin/ssl_hello_world_example console Erlang/OTP 17 [erts-6.2] [source-5c974be] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.2 (abort with ^G) (ssl_hello_world_example@REDACTED)1> application:which_applications(). [{ssl_hello_world,"Cowboy Hello World example with SSL.", "1"}, {ssl,"Erlang/OTP SSL application","5.3.6"}, {cowboy,"Small, fast, modular HTTP server.","1.0.0"}, {cowlib,"Support library for manipulating Web protocols.", "1.0.0"}, {public_key,"Public key infrastructure","0.22.1"}, {crypto,"CRYPTO","3.4.1"}, {ranch,"Socket acceptor pool for TCP protocols.","1.0.0"}, {asn1,"The Erlang ASN1 compiler version 3.0.2","3.0.2"}, {stdlib,"ERTS CXC 138 10","2.2"}, {kernel,"ERTS CXC 138 10","3.0.3"}] (ssl_hello_world_example@REDACTED)2> init:stop(). ok (ssl_hello_world_example@REDACTED)3> =ERROR REPORT==== 29-Sep-2014::16:49:15 === Error in process <0.69.0> on node 'ssl_hello_world_example@REDACTED' with exit value: {{case_clause,{error,closed}},[{ranch_acceptor,loop,3,[{file,"src/ranch_acceptor.erl"},{line,28}]}]} ... =ERROR REPORT==== 29-Sep-2014::16:49:15 === Error in process <0.164.0> on node 'ssl_hello_world_example@REDACTED' with exit value: {{case_clause,{error,closed}},[{ranch_acceptor,loop,3,[{file,"src/ranch_acceptor.erl"},{line,28}]}]} The listener socket has been closed. It seems the node is still running, but: (ssl_hello_world_example@REDACTED)3> application:which_applications(). ** exception exit: {timeout,{gen_server,call, [application_controller,which_applications]}} in function gen_server:call/2 (gen_server.erl, line 182) If I change this line https://github.com/ninenines/ranch/blob/1.0.0/src/ranch_acceptor.erl#L40 to {error, _Reason} -> ok the acceptor processes do not crash after calling init:stop/0, and the node shuts down correctly. This behavour appears only in Erlang 17.3 (I have tried it in Debian 7.6, Ubuntu 12.04.5, OS X 10.9.5). In Erlang R16B03 everything seems to work correctly. What else can I do to reveal the cause of this issue? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Tue Sep 30 16:35:31 2014 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 30 Sep 2014 16:35:31 +0200 Subject: [erlang-questions] The node (with release using ssl) does not shut down after calling init:stop/0 in Erlang 17.3 In-Reply-To: References: Message-ID: Hi! If you apply the following patch to 17.3 it might help. The SNI-contribution that we included in this release had missed the detail, that the server may optionally send a SNI-extension that must be empty. diff --git a/lib/ssl/src/ssl_handshake. erl b/lib/ssl/src/ssl_handshake.erl index 22673e4..eee33ef 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -1732,6 +1732,9 @@ dec_hello_extensions(<>, Acc) when Len == 0 -> + dec_hello_extensions(Rest, Acc#hello_extensions{sni = ""}); %% Server may send an empy SNI + dec_hello_extensions(<>, Acc) -> <> = ExtData, 2014-09-30 16:10 GMT+02:00 a a : > Hello, > > I use Erlang 17.3 downloaded from > https://www.erlang-solutions.com/downloads/download-erlang-otp > > When I am trying to stop the ssl_hello_world example from cowboy I have > encountered a strange behaviour. > > $ git clone https://github.com/ninenines/cowboy > $ cd cowboy > $ git checkout 1.0.0 > $ cd examples/ssl_hello_world > $ make > $ ./_rel/ssl_hello_world_example/bin/ssl_hello_world_example console > > Erlang/OTP 17 [erts-6.2] [source-5c974be] [64-bit] [smp:4:4] > [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V6.2 (abort with ^G) > (ssl_hello_world_example@REDACTED)1> application:which_applications(). > [{ssl_hello_world,"Cowboy Hello World example with SSL.", > "1"}, > {ssl,"Erlang/OTP SSL application","5.3.6"}, > {cowboy,"Small, fast, modular HTTP server.","1.0.0"}, > {cowlib,"Support library for manipulating Web protocols.", > "1.0.0"}, > {public_key,"Public key infrastructure","0.22.1"}, > {crypto,"CRYPTO","3.4.1"}, > {ranch,"Socket acceptor pool for TCP protocols.","1.0.0"}, > {asn1,"The Erlang ASN1 compiler version 3.0.2","3.0.2"}, > {stdlib,"ERTS CXC 138 10","2.2"}, > {kernel,"ERTS CXC 138 10","3.0.3"}] > (ssl_hello_world_example@REDACTED)2> init:stop(). > ok > (ssl_hello_world_example@REDACTED)3> > =ERROR REPORT==== 29-Sep-2014::16:49:15 === > Error in process <0.69.0> on node 'ssl_hello_world_example@REDACTED' > with exit value: > {{case_clause,{error,closed}},[{ranch_acceptor,loop,3,[{file,"src/ranch_acceptor.erl"},{line,28}]}]} > ... > =ERROR REPORT==== 29-Sep-2014::16:49:15 === > Error in process <0.164.0> on node 'ssl_hello_world_example@REDACTED' > with exit value: > {{case_clause,{error,closed}},[{ranch_acceptor,loop,3,[{file,"src/ranch_acceptor.erl"},{line,28}]}]} > > The listener socket has been closed. > It seems the node is still running, but: > > (ssl_hello_world_example@REDACTED)3> application:which_applications(). > ** exception exit: {timeout,{gen_server,call, > > [application_controller,which_applications]}} > in function gen_server:call/2 (gen_server.erl, line 182) > > If I change this line > https://github.com/ninenines/ranch/blob/1.0.0/src/ranch_acceptor.erl#L40 > to > {error, _Reason} -> ok > the acceptor processes do not crash after calling init:stop/0, and the > node shuts down correctly. > > This behavour appears only in Erlang 17.3 (I have tried it in Debian 7.6, > Ubuntu 12.04.5, OS X 10.9.5). In Erlang R16B03 everything seems to work > correctly. > > What else can I do to reveal the cause of this issue? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Tue Sep 30 17:46:39 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 30 Sep 2014 11:46:39 -0400 Subject: [erlang-questions] Get grandchild pid when using nested supervisors? In-Reply-To: References: Message-ID: <20140930154637.GA3743@ferdair.local> Rather than going that way there's the option of passing a callback function, say: Target = self(), WhenReady = fun() -> Target ! {self(), ready} end, {ok, _SupPid} = start_child_chain_think(M, F, Args ++ [WhenReady]), receive {ChildPid, ready}?-> ... end. which at this point lets you get something without assumptions of the supervision structure. On the other hand, the child in Y now has to know it has to call a given callback to report to a parent when it's ready. Regards, Fred. On 09/30, Vance Shipley wrote: > On Mon, Sep 29, 2014 at 9:40 PM, Roger Lipscombe > wrote: > > > > Calling supervisor:start_child(A, []) returns {ok, BPid}, where I > > really want WPid. What's a sensible way to get hold of it? > > > > {ok, BPid} = supervisor:start_child(A, []), > [{_, WPid, _, _}] = supervisor:which_children(BPid) > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mononcqc@REDACTED Tue Sep 30 17:58:05 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 30 Sep 2014 11:58:05 -0400 Subject: [erlang-questions] Pid after node restart In-Reply-To: References: Message-ID: <20140930155804.GB3743@ferdair.local> On 09/29, Pavel Baturko wrote: > My case is: > I have 2 connected nodes. > > [snip] > > Could someone explain this behavior? > What is changing in internal pid representation after node restart so > comparison in step 10 returns false? And why comparison in step 11 returns > true? The visual pid representation you see isn't the full thing. Interesting for this is to look at the external term format: http://erlang.org/doc/apps/erts/erl_ext_dist.html#id88514 1 N 4 4 1 103 Node ID Serial Creation Encode a process identifier object (obtained from spawn/3 or friends). The ID and Creation fields works just like in REFERENCE_EXT, while the Serial field is used to improve safety. In ID, only 15 bits are significant; the rest should be 0. [..] Creation is a byte containing a node serial number that makes it possible to separate old (crashed) nodes from a new one. What this tells us is that each pid you send around contains the full node name (internally you may have a hash, which is X in ). Locally, 'X' is always printed as '0' to be easy to recognize. Y and Z are then two counters you have in there, and hidden in it is the 'Creation' Looking at your list: > > 1. On node 1 I generate a pid P (<0.X.0>) > 2. P is transfered to node 2 (on node 2 I receive it as ) > 3. P is transfered back from node 2 to node 1 (I'm receiving it as P2) > 4. P2 is recognized as local pid (<0.X.0>) This is covered by the node ID and local/remote printing of pids > 5. I can compare P and P2 on node 1 and they are identical Covered by internal pid structure. > 6. I'm restarting node 1 > 7. On node 1 I generate the same pid P as in step 1 (<0.X.0>) The Creation id is likely set propelry there. > 8. I repeat step 3, again receive initial pid as P2 > 9. P2 is recognized as local pid (<0.X.0>) - as before > 10. But now P == P2 returns false. > 11. Also P == list_to_pid(pid_to_list(P2)) returns true The creation byte is likely what's at play there. You can term_to_binary(P) and term_to_binary(P2) to confirm the difference in there. list_to_pid does some fancy pants evaluation, but term_to_binary and binary_to_term will do the full form lossless serialization. Regards, Fred.